Large Language Models are a Sustaining Innovation

In the Innovator’s Dilemma [amazon affiliate link], Clay Christensen made a distinction between “disruptive” and “sustaining” innovations. Disruptive innovations favor new entrants and new categories of products because incumbents are harmed by adopting the innovation, and so they resist. Sustaining innovations favor incumbents because it improves their products and margins, and they have the resources and incentives to adopt them.

With this in mind, I think that Large Language Models will be readily adopted by incumbents who will be successful with them. To be clear, I’m not talking about OpenAI vs. Google, but their customers, mostly B2B SaaS companies who will be using LLMs to enhance their own software, not providing LLMs to others.

There are two advantages that incumbents have that will hard to overcome.

The first is that LLMs readily embed into established software. GitHub Copilot is the model. The “copilot” experience is being extended to Microsoft’s Office suite, and I think it fits well in almost any kind of software.

The second advantage is access to proprietary data. Incumbents already have customers and their data and can generate better content using that data in their training sets. A new entrant would be stuck with just public sources which is “ok” for some categories, but in the long tail of B2B SaaS companies would be anemic at best.

This is playing out for VSCode right now. Microsoft controls proprietary data (the private code in GitHub) and has the best content creating software. Their first iteration of enhancing that with LLMs is just a better VSCode. I use VSCode every day and adopting GitHub Copilot was seamless. It took seconds to install, see the benefit, and give over my credit card.

The case for a disruptive innovation is easier to make with things like Webflow, which obsolete the editor and establish a new proprietary datasource (their customers’ projects). This might happen to VSCode, but not to Microsoft, since it has its own no-code solutions (the Power Platform). So even this might not be disruptive.

Magnets for Innovation Needles in a Haystack

Each department in a business collects and spreads performance information throughout their organization. They know their own numbers well and are incentivized to improve them. For successful businesses, all of that data is telling them to keep doing what they are doing.

But, new kinds of information aren’t as easy to collect and spread, which is a problem because new information drives innovation.

The beginning of a new market regime, where your products don’t match the market, starts out insignificantly small. You might have a customer satisfaction of 97%, but the innovation might be living in that 3%. Or it might be pointing to an uninteresting niche that you should ignore. It’s hard to know if outliers are noise or a new signal.

If this problem is like finding a needle in a haystack, then we need a metal detector or, even better, a magnet.

In Noticing Opportunities Using an AI Agent I wrote that AI’s ability to consume and synthesize a lot of disparate information can be used to bring opportunities to you. I gave two examples of when I did it consciously and unconsciously in job searches.

For this to work, the organization needs to pour a lot of its internal data into a training set. They should start with sales conversation transcripts and customer support tickets. If there are any user communities, all of their posts should be put in as well. If their competition has public forums, bring those in. Each interaction in the dataset may be an anecdote, but if you have enough of them, at some point you need to accept that there are clusters worth exploring.

Humans still need vet what the AI finds, so I’d try to use this to generate experiments that can validate (or invalidate) a hypothesis.

One thing you’ll want to know is how big the potential new market is. Since these might not be your customers yet, the experiment might need to a content strategy rather than a product one.

It’s fitting then, that the concept of a Lead Magnet already exists. The normal use is to get a lead to start a sales process with. For an innovation, use lead magnets to research and validate a new market.

Look for Ways to Make Your Job Application More Specific

A couple of days ago I wrote about how my most popular blog post (the UML Cheatsheet) was based on a talk I did at a local .NET group. I could not have predicted that it would end up being the most popular thing on this site.

I did predict what the immediate reaction to my talk would be because I had planned for that.

I did the talk because I was trying to get a job at the company (Atalasoft) that hosted the group. After my presentation, I got an impromptu interview with the owner and the Chief Architect (and was offered the job a couple of weeks later).

It’s a lot of work for a single job application. It doesn’t scale, which is the point. But, their job description was perfect for me, so I was willing to go all out. Doing that let them know that I thought we were a good match.

This is an example of what I was getting at when I wrote that your resumé and your cover letter should make no sense. They should be so specific to the intended recipient that no one else would understand them. In this case, I saw a chance to make it even more specific and took it.

Write While True Episode 17: Make Art with Friends

I thought about things that were going well, where I had a lot of output, like my programming projects. One thing I realized that helps a lot is that I am proud of that work. Doing it is a self-esteem builder, and I have a lot of confidence that I can meet my own standards.

I’ve been programming for a long time, so this isn’t much of a surprise. All during that time I challenged myself by doing bigger projects and learning new things.

While learning new things and working on bigger and bigger projects helped me improve, it wasn’t the most important factor.

What helped me the most was the people I worked with.

Transcript

Blog Posts, Randomness, and Optionality

I started this blog in 2003. I have some favorites and ones that I think are worth reading. But I would never have guessed which one would be the most read.

I have a few posts that are probably one of the best places to learn about a specific technical problem, and Google sends people to them. For example, Understanding EXC_BAD_ACCESS clears a lot of misconceptions about what this error means. It’s popular, but not the most popular.

I have a few posts that are jokey movie reviews, where I take one technical aspect and just review that part (it’s based on a Letterman bit). In my review of Oz (the James Franco prequel to the Wizard of Oz), I tried to figure out how the Wizard’s projection technology might have worked (given 19th century constraints). This one is the most popular of the movie reviews because people keep searching for “wizard of oz machine“.

But, by far, the most popular page on this site is the one that hosts my UML cheatsheet. It dominates my search traffic. If I had nothing else on this site, my analytics wouldn’t even notice.

I wrote that post in 2006 based on a talk I gave to a local .NET users group. I didn’t know C# or any .NET yet, but I contributed what I could, UML sketching, which is applicable to any OO language.

There is no way I could have predicted that it would be my most popular post beforehand, which is part of what I mean when I say in Randomness is the Great Creator, that “I believe that the universe is a random, unknowable thing that offers infinite variety. We have an opportunity to tap into it with contributions to the randomness”. It’s why I put these posts out there.

It’s also related to the lesson I learned on the importance of optionality. One of the reasons to collect options is that you are positively exposed to randomness. In this sense, each blog post is an option. They have a low fixed cost to make, but each has a tiny chance at infinite upside.

Or at least some.

It’s also a signal of a direction that might be fruitful, which I’ll explore soon.

Make Art with Friends

I do an annual retrospective in December of every year and try to come up with a theme for the next year. I decided that I wanted to make more “art” in 2023, where “art” is loosely defined, but includes code, blog posts, podcasts, t-shirts, and sketches.

A sketch of a small wooden figure with a wooden do nothing machine (in front of the actual figure and machine)

I knew that this theme would not be good enough for me to actually do it. 2022 was a year of relative low non-code output for me.

I reflected on why coding had been relatively easy for me to do and working on my other interests was not. I realized that I had mostly coded in collaborations, but did almost everything else alone.

I set out to change that.

I decided to combine the goal of being more prolific with a goal to collaborate more and wrote a phrase on the first page of my 2023 journal: “Make Art with Friends”.

With that theme in place, I sought out like-minded groups. That’s how I found Toastmasters, which is an organization where people practice public speaking together.

I’m using the Toastmaster projects to drive my writing, and I’m hoping that practicing them will help me sound more natural and extemporaneous. It’s helping. I used my most recent speech to help restart my podcast and released Episode 16 of Write While True a few days ago. Next week’s episode is built from a different speech.

I am trying to play more infinite games, which are games you play for the enjoyment of playing them, like a game of catch. At Toastmasters, we are constantly throwing the “floor” to others to give them a chance to play.

It’s just more fun this way, and that’s what’s making the difference for me right now.

Write While True Episode 16: Try and a Teacher Will Appear

I did season one of this podcast a couple of years ago.

I did 15 episodes. They were about some of the basics of building up a writing habit, and then I stopped. In this season, I am going to explore how to restart projects in the context of restarting this one.

I want to talk about one of the lessons I’ve learned.

Transcript

More on Copilot for Learners

Yesterday, I wrote that you shouldn’t use GitHub Copilot if you are a new programmer:

But, if you are still learning to program, then I am worried that Copilot would inhibit that. The main way it does this is by not letting you practice remembering.

I just saw a paper called: GitHub Copilot AI pair programmer: Asset or Liability? that studied this issue and concludes:

Based on our findings, if Copilot is used by expert developers in software projects, it can become an asset since its suggestions could be comparable to humans’ contributions in terms of quality. However, Copilot can become a liability if it is used by novice developers who may fail to filter its buggy or non-optimal solutions due to a lack of expertise.

So, check that paper out if you want to see actual data supporting this belief. One of the core findings is that Copilot generates wrong code that is easy for experts to fix, which was my intuition based on using it.

Use GitHub Copilot When You Aren’t Learning

I’m trying to wrap my head around whether I think junior developers should be using GitHub Copilot or something similar. I am thoroughly convinced that experts that are mostly concerned with their output and productivity, and not with learning, should be using it. The more your coding is a performance of your abilities, the more I think you should use it.

But, if you are still learning to program, then I am worried that Copilot would inhibit that. The main way it does this is by not letting you practice remembering.

According to Make it Stick, the key to learning something is to practice remembering it. One way to do this is to try to solve problems without help and learning code by writing code is a great way to do this. But, if Copilot is constantly helping you, then even coding novel programs becomes the kind of puppetting that tutorials do.

Now, it may turn out that composing code completely from scratch is a dying skill and won’t be the way we program in the (near) future. I use Copilot exclusively, and I certainly feel like I’m guiding/collaborating with the AI, and coding feels very different to me now. But, I also think that my skill at seeing code is helping me do that. That skill was developed by writing and reading lots of code.

So, right now, in April 2023, if you are learning to code, I’d limit usage of Copilot to the times when you are not learning. If you are building something that is well within your ability, then I would use it wholeheartedly. There are Copilot skills to develop on top of your coding skill.

Mental Representations in Coding

I recently reread Peak [amazon affiliate link] by Anders Ericsson, which is the source of the term “Deliberate Practice” that was popularized by Malcom Gladwell’s “10,000 hours”. The truth is more complicated, and I highly recommend reading both Peak and Kathy Sierra’s Badass [amazon affiliate link], which deep dives on the practicalities of gaining expertise.

I wrote some thoughts on applying deliberate practice to learning how to code a couple of years ago. Looking that over, I was very focussed on what I thought you should do, which was to turn vague descriptions into code (not transcribe code tutorials). I had not remembered another important point, though. One purpose of the training is to help build a mental representation that makes it easier to do the task.

One of the mental representations I have is that UI layouts are primarily about nested rectangles and the relationships of their sizes and positions to each other.

If I were learning to code HTML/CSS now, I would try to learn this first and only make pages of different colored rectangles in common layouts. I would try to limit CSS to just the layout attributes and the HTML to just <div> and maybe just enough to see the layout.

Next, I would add the concept of invisible rectangles that are just helping with the relationships.

Next, we could explore media query breakpoints and other responsive features of web coding.

Only then, would I attempt to make “real” pages. And, I would start with getting the layout done first.

The key is to identify the mental representation that helps to do the harder task and to break it down to a simple task that you drill over and over to make it automatic.