Category Archives: Software Business

Making Sausage and Delivering Sausage

There’s an article about DevEx, a new developer productivity methodology, in ACM Queue. If you subscribe to the Pragmatic Engineer newsletter, there was an interview with the article’s authors last week. This is the latest methodology from the people behind DORA and SPACE.

DORA’s measurements were grounded in externally visible outcomes.

Deployment Frequency
Mean Time to Recovery
Change Failure Rate
Lead Time

The idea was to pick things that engineers could actually control.. Even though the elements of DORA are not directly translatable to business outcomes, they are still understandable to external stakeholders.

In SPACE, these metrics are still one kind that we collect, but SPACE also recognizes that there are other things besides Performance and Activity metrics (the P and A of SPACE). It also considers Satisfaction, Communication, and Efficiency, which are more internal to the team.

In DevEx, the emphasis is on internal metrics: Flow, Cognitive Load, and Feedback Loops.

I want to say upfront that I completely agree that these things do help engineers deliver software better and faster. But they are hard to share outside of the team. It’s how the sausage is made. The business ultimately needs to deliver sausage.

Aside from the rest of the business not understanding or caring about these metrics, I also worry that they will try to get too involved in them. Engineering leadership should care a great deal about the cognitive load of the members of their teams, and should work to lower it, but they need to find a better way to express that outside of engineering if they do.

I know the DevEx authors know this, and emphasis on these “making sausage” metrics doesn’t mean that they don’t also think externally visible performance isn’t important (they did after all design DORA and SPACE). But if you deliver on, for example, long flow states, but there isn’t more useful software on servers, you have failed in the business objective. This is the same thing I said about Story Points—they are too far removed from things people outside of engineering care about:

[…] regular people just translate [story points] to some notion of time anyway, and in that regard, I am very regular. If you are going to take some random number I provide and assign a date to it, I’d much rather I do the translation for you.

To the extent that you report directly on DevEx, try to emphasize the parts outsiders can help with. Frequency of meetings and speed of external feedback loops (especially from product management) are good examples of that.

Large Language Models are a Sustaining Innovation

In the Innovator’s Dilemma, Clay Christensen made a distinction between “disruptive” and “sustaining” innovations. Disruptive innovations favor new entrants and new categories of products because incumbents are harmed by adopting the innovation, and so they resist. Sustaining innovations favor incumbents because it improves their products and margins, and they have the resources and incentives to adopt them.

With this in mind, I think that Large Language Models will be readily adopted by incumbents who will be successful with them. To be clear, I’m not talking about OpenAI vs. Google, but their customers, mostly B2B SaaS companies who will be using LLMs to enhance their own software, not providing LLMs to others.

There are two advantages that incumbents have that will hard to overcome.

The first is that LLMs readily embed into established software. GitHub Copilot is the model. The “copilot” experience is being extended to Microsoft’s Office suite, and I think it fits well in almost any kind of software.

The second advantage is access to proprietary data. Incumbents already have customers and their data and can generate better content using that data in their training sets. A new entrant would be stuck with just public sources which is “ok” for some categories, but in the long tail of B2B SaaS companies would be anemic at best.

This is playing out for VSCode right now. Microsoft controls proprietary data (the private code in GitHub) and has the best content creating software. Their first iteration of enhancing that with LLMs is just a better VSCode. I use VSCode every day and adopting GitHub Copilot was seamless. It took seconds to install, see the benefit, and give over my credit card.

The case for a disruptive innovation is easier to make with things like Webflow, which obsolete the editor and establish a new proprietary datasource (their customers’ projects). This might happen to VSCode, but not to Microsoft, since it has its own no-code solutions (the Power Platform). So even this might not be disruptive.

Magnets for Innovation Needles in a Haystack

Each department in a business collects and spreads performance information throughout their organization. They know their own numbers well and are incentivized to improve them. For successful businesses, all of that data is telling them to keep doing what they are doing.

But, new kinds of information aren’t as easy to collect and spread, which is a problem because new information drives innovation.

The beginning of a new market regime, where your products don’t match the market, starts out insignificantly small. You might have a customer satisfaction of 97%, but the innovation might be living in that 3%. Or it might be pointing to an uninteresting niche that you should ignore. It’s hard to know if outliers are noise or a new signal.

If this problem is like finding a needle in a haystack, then we need a metal detector or, even better, a magnet.

In Noticing Opportunities Using an AI Agent I wrote that AI’s ability to consume and synthesize a lot of disparate information can be used to bring opportunities to you. I gave two examples of when I did it consciously and unconsciously in job searches.

For this to work, the organization needs to pour a lot of its internal data into a training set. They should start with sales conversation transcripts and customer support tickets. If there are any user communities, all of their posts should be put in as well. If their competition has public forums, bring those in. Each interaction in the dataset may be an anecdote, but if you have enough of them, at some point you need to accept that there are clusters worth exploring.

Humans still need vet what the AI finds, so I’d try to use this to generate experiments that can validate (or invalidate) a hypothesis.

One thing you’ll want to know is how big the potential new market is. Since these might not be your customers yet, the experiment might need to a content strategy rather than a product one.

It’s fitting then, that the concept of a Lead Magnet already exists. The normal use is to get a lead to start a sales process with. For an innovation, use lead magnets to research and validate a new market.

Sell Your Learning Plan, Not Your Roadmap

The reason software companies adopted agile practices was so that they could adapt their roadmaps to whatever they learned along the way.

If you learn something and do not change the roadmap, then there was no point to learning it. You might as well be doing a waterfall process. I think everyone accepts this. After all, waterfall is a myth. No one really thinks that the roadmap is set in stone.

But the business needs to plan. Marketing has events that need lead time to develop messages and material. Sales has prospects that want to know where the product is headed. Support, Training, and the Documentation team need time to get ready for new releases.

Some of this can be solved by developing support material alongside product development. For example, an embedded Product Marketing Manager can be closely following the trajectory of the roadmap and be developing messages at the same time. Even better, they can be helping to shape the roadmap from insights they get from their customer engagement.

The hard part is what to do with prospects. If you’re Apple, you never talk about what’s coming until you are ready to sell, but most of us can’t do that. Our prospects will certainly have unmet needs, and if you know you are actively working on them, it’s tough not to tell them. But, what should you say?

The truth is you don’t know the roadmap beyond the next few sprints. You are actively hoping that you learn something that will change it. New information is highly correlated with innovation.

But you are not changing your mission as easily. You know which customers you want to serve and what jobs of theirs you want to help with. So, talk about that and not the specific features you currently think will help. And talk about what you do know is coming (if you want). Even in a learning organization, the short-term roadmap should be pretty solid.

Product Development might not be able to commit to exact features, but they should commit to a learning plan, and that’s the roadmap that is safer to talk about.

Ad Attribution comes to iOS apps and the App Store

I have not heard a lot of hoopla about this, but I think this is a big deal. Back in 2019, WebKit announced a way to do ad attribution without per-user tracking. I put support for this in the AppStore on my WWDC 2019 wishlist.

The WebKit team just announced a privacy preserving ad attribution system for websites. I want the same thing to work for an ad for an app that appears on the web, App Store, or in another App. No need to attribute to a user — just the ad source is sufficient.

I explained in a followup:

The last few years have been a cat-and-mouse game between Ad Tech firms finding loopholes in iOS (e.g. shared Pasteboards) and Apple closing them. It would be much better if Apple provided a privacy preserving mechanism and then explicitly forbade anything else in the developer agreement.

I put this on my 2021 wishlist as well.

I just noticed this video on the WWDC Site: Meet privacy-preserving ad attribution, which appears to have done just that.

I don’t personally need this feature, but we need to do something to help out app developers who are under pressure to get some kind of ad effectiveness data. With IDFA basically gone, Ad Tech firms are going to resort to fabricating their own ID from whatever identifying information they can grab.

One thing to remember is that none of these firms have any kind of relationship with Apple. They are not parties to the developer agreement, so they have no obligation to follow it. It’s the developers that integrate their SDKs that take the risk and are the ones violating the agreement.

Another risk is that these SDKs inherit all of the permissions their host app has obtained. So, you could ask for location for a very good reason (e.g. you show the weather) and they could see that they have the permission and use it for something else (e.g. enriching their ad targeting database). Again, your app probably didn’t disclose this, but those SDKs don’t need to follow the rules—only you do.

So, I’m looking forward to this ad attribution method being adopted widely and for app integration to be done by the app developer just making HTTPS calls, not integrating an SDK. It may be too much to hope for, but it did require Apple to take the first step and offer a mechanism, which they now have.

Design by Comparing Opposites

In my podcast episode, Write While True Episode 7: Find Your Voice, I tried to express an idea that I find is useful in a lot of contexts. There are some choices where you are trying to make something specific or differentiating—in those cases, one way to know that you’ve done it is to see if the opposite choice is also reasonable.

In this case, I was talking about trying to find my “voice” in the podcast—what makes it mine and not generic. I recommended Joanna Wiebe’s video on voice and tone.

But, I cautioned against a voice that was “smart”. I said:

I actually don’t think smart is a good choice because the opposite of smart is very rarely appropriate. Everyone would choose to sound smart.

When you have a generically positive aspect that you want (like smart), a useful technique is to consider opposing ways to achieve it.

I recommended considering the choices of “expert” or “fellow learner” instead of smart. Both voices might be smart, but they are specific, opposite, and both are reasonable.

This is also a good technique in job seeking. It’s easier to find what you want if it’s not something that everyone would claim to have.

Sending a clear signal that “this is not for you” is the only way the people that it is for will recognize it.

Waiting to Promote Until You Are Sure is Wage-Theft

If you wait to promote people who are “already doing the job”, I think you’ll find that you don’t retain your best people. And if you are hiring people into senior positions instead of promoting, you don’t deserve to.

Even if you do, it’s a kind of wage-theft to not pay people for the work they are doing. A few months is ok, a year is absolutely not if you have the positions. If you’ve already done this, consider a bonus at promotion time.

If you have the positions, but people don’t seem ready, that’s on you. You should have supported and developed them before they were needed. If the opportunity is there, you should take the chance and responsibility for their success.

Meaning, if it doesn’t work out, you were the failure. You failed to develop them in time, and you failed to support them so that they were successful.

Applying The ONE Thing

I recently read The One Thing by Keller and Papasan. I recommend it, but the current VP of The ONE Thing was on an episode of Afford Anything, and it’s a very good introduction to the concept, if you want something quicker to digest.

The crux is that the authors ask you to focus on one question:

What’s the ONE Thing you can do such that by doing it everything else will be easier or unnecessary?

In a “five-why” fashion, you could start with your big goals and work backwards until you have the very first thing you need to do. They recommend dedicating a huge chunk of your time to that one thing, accomplishing it, and then moving on to the next ONE thing … ONE thing at a time.

For example, you might start with a goal to “Lose 10 pounds”, but end up with a ONE Thing like — learn how to cook five healthy, low calorie density, one-pot, bulk meals.

Don’t stop there, though. They want you to keep going—what is the one thing that would make that easier? Maybe you can find a local chef to come to your house and make them with you for the first three weeks. Perhaps you can get four friends to each learn one and teach the others (or swap freezer bags each week).

When I finished the book, I thought that my current ONE thing was to build up the number of tutorials on App-o-Mat, so I have been doing that. The eventual goal with App-o-Mat is to sell some kind of product there, but before that I need an audience, and you can’t get an audience unless you have useful content.

But, I am supposed to keep digging.

What is the The ONE Thing I could do to make making tutorials on App-o-Mat easier. One thing that has helped is having Sprint-o-Mat, because I just take code out of it as a basis for the tutorial. If I didn’t have that, making the app might be important.

But, if I break it down to something like this: I want 100 more articles on App-o-Mat by the end of July, then I see that I can only write about 25 of them at my current pace. What would make that easier?

Obviously, more writers. But 75 articles would cost about $15,000. So, is the real next ONE thing to get $15,000 of revenue? Or at least $5,000 in May?

I’m still working through this. If App-o-Mat were a serious business, investing $15,000 would be a no-brainer, but honestly, it’s not. Right now, I want to have the experience of writing, and I don’t want to manage and edit a bunch of writers.

But, I do think that a trickle of articles by others would be ok, so I will explore that. I also think that if I doubled-down on the time I spend writing articles, I could do a lot more, which means dropping some other things or just getting more disciplined.

Keller would say to break down “getting more disciplined” first because he’s kind of annoying that way.

So, to do that, I am going to make my One thing to come up with article series ideas that could each have many articles in them. Because then, I’d at least have a checklist I could work from and could put them on a schedule. I think I’d see the possibility of outsourcing better as well.

Tech Debt Happens to You

In the original ANSI C as described by K&R, there are a bunch of library functions that use internal state static variables. An example is strtok, which you call with a string to tokenize, and when you want the next token, you call it with NULL. strtok uses internal static variables to keep track of the string and an iterator in it.

In early C usage, this was fine. You had to hope that any 3rd party library call you made while iterating tokens weren’t also using strtok.

But, then when threads were introduced to UNIX and C, this broke down fast. Now, your algorithms couldn’t live in background threads if they used strtok. This specific problem was solved with thread-local variables, but the pervasive use of global state inside of C-functions was a constant source of issues in the most multi-threading and post multi-processor world.

And the world was changing from delivering desktop apps to web apps, so now a lot of your code lived in a multi-threaded back-end that serviced simultaneous requests. This was a problem because in early web development, we took C-libraries out of our desktop apps and made them work in CGI executables or NSAPI/ISAPI web-server extensions (similar to Apache mod_ extensions)

To make this work, we had to use third-party memory allocation libraries because the standard malloc/free/new/delete implementations slowed down as you added more processors (from constant lock contention). Standard reference-counting implementations used normal ++ and -- which aren’t thread-safe, and so we needed to buy a source code implementation of stl that we could alter to use InterlockedIncrement/InterlockedDecrement (which are atomic, lock-free, and thread-safe).

As the world changed around us, we could keep moving forward with these tech-debt payments.

Also, this was slow-paced problem—strtok/malloc/etc were written in the 70s and limped through the 90s. That’s actually not that bad.

But, the world doesn’t stop. Pretty soon, it was just too weird to implement back-ends as ISAPI extensions. So, you pick Java/SOAP because CORBA is just nuts, and well, that’s wrong because REST deprecates that, and then GraphQL deprecates that, and you picked Java, but were you supposed to wait for node/npm? Never mind what’s going on on the front-end as JS and CSS frameworks replace each other every 6 months. Even if you are happy with your choice, are you keeping your dependencies up to date, even through the major revisions that don’t follow Substitutable Versioning?

And I think that this is the main source of tech debt, not intentional debt that you take on or the debt you accumulate from cutting corners due to time constraints. The debt that comes with dependency and environment changes.

Being able to bring code into your project or build on a framework is probably the only thing that makes modern programming possible, but like mortgages, they come with constant interest payments and a looming balloon payment at some point.

There are some you can’t avoid, like the OS, language, and probably database, but as you go down the dependency list, remember to factor in the debt they inevitably bring with them.

Mitigating the Mitigations

The first step to dealing with project risk is to list mitigations—the steps you will take to avoid or handle the risk. But, don’t stop at just listing them.

The very next step should be to think about whether those mitigations themselves have obstacles.

For example, let’s say you have a project with a large mechanical portion, which doesn’t require communication or coordination to implement. If this work can be done in parallel, then adding engineers mid-way (once the a core has been established) could speed up the project, but even at the start, you must

Get budget approval
Find a source of engineers (contractors/internal)
Structure the work so that at some point it can be done without coordination and in parallel.

Projects like converting a lot of code from Objective-C to Swift or from an unsupported version of a framework to the current one work well for this. Projects that require a lot of cross-developer coordination are famously made later by adding engineers.

Depending on the stakes, it might be right to ask engineers to temporarily put in more hours, or to schedule time-off earlier in the project if possible. These things are easier to do if you talk about them earlier and gauge the impact.

A third option would be to clear off all other work so that everyone can concentrate on the high-priority work at hand. In that case, those projects have risks that need to be communicated to the project managers so that they can list and mitigate them.

It would be frustrating for a project risk to manifest itself, to have identified that early, and to have come up with an effective mitigation, which is not possible to do because it needed prior planning.