I estimate using time, not points. I do this even though I have mostly worked on product development teams inside of companies and not as a contractor for clients.

But, estimation on product development is not as high-stakes an activity as it is if you are billing based on it. The judgement of a successful product development project is based on whether it moves some business needle, not whether it was under budget. The margin on successful product development dwarfs reasonable overruns. There is also a lot of appetite for risk, where we know that we’ll win some and lose some.

That’s not true if the work is for clients. If they pay by the hour, they expect the estimate to be accurate. But there are very few (if any) big software projects that can be accurately estimated. When you’re wrong, you either have to eat it or have a very difficult conversation.

I don’t have a good answer, but I would start with triangle (or three-point) estimates. To use them, you estimate a most-likely estimate and then a best and worst case on either side. The three numbers describe a distribution (see the wikipedia page for how to combine them). The result is an estimate with a error range and a confidence interval. I would recommend sharing that instead of a single number.

The only task management system that I have seen that offers this option is DevStride, which carries three-point estimates throughout its system.

In early computing history, the government, academia, and industry collaborated to create languages for mainframes. The enduring ones were COBOL and Fortran.

You’d think that the PC era, when computing was delivering doublings in power every couple of years that it would also have been a time of programming language innovation. Aside from Microsoft, in the 70’s-2000’s, most of the PC companies didn’t do much here. IBM, Intel, AMD, Apple, Dell, HP, Sony, and Compaq made all the hardware, but contributed nothing to programming languages. Sun is the only hardware company that bucked this trend.

The photocopier, with Smalltalk, did more than the PC to drive programming language design than the PC. Fifty years of OO dominance followed.

But, for driving programming language development, nothing beats phones.

Telephone profits powered Bell Labs, which invented a lot of important tech, but some of the most enduring are the programming languages they developed. To limit it to just the ones I have used professionally, there are C, C++, Awk, KSH and the document processing languages troff, pic, and eqn. I’m probably leaving out about a dozen more.

AT&T was the most prolific converter of phone calls to programming languages, but Ericsson created Erlang, and Apple’s iPhone profits drove them to create Swift. You can give most of the credit for Kotlin to Android.

In the end, it’s probably money that does most of the work, and the real driver seems to be when it coalesces to a single winner.

A few days ago, I wrote about using Errors Per Million (EPM) instead of success rate to get better intuition on reliability. I also recently said that Visualizations Should Generate Actions. Sometimes it’s obvious what to do, but if not, you can think through the scenarios and pre-define what actions you would take.

Here’s an example. This is a mock up of what a dashboard showing EPM over time might look like. The blue line is the EPM value on a date:

The three horizontal lines set levels of acceptability. Between Green and Yellow is excellent, between Yellow and Red is acceptable, and above Red is unacceptable. When we did this, we thought about using numbered severity levels (like in the Atlassian incident response playbook), but we decided to use Green/Yellow/Red for simplicity and intuition.

We also pre-defined the response you should have at each level. It was something like this:

The advantage of this was that these actions were all pre-negotiated with management and product managers. This meant that we could just go ahead and fix things (at a certain level) instead of items getting lost in the backlog.

When we created this dashboard, we were in the Red, but we knew that going in. We worked to get ourselves Green and in practice, we were rarely not Green. This is another reason to pre-define your response, as it becomes too hard to remember how to handle situations that rarely happen.

Yesterday, I shared a heatmap visualization that I used to target manual testing time. I chose a heatmap to show this data because you can tell what you need to do just by looking at it.

In this example

It’s pretty clear that you should get an iPhone 13 with iOS 15 on it and start testing everything. You could also explore board creation on all devices. If the entire heatmap were green, you would know that you had probably covered most areas.

It would be easy to write a program that took this same data and generated a to-do list instead. Maybe that would be preferable, but people like visual dashboards, and it’s easier to see the why behind the task if you have a sense of the underlying data.

But, that’s a big clue to whether your dashboard visualization works. If you could easily generate a to-do list just by looking at it, then it probably works. If you look at your dashboard and have no response, it might look pretty, but it’s not doing its job.

At Trello, I built a simple visualization for understanding coverage of our app during Beta periods. We used Mode to analyze data, and so I used their Heatmap.

Here’s a recreation in Google Sheets:

Along the top was each device family and OS. Individual devices were grouped based on how likely they were to be similar in testing (based on size, version, etc). I used this list of Apple device codes (which were logged with analytic data).

Along the left side were the most important screens and features. It was a much longer list that was generated from analytic categories.

The center of the visualization was a heat map based on how much usage that feature got on that device (at the cross-section) normalized against how much usage it got in production. So, if a cell was green, it meant that it was tested a lot when compared to how much it was used in production. If a cell was red, it meant it was under tested.

Often, entire vertical columns would be near red because the combination of device/OS wasn’t used much by our beta testers. So, we could direct our own efforts towards those devices and turn an entire column from red to green.

We also made sure new features would get their own row. These could also be red because beta testers might not know about them. We could similarly target those areas on all devices. These features could not be normalized against production usage (since they were not in production yet), so we use a baseline usage as a default.

Mode kept snapshots of the heatmaps over time. We could watch it go from nearly all red at the beginning of the beta period to more green by the end. I can’t say we could get the entire heatmap to be green, but we could at least make sure we were testing efficiently.