Category Archives: Software Development

Visualizations Should Generate Actions

Yesterday, I shared a heatmap visualization that I used to target manual testing time. I chose a heatmap to show this data because you can tell what you need to do just by looking at it.

In this example

A heat map the test status of iOS devices across different features in an app

It’s pretty clear that you should get an iPhone 13 with iOS 15 on it and start testing everything. You could also explore board creation on all devices. If the entire heatmap were green, you would know that you had probably covered most areas.

It would be easy to write a program that took this same data and generated a to-do list instead. Maybe that would be preferable, but people like visual dashboards, and it’s easier to see the why behind the task if you have a sense of the underlying data.

But, that’s a big clue to whether your dashboard visualization works. If you could easily generate a to-do list just by looking at it, then it probably works. If you look at your dashboard and have no response, it might look pretty, but it’s not doing its job.

Use Heatmaps for iOS Beta Test Coverage

At Trello, I built a simple visualization for understanding coverage of our app during Beta periods. We used Mode to analyze data, and so I used their Heatmap.

Here’s a recreation in Google Sheets:

Along the top was each device family and OS. Individual devices were grouped based on how likely they were to be similar in testing (based on size, version, etc). I used this list of Apple device codes (which were logged with analytic data).

Along the left side were the most important screens and features. It was a much longer list that was generated from analytic categories.

The center of the visualization was a heat map based on how much usage that feature got on that device (at the cross-section) normalized against how much usage it got in production. So, if a cell was green, it meant that it was tested a lot when compared to how much it was used in production. If a cell was red, it meant it was under tested.

Often, entire vertical columns would be near red because the combination of device/OS wasn’t used much by our beta testers. So, we could direct our own efforts towards those devices and turn an entire column from red to green.

We also made sure new features would get their own row. These could also be red because beta testers might not know about them. We could similarly target those areas on all devices. These features could not be normalized against production usage (since they were not in production yet), so we use a baseline usage as a default.

Mode kept snapshots of the heatmaps over time. We could watch it go from nearly all red at the beginning of the beta period to more green by the end. I can’t say we could get the entire heatmap to be green, but we could at least make sure we were testing efficiently.

PR Authors Have a lot of Control on PR Idle Time

Getting a pull request reviewed quickly is often under the author’s control. This is great news because, according to DevEx, you should reduce feedback loops to increase developer productivity. There are other feedback loops that a developer experiences, but pull requests happen all of the time. You can have a big impact on productivity if they happen faster, and a big reason they don’t is because of the commits in the PR.

At Trello, during a hackathon, someone did an analysis on all of the PRs on all of the teams to see if they could get some insights. At the time, we probably had about 10 teams of about 7-10 developers each.

One thing they looked at was median time to approve a PR based on team, and there were two teams that were far outliers (with smaller waits for a PR to be approved). They went further to look at the PRs themselves and noticed that they generally had fewer commits and the commits themselves were smaller. The number of pull requests per developer-week was much higher than other teams.

I was on one of those teams, and the other one was very closely aligned with us (meaning we had a lot of shared processes and rituals). When we very small, 5 total developers, we were basically one team with a shared lead. The style of PR on these teams was very intentional. When I was onboarded, I was given very specific instructions on how to make a PR.

The essence of what we did was to completely rewrite the commits before making a pull-request to “tell a story”. I wrote about the details in Construct PRs to Make Reviewing Easy.

As a reviewer, this made it very easy to do approvals, which we could fit it in at almost any time. With all of us doing this, many PRs were approved within an hour and most in a few hours. A really good time to do some reviewing was right after submitting a PR, which made the throughput reach a steady-state. The PR list was rarely very long.

I worked in this style for the 6+ years I was on this team and know that it contributed to a high level of personal work satisfaction. Even though I had to wait for others to approve my work, I felt that my own productivity was largely under my control.

Related: Metrics that Resist Gaming

Just Started a New Software Engineering Job? Fix Onboarding

If you are about to start a new job as a software engineer, the way to have a big impact day one is to go through the onboarding with the intention to generate a list of improvements that you work on over time.

Here are some things to look for

Incorrect or outdated information. If you find these, just fix them as you find them.
Missing entry-point documentation. Even teams that have good documentation often do not have a document that is useful if you know nothing. Often they don’t go back and make a good “start here” kind of document.
Manual steps that could be scripted. Don’t go overboard, but if you see some quick wins to automate the dev setup steps, it’s a good first PR. It’s a tech debt payoff that is timed perfectly.
Dev Setup Automation bug fixes. If anything goes wrong while running the scripts to set up your machine, fix the bug or try to add in any detection (or better error messages) that would have helped diagnose the issue.

There is usually a lot of tech-debt in onboarding code and documents because no one really goes through them. Sometimes underlying things just change and tech debt happens to you. You are in a unique position to make it better for the next person and have some impact right away.

Invest 10% of a team (not of each dev) to pay back tech debt

If you budget 10% of your team’s time to paying down technical debt, there are a few ways you could do it.

Make sure 10% of the story points of each sprint are technical debt related
Assign every other Friday (10% of 2 weeks) to everyone paying down technical debt (see this article for a story about Tech Debt Friday)
Assign 10% of the team to spend 100% of their time paying down technical debt (rotating who this is every quarter or so, or at project boundaries)

I’ve done some variation on all of these ways, and in my experience, #3 works the best. When I was at Trello, my team allocated more like 30%, but tech debt was lumped together with anything “engineering driven”, which was more than just debt payoff (e.g. tooling).

The main reason #3 works better is how companies typically review and reward developers. Something that is 10% of your work is never going to show up on your review. Over time this is generally a disincentive to do it. But, if you are supposed to spend 100% of your time on something, then it has to show up on your review.

Making this someone’s full time job for a quarter means that they can plan bigger projects with more impact. It’s hard to get a PR done in one day, so it takes about a month to get anything deployed at all. When you work on it full-time, you can deploy much more frequently. When I was on a tech-debt project, I would use the first week to deploy some extra monitoring that could measure impact or catch problems.

It allows devs to get into the zone, which is really helpful in giant refactoring or restructuring/rewrite slogs. If you only get one day every two weeks, you have to reacquaint yourself with anything big, which eats into that one day quickly.

It will also make it more likely that this debt paydown is localized. This makes it easier to test that it hasn’t caused regressions.

Finally, (for managers) it’s easier to measure that you are actually spending your budget correctly because you don’t have to monitor individual stories over time. You just need to track how developers were allocated over time.

Knowing Assembly Language Helps a Little

I can’t say I recommend learning assembly, and I never really had to write much professionally, but knowing it has been helpful in giving me a mental model of what is happening inside a computer.

I started with assembly soon after I started programming in BASIC. In the eighties, all of the computer magazines listed assembly programs because that was the only way to do some things. Jim Butterfield’s Machine Language for the C64 was a classic.

In college, I used assembly in a few classes. In Computer Architecture we had to write a sort algorithm in VAX assembly, and in my Compilers course, we had to generate assembly from C (and then we were allowed to use an assembler to make the executable).

This was last time I wrote any significant amount of assembly, but in all of the time I worked in C, C++, Java, C#, and Objective-C, I found myself needing to read the generated assembly or bytecode on a lot of occasions. There were some bugs that I probably could have only figured out this way. Knowing how different calling conventions work in C on Windows was part of my interview at Atalasoft (and it was actually important to know that on the job).

So, if you have any interest in it, I would try it out. The main issue is that modern instruction sets are not optimized for humans to write. But, I learned 6502 assembler on a C64, and if you learn that then you can get into the wonderful world of C64 Demos.

Moore’s Law of Baseball

For almost my entire life (and before that all the way back to the dawn of baseball), the stats on the back of a baseball card were unchanged. If you got the box scores for your favorite player, you could calculate their stats yourself with a pencil. That’s not necessarily good. These stats were simple and misleading.

For example, it was clear in the 90’s that on-base percentage was more important than batting average. This got expanded on in the money ball era. Computers were brought in to analyze players, and so analyzing players was now subject to Moore’s Law, which can be simplified to say that we double computer power every 18 months. We’ve had about 20 doublings since then.

What the Moore’s Law of baseball? The number of stats is doubling every 18 months, all enabled by modern compute power.

There’s a stat called WAR or Wins Above Replacement, which tries to tell you how many wins a player adds to their team relative to the average player at their position (who has a WAR of 0). To calculate WAR for a single player you need every outcome from every player. It’s so complex, that we can’t agree on the right way to do it, so we have a dozen variants on it.

Stats like Exit Velocity, Launch Angle, Spin Rates, Pitch Tunneling, and Framing are only possible to know because of high-speed cameras and advanced vision processing enabled by Moore’s law. We’re not limited to describing what has happened already—some broadcasts put pitch-by-pitch outcome predictions on the screen.

Even with all this advancement, it still sometimes feels like we’re still at the dawn of this era. As a fan, these don’t feel like the right stats either. No one will be put in the hall of fame because they hit the ball hard a lot of times.

Just need a few more doublings, I guess.

LUI LUI

I go by Lou, but my entire family calls me Louie, so I smiled when I found out that there is such a thing called a Language User Interface that uses natural language to drive an application and that it was called a LUI.

In a LUI, you use natural language. So this is not the same as a keyword search or a terminal style UI that uses simple commands like the SABRE airline booking system.

In this video, it output responses on a printer. But the display terminal version was not that different. I worked on software that interfaced with it in 1992, and this 1960’s version is very recognizable to me.

But, this is not a LUI. A LUI does not make you remember a list of accepted commands and their parameters. You give it requests in just the way you would a person, with regular language.

In SABRE, a command might look like this:

    113JUNORDLGA5P

But, in a SABRE LUI, you’d say “What flights are leaving Chicago O’Hare for Laguardia at 5pm today?” which may be more learnable, but a trained airline representative would be a lot faster with the arcane commands.

With a more advanced version that understood “Rebook Lou Franco from his flight from here to New Orleans to NYC instead” that uses many underlying queries and commands (and understands context), the LUI would also be a lot faster.

This would have seemed far-fetched, but with ChatGPT and other LLM systems, it feels very much within reach today.

My Typing Teacher was a Genius

When I was in middle school, typing was a required subject. I don’t really know why.

In the early eighties it was not common for people to type at work. There were still specialists for that. Even in the late eighties when I worked in an accounting office and there were secretaries that took dictation and typed up memos. Computer spreadsheets existed, but the accountants there still used pencil and paper and secretaries typed them up if they needed look more formal.

This was the world my typing teacher, Mrs. Cohen, grew up in and probably worked in before becoming a teacher. I think, that deep down, she knew that we wouldn’t find typing relevant, and honestly, the class didn’t take it that seriously.

But one day, she read us an article from the local paper that said that kids needed to learn how to type because computers were going to be a big thing and soon everyone would need to know how to type. It had a huge impact on me—I still remember it very clearly.

I had already been exposed to programming and even had a computer at home. But, coding was just for fun. I didn’t think it would be a job, or that I would be typing every day at work. Mrs. Cohen was the first person that made me think that computers would be more than a toy.

The System Boundary is Defined by the External Pieces

In a C4 System Boundary diagram, you start by drawing a blue box in the center. That’s your system. And you draw some blue stick figures with arrows pointing at that box. Those are your users.

An empty blue box next to blue human shape. An arrow is pointing from the human to the box.

Every system in the world pretty much looks the same if you stop there. Put some words on the parts to make it more specific.

A blue box next to blue human shape. An arrow is pointing from the human to the box. The blue box says "Sprint-o-Mat: a watchOS app to guide programmed runs" and under the human is a caption "Runner". An arrow points from the human to the box and says "sets up and runs with". This is the system in a context diagram.

But this diagram is called a Context diagram for a reason. The most important part is not the system box (the three other types of C4 diagrams will elaborate on it), but the all of the gray boxes and stick figures you put around it.

These are the external pieces that are not the system and who are not your users. They are out-of-scope, but do a lot of work in the diagram to help describe the system.