Category Archives: Software Development

Tech Debt Happens to You

In the original ANSI C, there are a bunch of library functions that use internal static variables to keep track of state. For example, strtok is a function that tokenizes strings. You call it once with the string, and then you call it with NULL over and over until you finish reading all of the tokens. To do this, strtok uses static variables to keep track of the string and an iterator into it. In early C usage, this was fine. You had to hope that any 3rd party library calls you made while iterating tokens weren’t also using strtok because there could only be one iterator at a time.

But when threads were introduced to UNIX and C, this broke down fast. Now, your algorithms couldn’t live in background threads if they used strtok. This specific problem was solved with thread-local variables, but the pervasive use of global state inside of C-functions was a constant source of issues when multi-threading became mainstream.

The world was switching from desktop apps to web apps, so now a lot of your code lived in a multi-threaded back-end that serviced simultaneous requests. This was a problem because we took C-libraries out of our desktop apps and made them work in CGI executables or NSAPI/ISAPI web-server extensions (similar to Apache mod_ extensions)

To make this work, we had to use third-party memory allocation libraries because the standard malloc/free/new/delete implementations slowed down as you added more processors (from constant lock contention). Standard reference-counting implementations used normal ++ and -- which aren’t thread-safe, and so we needed to buy a source code implementation of stl that we could alter to use InterlockedIncrement/InterlockedDecrement (which are atomic, lock-free, and thread-safe).

As the world changed around us, we could keep moving forward with these tech-debt payments.

Also, this was slow-paced problem—strtok/malloc/etc were written in the 70s and limped through the 90s. That’s actually not that bad.

But, the world doesn’t stop. Pretty soon, it was just too weird to implement back-ends as ISAPI extensions. So, you pick Java/SOAP because CORBA is just nuts, and well, that’s wrong because REST deprecates that, and then GraphQL deprecates that, and you picked Java, but were you supposed to wait for node/npm? Never mind what’s going on on the front-end as JS and CSS frameworks replace each other every 6 months. Even if you are happy with your choice, are you keeping your dependencies up to date, even through the major revisions that don’t follow Substitutable Versioning?

And I think that this is the main source of tech debt, not intentional debt that you take on or the debt you accumulate from cutting corners due to time constraints. The debt that comes with dependency and environment changes.

Being able to bring code into your project or build on a framework is probably the only thing that makes modern programming possible, but like mortgages, they come with constant interest payments and a looming balloon payment at some point.

There are some you can’t avoid, like the OS, language, and probably database, but as you go down the dependency list, remember to factor in the debt they inevitably bring with them.

Timing Your Tech Debt Payments

It’s impossible to ignore that developers have a visceral reaction against tech debt. Even if they agree that it’s worth it. That’s because they are the ones that need to service the debt.

Tech debt is a cost similar to real-life debt like a mortgage. If you can use tech debt to bring forward revenue and growth, you can pay off the debt later.

But, until then, the interest must be paid.

So, when you are calculating the cost of taking on some debt, a factor in that calculation is how much future work is going to happen on that code. The more work you do, the more interest you pay. If you fix bugs or add features to debt-laden code, you are servicing the debt by making an interest payment. If you refactor, you are paying off principal, and future interest payments are lowered, but that only matters if there are going to be future interest payments.

If you have a system that works and doesn’t need any changes, the fact that it has tech debt doesn’t matter.

To carry the analogy forward, some mortgages have penalties for early payment. Paying off tech debt also has a penalty, usually in QA and system stability.

This is why my favorite time to pay off tech debt is just before a major feature is being added to indebted code. You are trading off the looming interest payments (which will balloon) and your penalty is already being incurred, because you need to QA that whole area again anyway.

AirTags Could Be Used for Precise Indoor Location

I don’t think there’s going to be an SDK for AirTags, and they seem to be designed to be found by a single person, but the same technology could be used to precisely locate myself indoors (if AirTags were installed to create a mesh).

This is supposedly what iBeacon’s do, but I’ve heard from people trying to deploy them that the technology doesn’t work very well. I don’t really know anything about this at all, but here’s a contrary view from someone who knows beacon tech better:

We don’t believe that these tags will replace the current generation of BLE beacons for a few reasons:

・ These UWB Tags will require a new (circa 2020) Apple or Samsung
・They will not be compatible with most of the existing gateways
・ These tags will most likely initially only work with the proprietary applications on Apple or Samsung Phones
・ Apple and Samsung UWB seem to be geared towards finding lost items, not providing all of the other sensor data that current BLE beacons do
・ BLE Beacons will be much, much cheaper than these UWB Tags will be

And, this is probably true today with AirTags as they are. But, this article also says that Google is dropping support for BLE beacons, so there is some problem here.

What is Bicycle?

Bicycle is an open-source framework that I’ve been working on with a couple of friends. One way to think about it is to compare it to a spreadsheet.

In a spreadsheet, we are building a directed, acyclical graph of cells, where cells are nodes and each formula in the cell defines the edges and direction.

If A1=B1+C1, then both B1 and C1 point to A1. The graph cannot contain cycles, so, in a spreadsheet, you can’t then say that B1=A1-C1 even though that is true, because it would cause a cycle in the graph.

Bicycle defines a data-structure and algorithm that gives meaning to a graph of formulas that does contain cycles where dependencies between nodes can be bidirectional (hence, Bicycle).

In Bicycle, you can define both of the formulas above and also complete the network with A1=C1-B1. In more complex networks, an individual field may have several different formulas, using different dependencies that can set its value.

Once you define a network, you can seed it with values. These are kept outside of the formula data-structure in a kind of priority queue. The highest priority values are seeded first, and each value is only accepted into the network if the network can remain consistent.

Meaning, I could define A1 as 2, B1 as 3, but if I then say C1 is 7, I have created an inconsistency. When you attach this to a UI, you would want the oldest value to be discarded.

We are also providing some help building SwiftUI based UIs with it. The network is meant to be hosted in an @ObservableObject and we provide a TextField Initializer that will bind to fields in it. Here’s a demo of a network that can convert between yards, feet, and inches.

Try to imagine replicating that in Excel. You’d have to pick one of the fields to be user-provided and put formulas in the other two. In Bicycle, you can provide as many formulas as you want (as long as they are consistent with each other) and seed-values are used as long as they are consistent.

It’s in very early stages and the API will probably change a lot, but if you want to take a look, see SwiftBicycle in GitHub.

In Defense of Tech Debt

I’m a fan of tech debt if used properly. Just like real debt, if you can pull some value forward and then invest that value so that it outgrows the debt considerably, it’s good.

Mortgages and tuition-debt can possibly do this. Credit card consumption debt does not. If your tech debt looks like the former, do it.

For example, if can can close a big enterprise deal with some tech debt, and the alternative is another round of VC to “do it right”, I think it’s obvious to hack away. When you close that deal, your valuation goes up. Maybe you don’t even need to raise.

The decision depends on the specifics. Tech debt isn’t “bad”, it’s a cost. Calculate the cost.

It can be worth it.

Audio Variables

We can apply principles of visual design to designing audio. In visual design, we can manipulate “visual variables” to get different communication effects. The variables are are:

  • Size
  • Value/Brightness
  • Hue/Color
  • Orientation/Rotation
  • Texture
  • Shape
  • Position

Continuing along with sonifications, here are some analogous audio variables:

  • Frequency / pitch
  • Beat
  • Amplitude / volume
  • Envelope / Waveform: e.g. a beep vs. a buzz
  • Source Location

We have different ways of perceiving visual variables. For example, for some of them, we have an order—size goes from small to big, but shapes are not ordered.

We also have different ideas about how many variations we can tell apart on the same canvas. We can distinguish between a huge number of shapes, but probably only a few levels of brightness.

These kinds of perception apply to audio variables as well. Waveform seems to be related to shape in that there can be many types (instruments), but they are not really ordered.

Amplitude is similarly related to size, pitch is like brightness, and source location is like position.

Using what we know about combining and contrasting visual variables is probably a good start for doing the same with audio.

Great Software Writing (that influenced me personally)

A few days ago a proposed a short-list of great software works. A couple of them are associated with the great works of software writing, and so I started to think about what the top five pieces of great software writing were.

This was a lot harder for me to narrow down, so in the end, I decided to make it personal. What had influenced me the most? This means I have to leave off The Art of Computer Programming, which is undoubtedly a great work, because I haven’t read it.

These works were chosen because they shape how I think about programming anything. They are language and framework independent. I chose three that are more about process and three that were more technical.

This is in the order I read the books or papers.

Process

Technical/Coding

For me, these are works that still affect how I program today. My main lessons:

From Peopleware, that the biggest factors in programming productivity were access to quiet and long blocks of time, and that this explained correlations of productivity within a workplace.

From eXtreme Programming, unit testing, continuous integration, pair programming, small releases, and refactoring which are all part of how I have worked from the day I finished reading it.

From Joel on Software, don’t rewrite software, UI design for programmers, tons of software business. Also, one of the main reasons I applied to (and worked for) Trello.

From Design Patterns, that it’s useful to look for common solutions and give them a good name and description (not the specific patterns, although I obviously had learned from them).

From A behavioral notion of subtyping, I learned about how to progress software. I think about substitutability for almost every line of code I write inside of deployed software. I wrote about this in my blog post on Substitutable Versioning.

From Purely functional data structures, I learned that functional programming was technically possible. I had read about FP (and even wrote about it) in college (~1990), but at the time, it wasn’t conceivable to me that this could work—granted, I was just a student. After clojure came out, and introduced me to Okasaki’s work, I was convinced that FP had a place in my toolset. I wrote about that revelation back in 2008 in my 20 days of clojure series, where I tried to learn clojure quickly because I was helping to host a Rich Hickey talk.

Some notable things that I read but didn’t include are Decline and Fall of the American Programmer (this was interesting but turned out to be mostly wrong), Writing Solid Code, and Code Complete (great, but read it too late in my career to be influential).

Great Works of Software

I was thinking about great works of art attributed to a single person or small group. Things like The David, The 5th Symphony, or Pride and Prejudice. They are each on the short list of the greatest sculpture, classical composition, and novel.

What would the short list for software be? I put together a list, but of course, it is personal to my experience and biases. Here is my top 5 in roughly chronological order.

  • C and UNIX
  • The ARM instruction set
  • TeX
  • VisiCalc
  • Web browser/server

I edited down this list from of about twenty. Notable removals were the collected works of NASA and Pong.

There are some similarities I see that bind them and maybe help to define great works.

  1. They are in wide use over a long period of time. Visicalc is possibly a stretch here, but its influence and core functionality are in 123, Excel, Google Sheets, etc.
  2. They were made to make something else. C was made to make UNIX to make microcomputers. Sophie Wilson used ARM to make microprocessors and Acorn machines. TeX was made to make The Art of Computer Programming [amazon affiliate link]. Dan Bricklin made Visicalc as an MBA student to analyze business cases.
  3. They are mostly attributed to a single person or pair. That was true of many of the entries on my list, especially programming languages.
  4. They define languages. Again Visicalc is a stretch, but I do think of Excel as a Programming Language and Microsoft is releasing Power Fx, a language based on spreadsheets.

This combination is also what drove Alan Kay and his team to develop Smalltalk, Object-Oriented programming, GUIs, etc to try to make Dynabooks, so that presumably they could write content in that new medium.

That work led directly to Macs and is even more directly used in NeXT/Objective-C, which is what Tim Berners-Lee used to make the first browser. And if you use it to edit a Google Sheet on an iPad, it’s on a UNIX variant running on ARM, probably using algorithms documented in TAoCP.

Deliberate Practice by Cloning

If you are trying to learn programming by implementing something new, you are conflating two skills and making it hard to do either of them well.

Yesterday, when I wrote about deliberate practice, I listed the two things you need to do to make practice effective

  1. Break it down so that you are practicing one thing and trying to make it automatic
  2. Set it up so that you have instant feedback on whether you are doing it right

As you work on your program, sometimes you are thinking about the syntax of the language and sometimes you are thinking about where to go next. It’s better if you can stay in one mode for a long period of time until that skill becomes automatic.

And, since there’s no set goal, it’s hard to know if you accomplished it. You have no objective feedback.

I remembered that in a recent Under the Radar episode, Marco Arment and David Smith talked about cloning apps as a learning exercise.

By cloning, we can concentrate on just coding without having to come up with the content too. Even this can be broken down further. Structure your practice sessions to concentrate on one aspect at a time. For example, just the UI.

You can think of the target app as an executable spec, which is easy to compare your work to. If you are learning UI programming you can screenshot both screens and flip back and forth between them to see how close you can match it. This establishes instant feedback.

I recently tried this exercise for the Breathe app on the Apple Watch, only working on the animation and wrote up a SwiftUI animation tutorial on App-o-Mat.