In Praise of Pamphlets

One of the influential books in my life is A Technique for Producing Ideas by James Webb Young. I spoke about it at length in episode 13 of my podcast, where I called it a pamphlet.

I meant this in the sense of it being short and focussed. He doesn’t waste any of his 48 pages on background or fluff. It’s almost entirely about the technique.

I read a lot of (much longer) books in the same basic genre of this one — “A wise, old knowledge worker gives you their secret to doing knowledge work.” But probably because of the tyranny of the publishing industry, they have to be fleshed out to 300 pages, and so I have to read the same old Steve Jobs anecdotes again. Because pretty much any good idea about creativity, business, or productivity has a Steve Jobs anecdote.

We’re just lucky that Webb’s book predates Steve Jobs, but I don’t really remember him telling any anecdotes. There are no lessons from pseudo-science (or worse, non-reproducible, but compelling, real science). He is 100% betting that you don’t need to be convinced, or that the technique itself is convincing, and he rewards you by keeping it short.

Being American, I learned about the great patriotic pamphlets of the revolutionary era, like Common Sense, and so maybe I have a soft spot for this kind of work. In a way, maybe all pamphlets are common sense.

The First 13 Weeks

The first 13 weeks of the year ends on my birthday, and so each year I try to set up those months to try new things. It’s not a resolution or a goal, but more of a direction to explore. This year, I’m trying to make “make more art”, where “art” is very loosely defined. To start, I’m doing more sketching, writing, and I’m even going to drop in on an improv group.

I journal in a Recurring Journal I made last year for 2023 — it splits the year into four 13 week cycles where you journal the same day in the cycle on the same page spread. So, on April 3rd, I’ll be journaling on the same page as January 2nd. The idea is that I can see some progress and renew my energy for the next 13 weeks.

Lessons from My First Real Program

(I commemorated this program in a t-shirt)

In 1983, in the wood shop of JHS 125 in Woodside, Queens, perhaps sensing my frustration in making a spice rack, my teacher asked if I’d rather be in “computer shop”. Mr. Abbey had good instincts.

My new shop teacher, Mr. Penner, gave us each a sheet of graph paper and told us to count 80 columns and 25 rows and draw a picture by filling in squares. I drew a skull (I think it might have been getting close to Halloween). Next, he taught us enough BASIC so that we could write a program to draw our pictures on the screen.

I looked up an old Commodore PET manual, PETASCII table, a PET memory map, and downloaded VICE, a PET emulator, to recreate it here (lines 1010+ would have the coordinates from the drawing):

10 REM DRAW AN AWESOME SKULL!!
20 REM BY LOUIS FRANCO
30 READ X, Y
40 IF X = -1 THEN END
50 POKE 32768 + y * 80 + x, 160
60 GOTO 30

1000 REM (THE X, Y COORDINATES FOR THE SKULL)
1010 DATA
1020 DATA
2000 DATA -1, -1

What I love about this introduction to code:

  1. It is pretty much the minimum code you can write to get a drawing on the screen.
  2. It has conditionals and loops, so it’s more interesting than Hello World.
  3. It takes advantage of something a 13 year-old might know — X, Y coordinates.
  4. He gave us lines 30-60 because you have to know minutia of PET architecture to understand it. He did explain it, but didn’t expect us to be able to write it.

The next lesson was to animate the drawing (cycling two frames), which we were mostly on our own to do.

T-shirt showing a PET computer with a pixelated skull on it in green

Making More Things in Public

I really try to put most of what I make out there, whether it’s code in GitHub or random thoughts here on this site. But, the truth is that I have a lot of half-baked (tenth-baked?) stuff on my drive. Also, I am doing more work that has to be private, but I am actively looking to lower the bar and share more.

One class of things I don’t share is vector drawings that I make mostly for my apps or to practice graphic design. But, I really wanted to make my own t-shirts with some of them, so I submitted to Cotton Bureau and opened a store a few days ago. You can check them out on my t-shirt page. My favorite is PET Skull, which is based on my first real program. You can use the coupon [USP6ZBJXUS] for 10% off until the end of 2022.

Introducing Page-o-Mat

As I mentioned in my post about recurring journals, I decided to write a python program to create a PDF that implemented my riff on the Da Capo journal. I finished it enough to share, and put it on GitHub. It’s called Page-o-Mat.

To make it more generally useful, instead of hard-coding this particular journal, I made it read a configuration file that can be used to describe lots of journaling styles.

Right now, the paper and page types are what I needed for my 2023 journal. I am about to do a test-run with getting it printed on LuLu. After that, I might make some more page templates.

The config file is in YAML. You could just list each page you want one-by-one, but to make things easier, there are a few ways to create enumerated pages.

  • Any page could have a count, which just repeats that page
  • Any page can have a variant list, which repeats the page, setting a variable that can be used in titles
  • Pages can have lists of sub-pages, and so on.
  • Pages can have sections

In the code, it’s implemented with nested loops and recursion. The indices of those loops are exposed to the config to use in date math, so that any section of any page can have a date calculated for it. Most journals would probably have sequential dates, but a recurring journal wants you to keep cycling back to the beginning of the book, so the date math is fairly complicated.

There are documented examples in the repo that (hopefully) explain this better. Start with daily.yaml, which is just a 365 page daily journal. 2023-recurring-journal.yaml shows the more complex example.

Recurring Journals

A few weeks ago, I backed a Kickstarter for a ”Day-based Journal”. The idea is that each spread of two pages represents a day of the month. For example, when you open it up, the left page has boxes for Jan 1, Feb 1, March 1, April 1, May 1, and June 1. The right-hand-side has July 1 … December 1. So, to journal in January, you turn pages for 31 days and then go back to page 1 to start February. You go back to page 1 to start March, and you can see each day in January and February as you progress.

This has two effects:

  1. You see your notes over and over (this reminds me of spaced repetition)
  2. You can see monthly progress and eventually a years worth of progress

I love the idea and hope it gets funded, but in the meantime, I’ve started designing a riff on this idea that I will probably print out for 2023 on LuLu.

As a prototype, I drew something similar to the Kickstarter for 62 pages of my current blank journal to cover the rest of the year (Sep-Dec).

Prototype of a recurring journal showing Sep 1, Oct 1 on left hand page and Nov 1 and Dec 1 on right hand page
Prototype of a recurring journal

Since that’s just four months, each box is much bigger — half a page each month to make up the two-page spread. For me, I think that’s the minimum size. I don’t think I could use the orignal Day-based journal as a primary way to journal, since I need space to do a time block for the day.

The next thing I noticed was that each day was a different day of the week. That is mentioned as a plus in the Kickstarter, but I think in weeks. So, for me, it would be much more useful if each day shown on the spread were the same day of the week. My Mondays are very different from my Fridays, and the weekend is nothing like a weekday.

So I started to sketch out what a 2023 Week-based ”Recurring” Journal might look like.

My solution so far is to run 2023 as 4 sets of 13 weeks. That’s 364 days, and since Jan 2 is a Monday (and I start my week on Mondays), I’ll treat Jan 1 as a special day and then make 2-page spreads for the rest of the year where each box is 13 weeks apart.

One downside is that they are not aligned to months or quarters, but that’s just the nature of weeks, and I want to embrace weeks. I decided to further sub-group the 13 weeks as 4, 4, and 5 week blocks. That’s almost a month, and while it’s not ideal that they are uneven, I am also embracing the 13, so I have to accept its primeness.

Also, you are only revisiting your notes every 13 weeks instead of every month. To address this there will be 28 summary pages with 13 boxes where you try to get the essence of the day. To get 13 equal sized boxes, I’ll borrow from Betsy Ross and just use horizontal stripes. I think this would be the best way to use the original Kickstarter journal.

I’m currently building a spreadsheet where I can plan out what each page of the journal will be. In addition to daily pages, there will be pages for planning and retros. I also want enough blank pages for random note taking. So far it’s over 300 pages.

When that’s done, I’m going to build a python script to generate the PDF for the interior pages. I will open-source it, so if you are interested, then watch this space.

Extending GraphQL Code Generation (Part II)

In the last post I said that even though I had used code generation throughout my career, the expressiveness of modern languages had reduced my usage. But, I had found that connecting code to external systems was better if I generated a type-safe API layer.

GraphQL Code Generation (which I mentioned in the last post) does this for GraphQL client code. You provide your queries and the endpoint to your server, and it will validate the queries and use the server’s schema to generate type-safe Typescript wrappers to use to execute those queries, including react hooks.

It doesn’t go far enough for me, but since it supports plugins, it was easy to add what I wanted. All I wanted was simpler, non-hook, type-safe functions to make GQL requests. The hook variants are very convenient when you are just populating a screen, but I had times when I wanted to make GQL requests and didn’t want to handle the result in the React lifecycle. This was even more true for mutations.

So, as a first step, I made a non-type-safe function to do any mutation given an Apollo client and document (NOTE: DocumentNode is a type that represents a GQL query).

export const gqlMutate = async (
  apolloClient: ApolloClient<object>,
  mutation: DocumentNode,
  responseProperty: string,
  variables: any,
  onSuccess: (response: SuccessResponse) => void,
  onError: (err: string) => void
): Promise<SuccessResponse> => {
  try {
    const response = await apolloClient.mutate({ mutation, variables });
    // handle response and call onSuccess or onError
  }
}

I could just stop here, but then when I write a call to gqlMutate, I get no code completion or type-checking.

But the GQL code generator actually generated concrete types for all of my GQL queries, so I could write a wrapper, like this:

export async function requestCreateOpening(
  apolloClient: ApolloClient<object>, 
  variables: gql.CreateOpeningMutationVariables, 
  onSuccess: (response: gql.SuccessResponse) => void, 
  onError: (err: string) => void): Promise<gql.SuccessResponse> {
  return await gqlMutate(
    apolloClient, gql.CreateOpeningDocument, 
    "createOpening", 
    variables, 
    onSuccess, 
    onError);
}

Which works great, but I have hundreds of mutations and I would have to write this code over and over and keep it up to date if anything changes. But, literally everything I need to know to write this is in the GQL query document. The code generator already generated this constant and type for me.

export type CreateOpeningMutationVariables = Exact<{
  teamId: Scalars['Int'];
  name: Scalars['String'];
  body: Scalars['String'];
  whatToShow: Scalars['String'];
}>;

export const CreateOpeningDocument = gql`
    mutation CreateOpening($teamId: Int!, $name: String!, $body: String!, $whatToShow: String!) {
  createOpening(
    teamId: $teamId
    name: $name
    body: $body
    whatToShow: $whatToShow
  ) {
    success
    error
  }
}
    `;

It’s not a stretch to get it to generate the wrapper around gqlMutate as well.

Normally, the drawback is having to parse the source language, in this case GQL. But GQL code generator already does that and already generates a lot of useful types for you. All you need to do is loop through your GQL documents (which are types) and write out the code. For details see the plugin documentation.

It’s too hard to explain my generator (write me if you need more help). But, if you are calling GQL with Apollo and typescript from your front-end, consider code generation. And, if you find yourself writing non-type-safe wrappers, know that you can extend it with plugins to make them type-safe as well.

Why I am Using Code Generation Again, Part I

In the nineties, I used Rational Rose and Perl to generate a lot of C++ code. I’m not sure if I knew the term at the time, but we also created DSLs (Domain Specific Languages) that probably fell into the “… poorly specified, half-implemented Lisp” trap.

Back then, I would say that I used code generation to overcome the limited expressiveness of my programming language. There was a lot of boilerplate in C++ (and later Java and C#) that I didn’t want to write—I didn’t even want it to be part of the source. I considered the DSL to be the source in these cases.

When I moved onto iOS development, I used code generation sparingly. It somewhat complicates the build process and the Swift language designers prioritized expressiveness enough to make it less necessary.

But, recently, I’ve begun to use code generation more while using Typescript. Typescript is a lot like Swift, so it’s expressive enough, but the main difference is that I am using Typescript to access things that don’t have types in their built-in API (SQL and GraphQL), but do have an underlying type system.

I decided to use TypeORM and TypeGraphQL, which allow me to define types that map onto the database and are passed around in the the GQL, and so on my backend, I have a strongly typed API to both. These systems are based on “decorators”, to tackle the problem of boilerplate code. But TypeORM generates the SQL to build the schema and each migration as I add to it, so there is code generation as well.

This is great for the backend. My server ends up exposing a GQL API which can also provide a schema. But, I still have the problem of having a typed API to GQL from my front-end. For this, I ended up using GraphQL Code Generator, which can connect to a GraphQL server, read the schema and your GQL queries, and generate a nice typed API, including React hooks for fetching your data.

Even with all of this, I didn’t think of myself as generating code. I was not writing parsers or generators—I was just using a system that happened to generate code. But having this system in my build flow already (and the fact that GQL code generator can have plugins) made it inevitable that I would eventually be generating my own code.

GQL Codegen is already doing all of the heavy lifting of getting the schema and parsing it—you can write a new plugin in a few lines of code. Once I wrote the plugin, I could delete hundreds of lines of boilerplate code around doing GQL mutations without having to resort to typeless wrappers. I’ll provide some details in Part II.

Seeing Code

There’s a saying that code is written to be read by programmers and incidentally to be run by a computer. I would argue that that doesn’t go far enough. The goal shouldn’t be readability, it should be to make it so you don’t have to read it.

When I have been working with a codebase for a while, I don’t need to read the code all of the time. I see it. I’m not perceiving it serially as lines of code—it’s more like seeing it all at once.

Some of this comes from familiarity. It’s in my head already. The code itself is a reminder. It’s something that comes from flow and extended time with the code.

But, it’s mostly a function of the code itself. Some code is easier to perceive this way.

What helps me is if the code is made up of memorable and predictable chunks. In a seeable codebase, I read one file and just know how the others would look without having to read them. When I open another one, it looks how I pictured it.

This idea is related to System Metaphor from XP

The system metaphor is a story that everyone – customers, programmers, and managers – can tell about how the system works. It’s a naming concept for classes and methods that should make it easy for a team member to guess the functionality of a particular class/method, from its name only.

and Convention over Configuration from Ruby on Rails.

The same goes even when you understand how all the pieces go together. When there’s an obvious next step for every change, we can scoot through the many parts of an application that is the same or very similar to all the other applications that went before it. A place for everything and everything in its place. Constraints liberate even the most able minds.

and the Principle of Least Astonishment

The principle aims to leverage the existing knowledge of users to minimize the learning curve […]. In more abstract settings like an API, the expectation that function or method names intuitively match their behavior is another example. This practice also involves the application of sensible defaults.

Astonishment is perhaps the best way to perceive code that resists being seen. It may be perfectly readable and understandable, but it should also already be predictable. If it surprises you, then you should ask yourself why.

The Unreasonable Effectiveness of Mathematics in Programming

(with apologies to Eugene Wigner)

My college didn’t have a CS major, but they let me put together one under a general engineering program. To fill up the requirements, I had to take a lot of math, which has been more useful than I expected a programmer.

I didn’t seek out jobs that required a lot of math. I optimized my job search around small companies doing product development for B2B, and didn’t care much about the specific technology they used. But, I was comfortable with the math, so it made my life easier.

The first eight years of my career was in FinTech, and the software I wrote was a nice UI around a lot of math. The core concept was options pricing (probability and statistics) and the sensitivity of that price to its inputs (calculus and differential equations). To do risk analysis, you have to build up huge matrices (linear algebra) for various purposes. Our company employed mathematicians, so we didn’t do the research, but we had to understand it to work on those parts.

Later, I contributed to a patent related in spreadsheets where graph theory was important. I also implemented numerical differentiation and root-finding algorithms as a way to run expressions backwards (numerical analysis and calculus). That patent expired, so I am reimplementing it in Swift and Typescript.

In 2005, I did a consulting project to implement a distributed monte-cargo engine for a decision support system. I would not have won this bid if I did not understand the math behind the engine.

From 2006-2013, I worked at an image processing tools vendor. This job was the closest to pure CS that I have had, and there was a lot of math, specifically linear algebra, but also some numerical analysis.

Every front-end position I have had uses at least a little linear algebra (for affine transformation). It’s not like you are doing the matrix multiplication yourself, but you’ll understand the more complex compositions better if you understand them. For example, if you know that matrix multiplication is not communicative, you’ll get why the order of the transformations matters.

Nearly every programming job now requires you to understand the analytics data that the software generates and to do statistical analysis on it. Forming a hypothesis and getting evidence to support or reject it is essential. At a bigger company, you might have a data-science team to do the heavy lifting, but it really helps if you can do it yourself—you also want to be able to read and understand their reports.

If you really want to go deep into the type theory behind type-safe languages (like Swift or Typescript), you have to understand set theory and maybe even HoTT. You don’t need it to use these languages, but if you had interest in compiler theory or implementing a language like this, it would help. Understanding set theory also helps with understanding relational databases.

When I was trying to find a Swift implementation of numpy a few years ago, I ended up finding Surge and contributed Eigen decomposition to it. I had to relearn it, but I would not have even tried if I hadn’t touched linear algebra since college.

Games are essentially physics simulators, which are ultimately math engines. I only write simple games as a hobby, but even for pong, I had to write vector math functions.

And, although I think my career has used a somewhat typical amount of math, there are programming jobs that require a lot of math. A deep neural network’s guts are a calculus and linear algebra engine (see this video from 3blue1brown). As I mentioned, data science makes heavy use of probability and statistics. I learned in Quantum Country that the “programming language” of a quantum computer is based on matrix multiplication with complex number entries. And while writing a game is a reasonable amount of math, writing a game engine is much more than that—and as more things (the blockchain, games, machine learning) have used the GPU for computing, the way you think of implementing solutions in those domains is more mathematical.

To be clear, you can do a lot of programming without understanding any of the math behind it. But, I have found it more enjoyable to go a little deeper, and it makes it easier to develop intuition about the higher level concepts.