Author Archives: Lou Franco

Audio Copilot

When I first wrote Robotic Pair Programmers I really was imagining that an AI code assistant would be more like a side channel and not inside autocomplete. To be fair, I was programming mostly in Xcode at the time and had not yet experienced VSCode, so my conception of how fast an IDE could be was antiquated.

I do think that the autocomplete integration is genius, but I have also been wanting an audio side-channel to my work that would be more like a human assistant. Think of it as a ChatGPT enabled Siri, but still fully integrated into my IDE.

Here’s a script of what that might look like (NOTE: the first draft of this script was generated by ChatGPT, which I am pointing out because I think it’s interesting in this case. I edited it to fit the article better.)

Programmer: “I want to do a binary search in Python. Can you help?”

AI assistant: “Sure. One way to start is to define a function called binary_search that takes in a sorted list and a target element. Inside the function, you can initialize variables called left and right to the beginning and end of the list, respectively.”

Programmer: “Okay, I’ve defined the function. What’s the next step?”

AI assistant “Next, you can use a while loop to iterate as long as the left index is less than or equal to the right index. Inside the while loop, you can calculate the middle index by taking the average of the left and right indices.”

Programmer: “Got it. And if the element at the middle index is the target element. I can return the index. What do I do if the middle index isn’t the target element?”

AI assistant: “If the target element is less than the element at the middle index, you can set the right index to be the middle index minus one. If the target element is greater than the element at the middle index, you can set the left index to be the middle index plus one. This way, the while loop will continue until the target element is found or the left and right indices cross each other.”

I would expect that the assistant would make incorrect assumptions or mistakes and then the programmer would clarify.

More importantly, when the programmer is programming, the AI assistant will still be making suggestions via autocomplete, but now is much more aware of the goal and so we’d expect the suggestions to be better.

The much bigger win will be when the assistant doesn’t wait for my requests, but interrupts me to help me when I am doing something wrong. To continue the binary_search example, if I set left to the middle index (off by one) then the assistant would let me know my mistake via audio (like a human pair would).

Just like in Assistance Oriented Programming, I think the key is to get intent in Copilot as early as possible.

Addendum

This example is simple, but I generated lots of interesting scripts in ChatGPT where the programmer and assistant collaborated on

Testing the binary search
Doing quicksort together, but I asked ChatGPT to make the assistant make incorrect assumptions that get corrected.
Building a burndown chart in a web based bug tracking program

They were all interesting, but I didn’t include these because the that isn’t the point of the article.

Book Recommendation: Make it Stick

I read Make it Stick [amazon affiliate link] last year, and it’s the book that made the biggest impact on me in 2022. It’s about the “science of successful learning”, and is co-written by researchers in the field who based it on their work studying how we learn.

The core idea is that you must use “retrieval” in various forms to learn a subject. This means that you practice remembering and applying the material instead of re-reading it. Some of the suggestions are:

Practice remembering with flash cards that are spaced, interleaved, and varied
Generate your own answers to problems before learning the technique
Elaborate on material by writing original text that draws from the material
Reflect on your learning sessions by writing a meta description of the material and your relationship to it (e.g. where you struggle, how it’s going)
Calibrate your knowledge with objective third-party sources

I had been primed to accept its suggestions because I was introduced to some of them already. I discovered the book in a video YouTube recommended to me because I watch videos about these topics often.

I learned about Spaced Repetition (using flash cards) a few years ago and have been using it nearly daily since then. I spoke about it at length in Episode 14 of my podcast. While reading this book, I created cards in my Anki deck to help me remember its core ideas. When those cards come up, they test my memory, but also remind me to use the practices.

The “Elaboration” suggestion is very much like Smart Notes, which I learned from How to take Smart Notes [amazon affiliate link]. I am writing this review to use “Elaboration” to help me remember its lessons.

And I’m a big believer in Calibration, so I wrote problem sets for beginning programmers trying to learn Swift.

But even with that background, I enjoyed the more expert coverage of the topics with more details on why these techniques work. Since I have been doing many of them for years, I was able to come to it with less skepticism.

An interesting side-note is that they use the techniques they suggest in the structure of the book, but they are limited by what you can do in a static text. If you are interested in this idea taken to a logical extreme, I would recommend reading Quantum Country, which embeds interactive flash cards in the text.

If you struggle in retaining material or need to learn a complex subject, I would certainly give this book a read and try to incorporate its suggestions into your process.

Assistance Oriented Programming

Here’s a simple SQL query:

SELECT p.id, p.name, p.age FROM person AS p

It’s a reasonable syntax for a simple operation. But, when Microsoft designed LINQ, they decided to put the datasource first:

from p in person select new { p.id, p.name, p.age }

LINQ was designed knowing that it would be used in Visual Studio, and so Microsoft made it easy for the IDE to autocomplete. If you tell it the datasource first, it will know the possible fields when you type dot.

The obj.member syntax predates modern IDE autocomplete. Even SQL is using it in the above example. The innovation in LINQ is getting the object name and type into scope before you need to access any fields.

So, as I continue to play with GitHub Copilot, I wonder if its widespread adoption will spur on language features that are designed to make better use of it.

One thing holding back that innovation is that Copilot needs to have a giant corpus of examples. If you make a new language, then Copilot can’t really help you write it. And if the main reason to use it is that it can be more easily assisted, then it can’t get off the ground. One way to get around that is to generate a corpus if the new language is a simple transformation of an existing one.

A more likely scenario is that programming style adapts to be more easily assisted. We see from the LINQ example that we want to get names and types known as quickly as possible. This will help AI assisted programming as well, but even better is getting your intent in the code as early as possible.

Two ways we signal intent in programming is with names and with comments. To get assistance, you should use good names. I notice that the quality of the suggestions is based on the names I use.

I’m torn about what will happen with comments. Comments that describe code aren’t worth keeping, but may result in good suggestions. Hopefully, most programmers won’t leave those comments in.

I wonder how good Copilot can do with comments that are more declarative. And if it’s good at it, will a loose, declarative, pseudocode that is embedded in comments become the de facto Assistance Oriented Programming Language I’m looking for?

Initial Thoughts on AI Assisted Programming

I’m fairly active on StackOverflow, so when people started answering questions by copy-pasting from ChatGPT, I noticed. Unlike the AI generated code shared on social media (which appears to be cherry-picked), in actual use, the ChatGPT answers were nonsense. And so, StackOverflow banned its use.

The problem was not that ChatGPT was generating nonsense — the problem was that people were posting the nonsense without vetting it. That’s partly because the answerers were not even trying (they thought it was fun), but also, I would guess that many of them didn’t know if the answer was good or not. The answers were obvious nonsense to experts, but less so to beginners or laypeople.

Right now, AI generated code can’t be used without an expert. But is it useful if you are an expert?

Two years ago, I said that I wanted a Robotic Pair Programmer. In that post, I made some suggestions for what I’d want:

One way that seems fruitful to me is rare API calls. There will be times when I am using an API that appears very infrequently in the corpus or my own repositories. In that case, it should infer that I probably need more help than usual and offer up a tutorial or the top Stack Overflow questions.

And …

Another trigger might be my new comments. If I comment before I code, then it should be interpreted as a search query:
// Parse the JSON I get back from the data task
That should bring up links to likely API classes in the help pane (just like it would if I already knew the class). Maybe offer up imports to auto-add. Maybe offer a snippet.

One important thing is that this needs to be just in the IDE, not a chat interface. And last March, GitHub Copilot was released as a VSCode plugin. I ignored it back then, but seeing how far ChatGPT had come made me think that Copilot had a chance of being good.

The best thing about Copilot is its UI. It feels like autocomplete, but offers up more than usual autocomplete would. Sometimes it can complete a line — sometimes it can give you a few lines. In either case, since IDE users are already used to this interaction, it doesn’t get in the way. It’s also fast, which is vital.

It also does the comment trigger I wanted — meaning, I can comment my intention it will offer up snippets. Many of these are useful.

One worry I had in 2021 was that a system like this would offer bad suggestions often. And, that’s the main drawback to Copilot. Much of what it suggests is wrong. Some of it is scarily accurate, and the rest is in-between — not right, but still helpful. After a few weeks with it, I can’t decide if this is the right balance. I haven’t felt the need to turn it off, but also, I’m not sure I’d miss it once it was gone.

My main reason for trying it out was that it feels obvious to me that this is the future of programming. I program mostly for myself, so I really want the productivity gain. It costs $100/year, and I think that’s a no-brainer, because it pays for itself pretty quickly in productivity gains. If you use VSCode and one of the languages it supports, I’d recommend trying it out.

In Praise of Pamphlets

One of the influential books in my life is A Technique for Producing Ideas [amazon affiliate link] by James Webb Young. I spoke about it at length in episode 13 of my podcast, where I called it a pamphlet.

I meant this in the sense of it being short and focussed. He doesn’t waste any of his 48 pages on background or fluff. It’s almost entirely about the technique.

I read a lot of (much longer) books in the same basic genre of this one — “A wise, old knowledge worker gives you their secret to doing knowledge work.” But probably because of the tyranny of the publishing industry, they have to be fleshed out to 300 pages, and so I have to read the same old Steve Jobs anecdotes again. Because pretty much any good idea about creativity, business, or productivity has a Steve Jobs anecdote.

We’re just lucky that Webb’s book predates Steve Jobs, but I don’t really remember him telling any anecdotes. There are no lessons from pseudo-science (or worse, non-reproducible, but compelling, real science). He is 100% betting that you don’t need to be convinced, or that the technique itself is convincing, and he rewards you by keeping it short.

Being American, I learned about the great patriotic pamphlets of the revolutionary era, like Common Sense, and so maybe I have a soft spot for this kind of work. In a way, maybe all pamphlets are common sense.

The First 13 Weeks

The first 13 weeks of the year ends on my birthday, and so each year I try to set up those months to try new things. It’s not a resolution or a goal, but more of a direction to explore. This year, I’m trying to make “make more art”, where “art” is very loosely defined. To start, I’m doing more sketching, writing, and I’m even going to drop in on an improv group.

I journal in a Recurring Journal I made last year for 2023 — it splits the year into four 13 week cycles where you journal the same day in the cycle on the same page spread. So, on April 3rd, I’ll be journaling on the same page as January 2nd. The idea is that I can see some progress and renew my energy for the next 13 weeks.

Lessons from My First Real Program

(I commemorated this program in a t-shirt)

In 1983, in the wood shop of JHS 125 in Woodside, Queens, perhaps sensing that my friend and I we’re acting up out of boredom, our teacher asked if we’d rather be in “computer shop”. Mr. Abbe had good instincts.

My new shop teacher, Mr. Penner, gave us each a sheet of graph paper and told us to count 80 columns and 25 rows and draw a picture by filling in squares. I drew a skull (I think it might have been getting close to Halloween). Next, he taught us enough BASIC so that we could write a program to draw our pictures on the screen.

I looked up an old Commodore PET manual, PETASCII table, a PET memory map, and downloaded VICE, a PET emulator, to recreate it here (lines 1010+ would have the coordinates from the drawing):

10 REM DRAW AN AWESOME SKULL!! 20 REM BY LOUIS FRANCO 30 READ X, Y 40 IF X = -1 THEN END 50 POKE 32768 + y * 80 + x, 160 60 GOTO 30 1000 REM (THE X, Y COORDINATES FOR THE SKULL) 1010 DATA 1020 DATA 2000 DATA -1, -1

What I love about this introduction to code:

It is pretty much the minimum code you can write to get a drawing on the screen.
It has conditionals and loops, so it’s more interesting than Hello World.
It takes advantage of something a 13 year-old might know — X, Y coordinates.
He gave us lines 30-60 because you have to know minutia of PET architecture to understand it. He did explain it, but didn’t expect us to be able to write it.

The next lesson was to animate the drawing (cycling two frames), which we were mostly on our own to do.

T-shirt showing a PET computer with a pixelated skull on it in green

Making More Things in Public

I really try to put most of what I make out there, whether it’s code in GitHub or random thoughts here on this site. But, the truth is that I have a lot of half-baked (tenth-baked?) stuff on my drive. Also, I am doing more work that has to be private, but I am actively looking to lower the bar and share more.

One class of things I don’t share is vector drawings that I make mostly for my apps or to practice graphic design. But, I really wanted to make my own t-shirts with some of them, so I submitted to Cotton Bureau and opened a store a few days ago. You can check them out on my t-shirt page. My favorite is PET Skull, which is based on my first real program. You can use the coupon [USP6ZBJXUS] for 10% off until the end of 2022.

Introducing Page-o-Mat

As I mentioned in my post about recurring journals, I decided to write a python program to create a PDF that implemented my riff on the Da Capo journal. I finished it enough to share, and put it on GitHub. It’s called Page-o-Mat.

To make it more generally useful, instead of hard-coding this particular journal, I made it read a configuration file that can be used to describe lots of journaling styles.

Right now, the paper and page types are what I needed for my 2023 journal. I am about to do a test-run with getting it printed on LuLu. After that, I might make some more page templates.

The config file is in YAML. You could just list each page you want one-by-one, but to make things easier, there are a few ways to create enumerated pages.

Any page could have a count, which just repeats that page
Any page can have a variant list, which repeats the page, setting a variable that can be used in titles
Pages can have lists of sub-pages, and so on.
Pages can have sections

In the code, it’s implemented with nested loops and recursion. The indices of those loops are exposed to the config to use in date math, so that any section of any page can have a date calculated for it. Most journals would probably have sequential dates, but a recurring journal wants you to keep cycling back to the beginning of the book, so the date math is fairly complicated.

There are documented examples in the repo that (hopefully) explain this better. Start with daily.yaml, which is just a 365 page daily journal. 2023-recurring-journal.yaml shows the more complex example.

Recurring Journals

A few weeks ago, I backed a Kickstarter for a ”Day-based Journal”. The idea is that each spread of two pages represents a day of the month. For example, when you open it up, the left page has boxes for Jan 1, Feb 1, March 1, April 1, May 1, and June 1. The right-hand-side has July 1 … December 1. So, to journal in January, you turn pages for 31 days and then go back to page 1 to start February. You go back to page 1 to start March, and you can see each day in January and February as you progress.

This has two effects:

You see your notes over and over (this reminds me of spaced repetition)
You can see monthly progress and eventually a years worth of progress

I love the idea and hope it gets funded, but in the meantime, I’ve started designing a riff on this idea that I will probably print out for 2023 on LuLu.

As a prototype, I drew something similar to the Kickstarter for 62 pages of my current blank journal to cover the rest of the year (Sep-Dec).

Prototype of a recurring journal showing Sep 1, Oct 1 on left hand page and Nov 1 and Dec 1 on right hand page — Prototype of a recurring journal

Since that’s just four months, each box is much bigger — half a page each month to make up the two-page spread. For me, I think that’s the minimum size. I don’t think I could use the orignal Day-based journal as a primary way to journal, since I need space to do a time block for the day.

The next thing I noticed was that each day was a different day of the week. That is mentioned as a plus in the Kickstarter, but I think in weeks. So, for me, it would be much more useful if each day shown on the spread were the same day of the week. My Mondays are very different from my Fridays, and the weekend is nothing like a weekday.

So I started to sketch out what a 2023 Week-based ”Recurring” Journal might look like.

My solution so far is to run 2023 as 4 sets of 13 weeks. That’s 364 days, and since Jan 2 is a Monday (and I start my week on Mondays), I’ll treat Jan 1 as a special day and then make 2-page spreads for the rest of the year where each box is 13 weeks apart.

One downside is that they are not aligned to months or quarters, but that’s just the nature of weeks, and I want to embrace weeks. I decided to further sub-group the 13 weeks as 4, 4, and 5 week blocks. That’s almost a month, and while it’s not ideal that they are uneven, I am also embracing the 13, so I have to accept its primeness.

Also, you are only revisiting your notes every 13 weeks instead of every month. To address this there will be 28 summary pages with 13 boxes where you try to get the essence of the day. To get 13 equal sized boxes, I’ll borrow from Betsy Ross and just use horizontal stripes. I think this would be the best way to use the original Kickstarter journal.

I’m currently building a spreadsheet where I can plan out what each page of the journal will be. In addition to daily pages, there will be pages for planning and retros. I also want enough blank pages for random note taking. So far it’s over 300 pages.

When that’s done, I’m going to build a python script to generate the PDF for the interior pages. I will open-source it, so if you are interested, then watch this space.