When you read, your visual senses are completely flooded. You should also be engaging in Slow Thinking (System II type thinking) as described in Thinking, Fast and Slow. This is self-aware, effortful thinking.

You are generally not capable of directing slow thinking at two tasks, which is why you shouldn’t text and drive. And it’s why you can’t read and listen to a podcast.

But what if you could listen to something? What ever the book is for, could your audio senses augment what you are reading?

I am not just talking about an embedded audio clip. To listen to that, you’d stop reading. I am talking about a book’s soundtrack.

In a movie, the soundtrack is integrated to provide a deeper experience. Sometimes it’s meant to be as prominent as the foreground, and other times, it might not be consciously noticed.

Movie visuals and audio are synchronized. So, to do this, the reading device would need to know exactly what you were reading at any point. In this sense, it’s more like a video game soundtrack, which is also synchronized, but has to follow the player’s actions.

Let’s assume that’s possible. Given a book device that knows exactly what you are reading at any point and can produce sound, here are things it could do:

1. If you stop and stare at a word or phrase that is jargon the book defined, it quickly reminds you of the definition.
2. Like Peter and the Wolf, we could assign small musical themes to the major ideas of the book. When you are reading something that is related to one of those ideas, the theme would play.
3. It could use generated sound or music that goes from calm to more of crescendo as you progress through a chapter, giving you a sense of how close you are to the next break. For example, rainfall that becomes more of a storm—an audio progress bar.
4. I think fiction would use this more for entertainment/art, but one exception is trying to read Shakespeare. I remember the left-hand side page having contextual information for the script on the right-hand side—maybe the context could be delivered aurally while you read.

If we could do this, it would drive book design to take sound into account (rather than it just be guessed at by a device) and eventually evolve the medium further away from text-only books.