Solving Bach's last Fugue

 

One of classical music's great mysteries is J.S. Bach's unfinished piece in The Art of the Fugue musical work; specifically the Fuga a 3 Soggetti piece which ends rather abruptly. Probably many authors, artists or composers leave unfinished work before they're ready to meet their maker, but what leaves this specific unfinished work even more puzzling is that the author, J.S. Bach, finished it with the musical motif B-A-C-H  (B♭–A–C–B♮ is the english notation). Did Bach intentionally leave his name as the last motif just so he can laugh in his grave while we, centuries later, still squeeze our minds thinking about it? Hard to say, but it did indeed inspire many composers after him to try to finish the work, more than 80 attempts in fact [1].

Since Bach is famous for his mathematical precision in composition, this blog will approach completing the fugue as a mathematical puzzle to solve. I will take a probabilistic approach using Markov chains, try state of the art AI models and leave you, the reader, to decide which "sounds" more like Bach. This blog aims to illuminate the inherent beauty of mathematics while critically examining the true potential—and the limitations—of modern AI.

What is a fugue

At its essence, a fugue is a musical composition technique where the composer explores a melodic idea by a definite number of voices in imitation. You can tell a fugue by hearing multiple "voices" at the same time revolving around a theme. It might sound argumentative sometimes, but gives music pieces a whole new depth. Watch this video if you're interested to know more about fugues.

A key sentence in the definition of a fugue is "having multiple voices independent of each other". These voices are rhythmically and melodically independent, but are dependent harmonically which results in the complex feel of a fugue. This is the key feature we will use to attempt completing or at least extending the fugue.

Breaking the problem down 

What inspired solving the problem initially was watching 3Blue1Brown's Riemann zeta function video about analytical continuation, wouldn't be beautiful if the musical piece is a smooth time-series and we can extrapolate? This idea quickly disappeared as one might expect, in a time-series, the different voices of fugue mix together and we are making hard to solve.

To simplify the problem, we will switch from the analytic domain to the logic domain, utilising MIDI files as it breaks music into discrete notes and instruments. In Bach's unfinished fugue, there are 4 voices (or instruments) that we will solve independently as they are melodically and rhythmically independent of each other. Visualising MIDI file of one of the voices in the fugue looks like this:

Each note has a pitch (resolution of 7 bits) start and end time, and lastly velocity, which you can think of as how hard you press the note. These will be our variable to use to try to solve it.

Markov chains

Markov chains are ubiquitous, used everywhere from robotics to predicting stock markets. Markov chains are useful when a probability of an event is linked to the current "state" of the system [2]. In our case, the next note is certainly a function of not only the the note before, but a number of notes before. 

 The key ingredients of Markov chains are states and transition matrices that are typically computed beforehand by learning from the data. In its simplest form, we can use it to predict the weather as in this example:

Figure 1: A simple Markov chain of 2 states, Sunny and Rainy [3]

In this system [Figure 1], we have two states; sunny and rainy. For example, if the current state is sunny, there is an 80% probability that the next step (or day) is going to be sunny and 20% rainy. 

In our case, we will create a state that encodes the information that we care about. Since our data are in the form of a MIDI file, a state can be a tuple of pitch, duration and time step. Pitch in this case will encode melody, while rhythm is encoded in duration and time steps. the information that we care to capture is melody (pitch) and rhythm (note duration and step time). 

Typically, creating a transition matrix for for 127 different pitches and 12 order Markov chain will cost us dearly in memory, in fact it might be impossible with today's technology. A practical approach is to compute the next note by using lazy evaluation techniques, rather than making a transition matrix.

AI models

It is only natural nowadays to include the word AI in every blog. It is even more relevant in our case as there are so many music generation tools to choose from. I decided for this comparison to choose the best MIDI note generator tool. Magenta Studio stood out for multiple reasons; A) It is developed by Google Brain, the team that developed the transformer architecture that arguably started the AI revolution and B) it has a specific tool called Continue, which as the name implies, continues a melody for a specified length of bars. It uses RNN architecture to capture as much information from the past to predict the future.

To be fair, Magenta Studio is considered a bit old for an AI timeframe and it is not really maintained anymore, but it seems to be the only tool do MIDI to MIDI operations, the rest are mostly text-MIDI. Also to be fair, Markov chains was invented more than 100 years ago.

Results

You can listen to Glenn Gould work on the Art of the Fugue here. I will only show the last few seconds before the piece ends abrubtly. We will compare original work to Markov chain's attempt and Magenta Studio's Continue tool. 

Original

With Markov chain with variable order, I set maximum order to be 12 but the average seems to be around 10.

Magenta Studio Continue tool

Conclusion

It is probably clear which one sounds better to the average listener, now which one would Bach have composed, that is difficult to answer. I would like to include a few resources that inspired me to do this work. First, reading Gödel, Escher, Bach: an Eternal Golden Braid by Douglas Hofstadter, it explores these ideas and many, many more in more depth. 

A more recent inspiration is a video by 3Blue1Brown about mathematically analysing and recreating Escher style in images.

 Also, credit to Gemini helping me brainstorm and writing the algorithm. The function used to generate Markov chain continuation is available on my Github repository

References

[1] https://researchspace.auckland.ac.nz/items/d8c40a5f-37dc-4247-b1b8-a481d588aef3

[2] https://www.statslab.cam.ac.uk/~rrw1/markov/M.pdf#page=4.02

[3]  https://bookdown.org/probability/beta/markov-chains.html