Generative Model That Won 2024 Nobel Prize

For most of history, computers were seen as purely logical machines, mechanically crunching numbers to produce rigid, unambiguous solutions. There was no place for creativity or ambiguity. After all, when calculating a trajectory to launch a rocket into space, the last thing you want is for your calculator to dream up some funky, non-existing formula or improvise on the spot. If you asked anyone whether a computer program would sooner master driving a car or composing a song, the answer would have been unanimous. Fast forward to 2024, however, and we still haven't quite achieved autonomous driving.

But the generative AI of all flavors is taken for granted at this point. So what sparked this shift? At what point did neural networks transcend mere deterministic computation and begin to create, synthesizing things that never existed before? Meet the Boltzmann machine, a type of neural network that dared to embrace chaos and change the course of AI forever. Developed in the 1980s, Boltzmann machines introduced a radical notion.

What if we built uncertainty and randomness into the very fabric of machine learning? What if, instead of storing rigid facts and performing deterministic computations, our AI could grasp the underlying probabilistic rules that govern the world around us? In this video, we will build a Boltzmann machine from first principles and explore how concepts of probability and inherent uncertainty can be reconciled with the seemingly rigid nature of computer operations. If you're interested, stay tuned. To understand Boltzmann machines, we must first understand their simpler predecessors, associative memory networks, also known as Hopfield networks.

We explored these concepts in depth in the previous video, so if you haven't seen it, I highly recommend watching it before continuing with this one, as we'll be directly building on those ideas. But here's a quick refresher: A Hopfield network is a model of associative memory, inspired by the brain's ability to recall complete patterns from partial or noisy inputs. It operates by assigning a specific energy value to each possible state and then iteratively minimizing this energy by descending along the energy surface into the nearest well, thus recalling the best matching stored memory. This energy landscape is shaped by network weights, which are learned by observing data points, patterns we want to memorize, and adjusting the weights to lower the energy associated with those patterns.

Given enough neurons, a Hopfield network has essentially perfect memory and excels at mechanical tasks like pattern completion. Think of it as a virtuoso classical musician who can recognize and flawlessly reproduce a well-known masterpiece from just a few initial notes. However, while impressive, a Hopfield network's ability to recall and complete patterns is limited to reproducing what it has explicitly learned. It cannot create new patterns or understand the underlying structure of the data it has seen. This is where Boltzmann machines come in, offering a more flexible and creative approach to information processing.

To illustrate the difference, let's extend our musical analogy. Imagine a jazz musician who has internalized not just specific songs, but also the fundamental rules and structures inherent to the music itself. When given a few opening notes, this musician doesn't simply recall and play an existing piece. Instead, they leverage a deep understanding of musical theory, combined with creativity, to improvise and produce something entirely new. This jazz musician represents a Boltzmann machine.

An associative network does not simply memorize data points. Instead, it learns the underlying probability distribution of the data, capturing the essence of what makes a pattern belong to a particular category or style, while incorporating inherent uncertainty into its computations. At first glance, these two systems might seem fundamentally different, with little in common algorithmically. In fact, they are very closely related. Just two key technical modifications can transform any Hopfield network into a Boltzmann machine: namely, stochasticity and hidden units.

Let's explore each of them in detail. We will first sprinkle in a dash of randomness and talk about how Boltzmann Machines earned their name. We begin in Austria, 19th century, where a young physicist, Ludwig Boltzmann, is grappling with a fundamental problem. Imagine a system of particles, like a gas. Each particle has its own energy, determined by factors such as its velocity.

We can measure the average energy of particles on a macroscopic scale by measuring the temperature. But what happens at the individual particle level? We might imagine that particles differ in terms of their exact energy values. Indeed, collisions can cause some particles to move faster than others, resulting in a range of energies. Boltzmann's quest was to understand this energy distribution.

In other words, if we randomly select a particle, what is the probability that it will have a specific energy value? Boltzmann's insight was to link a state's probability to its energy through an exponential relationship. Specifically, the probability of a state \( s \) with energy \( e \) is proportional to the exponent of the negative energy divided by temperature.

Intuitively, lower energy states are more probable than higher energy states, and this fundamental relationship quantifies exactly how much more probable. To understand why the exponent arises here, imagine energy levels as steps on a staircase, with particles jumping between them. Each step represents a small energy increment, epsilon. For a particle to move up one step, it must gain epsilon units of energy, perhaps through a collision with another particle. Let's call the probability of such a collision p. Given a large number of particles, this probability is essentially constant and depends only on average particle velocity or temperature.

If a particle jumps up one level with a probability \( p \), it might immediately jump again with the same probability. Since probabilities multiply for independent events, the chance of jumping two levels is \( p^2 \), three levels is \( p^3 \), and so on. We see a pattern: the probability of jumping \( n \) levels is \( p^n \).

The probability of jumping \( n \) levels is \( p^n \). Now, consider a particle increasing its energy by \( \delta e \). How many steps must it climb? Since the gap between the steps is constant, the number of steps is \( \delta e \) divided by \( \epsilon \). Thus, the probability of making this transition to a higher energy state is \( p^{\frac{\delta e}{\epsilon}} \). To present it in a more familiar form, let's repackage the different constants.

We can move the temperature dependency of \( p \) into the exponent and change the base to \( e \) (Euler's number), which is conventionally used in exponentials. Note that since \( p \) is less than 1 by the definition of probability, while \( e \) is greater than 1, this necessitates a minus sign before the energy in the exponent, since the temperature is always positive. Consequently, the probability of an energy increase of \( \Delta E \) is equal to the exponent of \(-\Delta E\) over temperature. Furthermore, in textbooks, you will usually find a version of this equation with the Boltzmann constant \( k \) in front of the temperature. This constant is used to convert the units of temperature, measured in degrees Kelvin, to energy, measured in joules.

But in this video, we will absorb the Boltzmann constant into temperature directly for brevity; we don't really care about the exact physical units. This equation gives us the relative probability of transitioning from one state to another as a function of the energy difference between them. But how can we find the absolute probability of a particular energy state? Here is what I mean. Consider the following toy example.

Suppose there are only three states our system can exist in, with energy values of 1, 2, and 3, measured in arbitrary units. Let's say the temperature is equal to 1. This equation tells us that finding the system in state 2 is \( \frac{1}{e} \) times as likely as finding it in state 1, which has lower energy. Additionally, finding it in state 3 is \( \frac{1}{e^2} \) times as likely compared to state 1.

But what about the absolute values of probability? Probabilities, rather than their ratios. We don't really know the baseline probability of state 1 in the first place. So how can we find it? The missing link here is that all absolute probabilities must add up to 1.

Indeed, the system is guaranteed to exist in one of the possible states. If we denote the absolute probability of state 1 as \( x \), we can express the probabilities of other states using \( x \) since we know their ratios. From this, we can write down the law of total probability. By doing so, we can solve for \( x \) and then find the absolute probabilities for all other states as well. This demonstrates how we can transition from relative probabilities of energy increases, as given by the Boltzmann formula we derived, to absolute values by solving the equation that includes the summation over all possible states. Let's plug the absolute energy values into the exponential formula, substituting \( \Delta E \) for \( E \) for now, and plot those relative probabilities as a function of energy.

We can plot the absolute probabilities that we found through the previous procedure as well. Notice that one shape looks like a vertically rescaled version of the other. This is a crucial insight. Since absolute probabilities must be proportional to relative transition probabilities, we can express the absolute probability of a state with energy \( e \) as the exponent of its negative energy that we found before, divided by some constant factor \( z \). This constant corresponds to the appropriate rescaling.

The value of \( z \) can be found by ensuring that the probabilities of all possible states add up to one. This normalization factor is known as the partition function. It takes into account all possible states and how energy is distributed across them. This is the complete and final version of the Boltzmann distribution, which links energy to probability. To use it, first look at all the possible states.

Of the Boltzmann distribution, which links energy to probability: to use it, first look at all the possible states and sum together their negative energies, obtaining the value of Z. Then, to find the probability of a system being in a particular state with a certain energy, compute the exponent of the negative of that specific energy and divide it by Z. Now that we have established the Boltzmann distribution, let's apply it to Hopfield networks to make them more stochastic. Recall that in Hopfield networks...

Each neuron updates its state deterministically based on its inputs. If the total input is positive, it turns on; if negative, it turns off. This corresponds to always moving to the lowest energy state available. Boltzmann machines, however, embrace uncertainty.

Instead of always choosing the lowest energy state, they make probabilistic decisions based on the Boltzmann distribution we derived. Here is how: A single neuron in our network acts as an eye. At a given update step, we essentially have two candidate states: the neuron being on or off, with the rest of the network remaining fixed. Using our definition of energy as the degree of conflict between weights and pairwise states, let's write down the energy for these two alternative states.

Here, the first term is the contribution of the edges of neuron \(i\) to the total energy, while the second term represents the energy contributed by the rest of the network, which is not affected by the state of neuron \(i\). Given these two alternative choices, we can express the probability of neuron \(i\) being on using the Boltzmann distribution for the case when there are only two possible states, which differ only by the value of neuron \(i\). Note that because we are taking the ratio, the energy term from the network not affected by neuron \(i\) cancels out. Therefore, the probability of this neuron's update is fully determined by its local connections. After dividing by the numerator, we can express the probability of switching on as a function of the energy difference gained by that update.

Now, let's examine the energy difference between those two states. From the definition, it is simply two times the weighted input to neuron i. Substituting this into our probability equation gives us the following formula. This is called the sigmoid function of the weighted sum of inputs. It tells us that when the input to a neuron is positive, the neuron is more likely to switch to the "on" state, with a higher probability for larger inputs.

When the input is negative, the probability of switching on decreases, approaching zero for very negative values of the weighted input. Our stochastic update rule thus becomes the following: First, calculate the weighted input for neuron i. Next, compute the probability \( p \) using the sigmoid function mentioned above. Finally, generate a random number between zero and one.

If that random number is less than the probability, set the neuron's state to 1; otherwise, set it to -1. This rule allows neurons to occasionally switch to higher energy states, with a probability that depends on the energy difference and temperature. At high temperatures, the decisions become more random, while at low temperatures, they approach the deterministic behavior of Hopfield networks. Temperature is usually a hyperparameter that we can adjust, depending on how creative we want the model to be. This stochastic rule is crucial for Boltzmann machines.

It allows the network to escape local minima in the energy landscape and explore a wider range of states, enabling it to learn more complex probability distributions and generate more diverse outputs. The random update rule is the key modification for inference in Boltzmann machines. But you might wonder, does this stochasticity also change how we learn and how we sculpt the energy landscape in the first place? Indeed it does, and as we'll see shortly, it leads to a fascinating concept known as the contrastive learning rule. In Hopfield networks, learning was straightforward.

We adjusted the weights to lower the energy of patterns we wanted to store. However, with Boltzmann machines, our goal shifts. Instead of memorizing specific patterns, we want to learn the underlying probability distribution of our data. Let's think about what this means. Ideally, as the network stochastically explores the landscape of possible states, we want it to spend more time in states that correspond to patterns in our training data because they are examples of what is realistic.

In other words, we want these states to have a higher probability. Recall the Boltzmann distribution, which links the probability to energy. According to this formula, to increase the probability of a state, we need to lower its energy relative to other states. But here's the catch: changing the energy of one state directly also affects the partition function \(Z\), which depends on the energies of all other possible states.

This interplay leads us to a new learning objective. We want to maximize the probability of these states corresponding to our training data while accounting for the overall distribution of states the network can reach. We are going to need a new learning rule based on probability rather than energy per se, so let's derive it from scratch. Remember, the ultimate goal is to maximize the probability of our training data under the model. Let's say we have a set of training patterns, \( x_1 \) through \( x_n \).

We want to maximize their joint probability, which is the product of probabilities assigned to each individual example. It is often easier to work with sums rather than products. So, let's take the logarithm of both sides. Since the logarithm is a monotonic function, maximizing the probability is equivalent to maximizing its logarithm.

Now, let's express the probability of each pattern with its energy using the Boltzmann distribution. Expanding this according to the properties of the logarithm gives us a crucial insight. To maximize the log probability of our data, we need to simultaneously minimize the energy of our training patterns and minimize the partition function. The first part makes intuitive sense; we want our training patterns to sit in deep energy wells.

But why minimize \( z \)? Remember, the partition function sums over all possibilities and possible states. By minimizing it, we are effectively increasing the energy of states that are not in our training data. This prevents the network from assigning low energy to too many states, which would dilute the probability of our desired patterns.

It essentially creates two opposing forces. One is digging energy wells around the desired data, while the other is pulling the energy surface up for the undesired data. To derive the learning rule from this, we can take the derivative of the log probability with respect to a given weight and then make iterative adjustments to the weights to maximize it. I don't want to overwhelm this video by taking derivatives and shuffling symbols around. If you're interested in this step-by-step derivation, I will make an extended version of the script with all the math details available to my Patreon supporters.

But after you go through the math, you will arrive at what is known as the Contrastive Hebbian Learning rule. The interpretation of it is really elegant. The first term is the average product of states, x_i and x_j, when the network is exposed to the training data. This is known as the Hebbian term, and it is directly analogous to what we observed in Hopfield networks.

It strengthens connections between neurons that are often active together in the training data. The second term is the average product of those two neurons when the network is running freely. This is what we will call an anti-Hebbian term. Notice that it is taken with a minus sign. Effectively, what this is saying is we want to make sure the weights do not reinforce images generated during dreaming.

That are far away from the training examples. This rule is called contrastive because it contrasts the behavior of the network when it is constrained by the data versus when it is daydreaming on its own. It lowers the energy of data patterns while also capturing the underlying probability distribution, allowing for both accurate recall and creative generation. In practice, to obtain the first term of the probability, we simply go through each training example, examine the pairwise products between a pair of neurons, and adjust the weight between this pair in proportion to the average. But what about the anti-Hebbian term? How can we let the model hallucinate? Essentially, running freely here means allowing the network to evolve according to its update rule without any external input. Here is how we do it.

First, start with a random configuration of the network. Then, repeatedly update the states of all units according to the stochastic update rule. Continue this process for many steps, allowing the network to reach its equilibrium distribution. Once at equilibrium, examine the pairwise states for each pair of connected neurons. Repeat this process multiple times and take the average.

Back in the case of Hopfield networks, we had an explicit formula for the weights as a function of training patterns, allowing us to set them instantaneously. One major difference for Boltzmann machines is that learning is no longer instantaneous. Instead, it involves an iterative procedure, and the stochastic update rule is applied many times in order to iteratively find better weights, not just for inference. This learning process alternates between two phases.

The positive phase, where we set the neurons to encode the training patterns and compute pairwise state products \(x_i \times x_j\), and the negative phase, where we allow the network to run freely to compute \(x_i \times x_j\). We then update the weights according to this formula. This process is repeated many times over the entire training dataset. Gradually, the network learns to shape its energy landscape so that the valleys correspond to patterns in the training data and the peaks correspond to unrealistic examples, capturing the uncertainty in the underlying distribution that generated that data. Great!

But so far, we have explored networks with only visible units—neurons directly in the network encoding the data. To truly harness the stochastic power of Boltzmann machines, we need one final architectural modification: the addition of hidden units. Essentially, hidden units are neurons that don't directly correspond to any part of the input or the output. Instead, they serve as the model's internal representation, capturing abstract features and higher-order correlations in the data that are not immediately apparent in the visible units alone.

Implementing hidden units is straightforward. We simply increase the number of neurons in the network, designating some as visible and others as hidden. The number of visible units usually corresponds to the data's dimensionality. For instance, a 32x32 pixel image would require 1,024 visible neurons, one for each pixel. The number of hidden units, however, is a design choice and can be arbitrarily high.

Importantly, while there is a conceptual distinction between visible and hidden units, the network treats them identically in terms of the update rule. It computes weighted inputs and performs stochastic updates on one neuron at a time, regardless of the type. You might wonder, if setting weights requires known states from the training data, how do we handle the weights involving hidden units, whose correct states are never directly observed? This is where the elegance of the contrastive learning rule shines.

The weight adjustment, which is an iterative procedure, looks like this: In the positive phase, we clamp the visible units to a training pattern and allow the hidden units to update freely using our stochastic update rule. After reaching equilibrium, we measure the product of \( x_i \) and \( x_j \) for all unit pairs, including those involving hidden units. In the negative phase, we let all units, both visible and hidden, update freely, starting from a random configuration. We then update all weights, including those connected to hidden units, using our contrastive update rule.

This process enables the network to learn appropriate states for hidden units that capture the data structure without explicitly specifying what these states should be. Over time, hidden units develop representations that capture important data features. The network learns through optimization to leverage these hidden representations to better model the training data's probability distribution. Before we conclude, let's briefly touch on what is called Restricted Boltzmann Machines, or RBM. Essentially, it is a modification of what we discussed today, but where connections between visible units or between hidden units are prohibited.

Only connections between visible and hidden units are allowed. This restriction might seem limiting, but it actually offers a significant advantage. It allows for parallel updates of all units in a layer. In a standard Boltzmann machine, we update units one at a time because each neuron's update depends on every other neuron.

In an RBM, all visible units can be updated simultaneously given the states of all hidden units, and vice versa. This parallelization dramatically speeds up both learning and inference. Despite the connectivity restriction, RBMs retain much of the expressive power of full Boltzmann machines while being much more computationally efficient. This efficiency made Restricted Boltzmann Machines practical for many real-world applications. Alright, let's try to tie everything together.

In this video, we have seen how Hopfield networks, which can store and recall specific patterns, can be modified for more creative problems involving the generation of new data. In particular, we explored how incorporating randomness into the update rule, governed by the Boltzmann distribution, and rephrasing the learning objective in terms of maximizing the probability of training data leads to the creation of a powerful generative model known as the Boltzmann machine. This stochastic approach, combined with hidden units, allows Boltzmann machines to learn and capture the underlying probability distribution of the training data, rather than simply memorizing specific patterns. By detecting abstract hidden features, these machines possess the ability not only to recognize but also to understand and generate data. This capability makes Boltzmann machines a crucial stepping stone in the development of modern machine learning.

And while in practice, they have been largely replaced by more advanced models such as multi-layered networks trained through backpropagation, the underlying principles of modeling uncertainty and learning abstract features form the foundation of even the most recent generative AI systems. Speaking of abstract understanding as opposed to mere memorization, I would like to thank the sponsor of today’s video. Shortform is an innovative platform that transforms how we engage with books and other information-dense content. Shortform goes beyond traditional summaries by offering in-depth book guides that provide a comprehensive understanding of the material. Along with a summary of the main points, which is usually more detailed than what you might find on other platforms, Shortform guides contain multiple references and explain ideas from relevant sources, such as other books or research articles.

It's like having a knowledgeable reading companion who highlights the most crucial insights and shows you how they fit into a broader context. Shortform's rapidly growing library of books covers a wide range of titles and topics, such as science, technology, and education. They also have an impressive browser extension that can generate similar guides for virtually any online content you encounter. Don't hesitate to supercharge your reading by clicking the link down in the description to get five days of unlimited access and 20% off an annual membership.

If you liked the video, share it with your friends, press the like button, and subscribe to the channel if you haven't already. Stay tuned for more topics on computational neuroscience and machine learning coming up. Thank you!