Answers · How models work
Why does diffusion add Gaussian noise instead of any random corruption?
In shortDiffusion adds Gaussian noise because Gaussian errors combine exactly: each step keeps the spread fixed and gives a closed-form noisy picture at any later step, which makes the reverse step learnable.
Diffusion adds Gaussian noise because Gaussian errors combine exactly. Each forward step is a linear shrink plus independent Gaussian error, so the noisy picture at any later step has a closed-form Gaussian distribution with a known mean and variance, and the reverse step becomes a Gaussian conditioning problem. That is the property any other corruption loses.
The spread has to stay put
Write . If the values at step have variance 1, the next step has variance . Take , so . The variance after one step is . The shrink is doing exactly enough to cancel the fresh noise. Drop it and use : after 100 steps the variance is . The path would then end somewhere that depends on how long it was run, and a sampler would not know which spread to start from. The Gaussian error is what lets this variance arithmetic be exact: multiplying by a constant scales variance by the square of that constant, and adding independent noise adds its variance.
Now apply the rule twice with the same . The coefficient on is . The first step contributes noise variance to the second step, the second contributes , and the two independent Gaussian errors add to one Gaussian error with variance . That is exactly , so two steps have the same form as one step with a combined . For a schedule, the product does the same job, and the whole chain collapses to one formula.
The natural guess is that any corruption would do: add uniform noise, or flip a random pixel, and train a network to undo it. That loses the closed form. Add independent uniform noise instead. After one step the values have a flat-topped distribution; after two steps the sum of two uniforms is triangular; after many steps it only approaches a bell shape. The variance still adds, but the distribution at step is not given by the one-line Gaussian formula, so producing a training pair at an arbitrary means running the chain rather than sampling directly from . Flipping pixels is worse: it is not additive at all, so there is no variance recursion and no fixed point for the spread.
Why the reverse is a Gaussian problem
Because is multiplied by a number plus independent Gaussian error, the pair is jointly Gaussian once is fixed. Conditioning one Gaussian on another gives a Gaussian. So has a closed-form mean and a variance built from and the accumulated . A network only has to predict the missing or the noise ; the rest of the reverse step is arithmetic fixed by the schedule. With uniform noise or pixel flips the reverse conditional has no such summary: the network would have to learn the whole shape of an arbitrary conditional distribution, not one mean and one known variance.
Where it stops
The exact composition survives any schedule, but the reverse transition is a mixture over possible values and is not itself exactly Gaussian. The Gaussian formula is exact for , and sampling uses a network to approximate the mixture. The approximation is best when each is small, because each reverse step then stays close to a single Gaussian. If is large, one reverse step has to undo a lot and the local Gaussian picture is poor. If the noise is correlated across pixels, the algebra still works with covariance matrices, but the isotropic formula and the simple variance fixed point do not. If the noise has infinite variance, the fixed point argument fails altogether.
Worth keeping
- Gaussian noise is the corruption whose sum stays Gaussian, so many small forward steps collapse to one closed-form distribution.
- With the shrink, a spread of one is a fixed point; without it, variance grows by beta per step.
- Any other corruption can still be undone by a network in principle, but it loses the exact marginal and the simple Gaussian reverse step.
Where this goes next
This question is one step of How Noise Becomes a Picture, a course that works through its subject from the beginning. The page that teaches this part of it is Destroying a Picture on Purpose, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- How do you turn the predicted noise into the mean of the previous image?Subtract a small scheduled fraction of the predicted noise from the current image, then rescale to undo the forward step's shrinkage.
- Why does the forward noising process have to be a Markov chain?The forward process is made memoryless so that each step is a fixed random rule and any later state can be computed straight from the original picture.
- What does classifier-free guidance actually add together, and why does it make the prompt stronger?It adds the unconditional prediction to several times the gap between the conditional and unconditional predictions, pushing past the conditional answer so the prompt is obeyed more strictly.
- Why is the model's noise prediction really the score of the noised image distribution?The network's noise prediction, divided by the noise level and given a minus sign, is the direction that makes a noised image more plausible.
- Why does the model get stuck repeating the same sentence over and over?The model is not stuck in a memory loop: once it has written a phrase, that phrase is in its context and makes writing it again more likely, so a decoder that always picks the top word cannot leave.
- Why does the same request cost more tokens in some languages than in others?A token is learned from a corpus, so a language that was rare in the training text is split into many more pieces for the same meaning, and the price multiplier follows.