How a Model Picks Its Next Word
The Loop Is the Most Probable Continuation
Last timeCutting Off the Tail
Repetition is not a bug in the decoder. Under the model, a phrase already written is more likely to be written again, so the most probable continuation of a repeated phrase is another copy of it.
Ask a model for a long passage with greedy decoding and eventually it will begin
repeating a phrase, and then repeat it until it runs out of budget. This is
usually described as a decoder problem. It is easier to fix in the decoder, but
it does not start there.
The probability climbs
Measure what the model says about the repeated phrase as it repeats. The number
does not stay flat.
| step | times the phrase has appeared | probability of starting it again | probability of anything else | what happened |
|---|---|---|---|---|
| 1 | 1 | 0.14 | 0.86 | The first recurrence. Nothing unusual: the phrase is one plausible continuation among many. |
| 2 | 2 | 0.42 | 0.58 | Having seen it twice, the model now takes this to be a context in which the phrase recurs. |
| 3 | 3 | 0.78 | 0.22 | Past the point where any competing token can win an argmax. |
| 4 | 5 | 0.97 | 0.03 | Settled. The model is now nearly certain that the next thing is another copy. |
The model is not malfunctioning. Text in which a phrase has just appeared three
times really is text in which it is likely to appear again; lists, refrains and
boilerplate all behave this way. The model learned a correct fact about text and
the decoder walks into it.
The lesson stops here
6 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents