Answers · How models work

Why does the model get stuck repeating the same sentence over and over?

In shortThe model is not stuck in a memory loop: once it has written a phrase, that phrase is in its context and makes writing it again more likely, so a decoder that always picks the top word cannot leave.

The loop is not a bug and it is not a looping memory. Once a phrase has been written, it is part of the context, and the model gives a higher probability to writing it again; a decoder that always takes the top token then has no reason to leave.

The probability climbs

Measure the model's next-token distribution as a phrase repeats and the number does not stay flat. On the first recurrence the repeat has probability $0.14$ and everything else has $0.86$ . On the second it is $0.42$ against $0.58$ . On the third it is $0.78$ against $0.22$ , and on the fifth it is $0.97$ against $0.03$ . These are model probabilities, taken before any decoder has chosen anything.

FIG 1The model probability of continuing a loop, measured across repeats

step	times the phrase has appeared	probability of starting it again	probability of anything else	what happened
1	1	0.14	0.86	The first recurrence. The phrase is one plausible continuation among many.
2	2	0.42	0.58	Having seen it twice, the model now takes this to be a context in which the phrase recurs.
3	3	0.78	0.22	Past the point where any competing token can win an argmax.
4	5	0.97	0.03	Settled. The model is nearly certain that the next thing is another copy.

4 steps

The direction is the point: each repetition is evidence for the next one, so the loop is self-reinforcing in the model before any decoder has made a choice.

\frac{0.14}{0.86}\approx 0.16,\quad \frac{0.42}{0.58}\approx 0.72,\quad \frac{0.78}{0.22}\approx 3.55,\quad \frac{0.97}{0.03}\approx 32.3

Those are odds, not probabilities. The odds of repeating rather than not repeating rise from about $0.16$ at one appearance to $0.72$ at two, which is $0.72/0.16\approx 4.5$ times larger. From two to three they rise to $3.55$ , about $3.55/0.72\approx 4.9$ times larger again. From three to five they reach $32.3$ , about $32.3/3.55\approx 9.1$ times larger again. Each copy is evidence for the next copy, so the loop is self-reinforcing inside the model.

The natural guess is that something is stuck: a bug, a cached state, a memory of the last sentence. There is no separate loop variable to reset. The repeated text is the context, and changing that context is what changes the distribution. If you cut the loop out of the context, the next-token probabilities change immediately; if you leave it there, the model is doing exactly what it was trained to do, because refrains, lists and boilerplate really do recur in text.

Why greedy cannot leave

Greedy decoding takes the highest scoring token at every step. When the repeat has probability $0.78$ , every other token together has $0.22$ , so no single alternative can beat it: $0.78>0.22$ , and therefore $0.78$ is larger than any one of them. Greedy repeats. Beam search does not rescue this, because every candidate in the beam is a variant of the loop and the loop scores best. Escaping requires choosing a token that is not the top one, which is the one thing a deterministic decoder will not do.

A sampler can leave because it can take a non-top token. If the escape probability at each step is $q$ , the chance of still being in the loop after $n$ steps is $(1-q)^n$ , so the chance of having escaped at least once is $1-(1-q)^n$ . With $q=0.03$ , after 20 steps the escape chance is $1-0.97^{20}\approx 0.456$ , and after 60 steps it is $1-0.97^{60}\approx 0.839$ . With $q=0.001$ , after 60 steps it is only $1-0.999^{60}\approx 0.058$ . A tight top- $p$ or a low temperature does not create the loop, but it can discard the token that would have escaped, which is why the same model loops under one configuration and not another.

FIG 2The chance of having escaped a loop, by the number of steps taken

three percent escape per stepone tenth of a percent per step

The upper curve is a looser sampler. The lower one is tightly truncated and behaves almost like greedy: the escape route was among the tokens the truncation rule discarded.

What a penalty does

The usual remedy subtracts a constant $c$ from the score of every token that has already appeared. In the softmax, a score $z$ becomes proportional to $e^z$ . Subtract $c$ and the seen token's weight becomes $e^{z-c}=e^{-c}e^z$ . Against an unseen token with score $w$ , the odds change from $e^z/e^w$ to $e^{z-c}/e^w=e^{-c}e^z/e^w$ . The factor is $e^{-c}$ , the same for every seen token.

\frac{e^{z-c}}{e^{w}}=e^{-c}\,\frac{e^{z}}{e^{w}}

With $c=1.2$ , $e^{-1.2}\approx 0.301$ , so a seen token keeps about 30 percent of its previous odds. That is enough to break many loops. The rule still cannot tell a refrain from the word "the". English needs the same function words in nearly every sentence, and they are exactly the tokens that have already appeared. Turn the penalty up and the text starts avoiding its own vocabulary: correct words are replaced by odd synonyms, and names or numbers that genuinely need repeating begin to drift.

The loop is a property of the model plus a decoder that refuses to take a risk. If the repeated phrase is legitimate, as in a list or a refrain, the model is right and the decoder is merely obeying it. If the sampler is wide enough, the loop is finite; if the decoder is greedy or tightly truncated, it is permanent. A penalty changes the scores but not the fact that the context now supports the repetition, so it can suppress a loop while damaging words that must recur. That is the boundary: the model supplies the loop, and the decoder decides whether it becomes permanent.

Worth keeping

A repeated phrase raises its own probability on the next step, so repetition is self-reinforcing in the model before any decoding choice is made.
Greedy and beam search cannot leave a loop because leaving needs a token that is not the highest scoring one.
A repetition penalty multiplies the odds of every seen token by the same factor, so it suppresses the symptom and also penalises words that must recur.

Where this goes next

This question is one step of How a Model Picks Its Next Word, a course that works through its subject from the beginning. The page that teaches this part of it is Why It Gets Stuck, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course