Basis Lab

Answers · How models work

Why doesn't temperature scaling change which answer the model picks?

In shortTemperature scaling changes how confident the model sounds, not which answer it ranks first, because dividing every score by the same positive number preserves their order.

Temperature scaling divides every score by the same positive number before the softmax is taken, so the largest score remains the largest and the model's top answer cannot change. It changes the confidence printed beside that answer, not the ranking that selects it.

pi=ezi/T∑jezj/T,T>0p_i = \frac{e^{z_i/T}}{\sum_j e^{z_j/T}}, \qquad T>0

Write the raw score for option ii as ziz_i and the shared temperature as T>0T>0. The reported probability is the usual softmax with ziz_i replaced by zi/Tz_i/T. A value of TT above one compresses the scores toward zero, so the distribution flattens; a value below one stretches them away from zero, so the distribution sharpens. Neither operation deletes a score or changes its sign relative to another score.

zi>zj⟹ziT>zjT⟹ezi/T>ezj/T⟹ezi/TS>ezj/TSz_i > z_j \quad\Longrightarrow\quad \frac{z_i}{T} > \frac{z_j}{T} \quad\Longrightarrow\quad e^{z_i/T} > e^{z_j/T} \quad\Longrightarrow\quad \frac{e^{z_i/T}}{S} > \frac{e^{z_j/T}}{S}

The denominator S=∑kezk/TS=\sum_k e^{z_k/T} is positive and identical for every option, so dividing by it cannot reverse an inequality either. If ziz_i and zjz_j were equal, their exponentials remain equal and their probabilities remain equal. That is the entire proof: the softmax output is a strictly increasing function of each score once the common normaliser is fixed, and a strictly increasing function preserves every pairwise comparison.

FIG 1
optionscore
A20.6650.5060.867
B10.2450.3070.117
C00.0900.1860.016

Read down any column. At T=1T=1 the leader has probability 0.665 and the runner-up 0.245. At T=2T=2 those become 0.506 and 0.307: the gap has narrowed, but the leader is still ahead. At T=0.5T=0.5 they become 0.867 and 0.117: the gap has widened, but the leader is still the same option. The third option stays last throughout. Since the model's answer is the argmax, and the argmax is unchanged, every top-one decision is unchanged. The same holds for a top-five list or any other decision that depends only on the ranking.

The tempting picture is that rescaling probabilities is like turning a volume knob on each option separately, so a loud runner-up could overtake a quiet leader. Temperature does not do that. It applies one positive divisor to every score before the exponential, and the exponential is monotone. A lower temperature can make the leader's 0.665 look like 0.867; it cannot turn that lead into a deficit. A higher temperature can make the same leader's 0.665 look like 0.506, but it cannot put the other option first. The only way to reorder options with a temperature-like parameter is to let the divisor differ between options, or to add a class-specific offset after the scores, and then it is no longer the single shared temperature that repairs overconfidence.

What still moves

The answer is unchanged, but the number attached to it is not, and that number can still change an action. If a system acts only when the reported confidence exceeds a fixed cut-off, then temperature scaling can move the same top answer above or below that cut-off without changing which answer it is. That is a decision about whether to act, not about which class wins. It is also why temperature is fitted on held-out data: the fitted value is chosen to make the reported probabilities match observed frequencies, and on training data the search is pulled toward certainty rather than honesty. The ranking property holds for any positive TT, fitted or not.

The property fails as soon as the correction stops being one shared positive scalar. Per-class temperatures, class-specific biases added to the scores, or a post-hoc adjustment that treats options differently can all reorder the list. A negative temperature would reverse the order, which is why T>0T>0 is part of the definition rather than a detail. At T=0T=0 the formula is undefined, and the limiting behaviour sends all mass to the highest score, with ties needing a separate rule. Within the intended use, a single positive TT shared by every option for a given input, the top answer is invariant; if the decision uses a probability threshold rather than a ranking, the action can still change even though the answer does not.

Worth keeping

  • Dividing every score by the same positive number preserves every pairwise comparison, so the top-ranked answer cannot change.
  • Temperature scaling changes the reported confidence, and a fixed confidence threshold can therefore change whether a system acts, even though the chosen answer stays the same.
  • The invariance requires one positive temperature shared by all options; per-class temperatures or class-specific offsets can reorder the list.

Where this goes next

This question is one step of When a Model Is Sure and Wrong, a course that works through its subject from the beginning. The page that teaches this part of it is One Number That Fixes Most of It, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. Why can a completely made-up answer come with high confidence?Because the confidence score measures how likely the words are, and a fluent invention is built from likely words, not because the fact was checked.
  2. When a model says it is 90% sure, what is that 90% a probability of?It is the share of the model's score mass given to that option out of the options it was offered, not the chance that the answer is correct.
  3. When should a model refuse to answer instead of guessing?Refuse when the chance of being wrong times the cost of an error is higher than the cost of handing the case to a person.
  4. Why can't I fit temperature on the same data I trained the model on?Because training predictions are already too confident, fitting temperature there confirms that confidence instead of correcting it.
  5. Why does the model get stuck repeating the same sentence over and over?The model is not stuck in a memory loop: once it has written a phrase, that phrase is in its context and makes writing it again more likely, so a decoder that always picks the top word cannot leave.
  6. Why does diffusion add Gaussian noise instead of any random corruption?Diffusion adds Gaussian noise because Gaussian errors combine exactly: each step keeps the spread fixed and gives a closed-form noisy picture at any later step, which makes the reverse step learnable.

Read alongside