Why does attention use softmax instead of just dividing each score by the sum of all the scores?

Dividing each score by the sum of all the scores normalises magnitude but does not make attention weights safe. Scores are dot products and can be negative, so that division can produce a negative weight or divide by zero; softmax exponentiates first, which forces every weight positive and turns score gaps into fixed ratios.

Those three weights do sum to one: $1+0.5-0.5=1$. But the third weight is negative, so the third token's value would be subtracted rather than attended to. The failure is not a rare edge case. If the scores are $(2,-1,-1)$, the sum is $2-1-1=0$, and dividing by the sum is undefined before any weight exists.

All three softmax weights are positive and their sum is $0.705+0.260+0.035=1.000$. The first weight divided by the second is $0.705/0.260=2.71$, which matches $e^{2-1}=e=2.718$. Dividing the original scores gives $2/1=2$ for that same pair. The exponential does not merely rescale all scores by a common factor; it fixes the exchange rate between a score gap and a weight ratio. A gap of one gives a ratio of $e=2.718$ whether the scores are $(2,1)$ or $(20,19)$. Linear division has no such fixed exchange rate: $20/19=1.053$, while $2/1=2$, for the same one-point gap.

The natural guess is that any normalisation making the weights sum to one would behave the same. The table shows why it does not. Dividing by the sum fails non-negativity as soon as a score is negative. Hard argmax is non-negative and sums to one but has zero gradient almost everywhere, so no learning signal reaches the query and key projections. Squaring removes the sign problem but maps $-3$ and $3$ to the same weight, destroying the ordering the scores were computed to express. Among the obvious candidates softmax is the one that satisfies all four requirements at once. More strongly, any other function satisfying all four is softmax with the score scale changed, so the shape is pinned down and only the exchange rate between score gaps and weight ratios remains free.

Adding a constant $c$ to every score leaves softmax unchanged, because the factor $e^c$ appears in both the numerator and the denominator and cancels. That is not cosmetic. A causal mask needs forbidden positions to receive exactly zero weight. Set their scores to $-\infty$ before the softmax: $e^{-\infty}=0$, those positions drop out of the denominator, and the remaining weights renormalise by themselves. With scores $(1,2,-\infty)$ the exponentials are $2.718$, $7.389$ and $0$, the total is $10.107$, and the weights are $2.718/10.107=0.269$ and $7.389/10.107=0.731$, with the forbidden position at zero. Dividing by the sum cannot express this: $1+2+(-\infty)$ is $-\infty$, so the normalisation is undefined rather than exactly zero.

Softmax always returns a full distribution. If no token is relevant, the head still has to put its entire weight somewhere, because the weights sum to one and there is no abstention option. Trained models work around this by learning an attention sink, usually a large share on the first token, whose value contributes little. That behaviour is a consequence of the normalisation, not an extra rule. If all scores were guaranteed positive, divide-by-sum would at least satisfy non-negativity, but it would still lack shift invariance and its ratios would depend on the absolute score level. Softmax is not unique as a formula: changing the base or multiplying every score by a constant changes the exchange rate while preserving the four properties. The four properties fix the shape; the scale remains a modelling choice.