Answers · The mathematics underneath

Why does softmax followed by cross entropy have gradient p minus y?

In shortBecause the derivative of the loss through softmax simplifies to the predicted probability minus the target, one number per class, with the exponentials and fractions cancelling out.

The gradient of the pair, taken with respect to the scores that go into the softmax, is $\bar z_i = p_i - t_i$ : the predicted probability minus the target, one number per option. Neither the exponentials nor the division of the softmax survives into the answer, and there is no special case for the position that happens to be correct. What arrives back at the last layer is a subtraction whose size is the error.

Two options, by hand

Two scores, $a$ for the correct option and $b$ for the other, is the smallest case that shows the whole thing. The loss is the negative logarithm of the first probability, and splitting that logarithm leaves an expression with no fraction in it.

L = -\log\frac{e^{a}}{e^{a}+e^{b}} = -a + \log\left(e^{a}+e^{b}\right)

Differentiate against $a$ . The $-a$ contributes $-1$ . The logarithm contributes the derivative of its argument divided by the argument, which is $e^{a}/(e^{a}+e^{b})$ , and that is exactly $p_1$ , the probability the model put on the correct option. So the derivative against $a$ is $p_1 - 1$ . Differentiate against $b$ and the first term contributes nothing, while the second contributes $e^{b}/(e^{a}+e^{b}) = p_2$ . The target for the correct option is one and the target for a wrong one is zero, so both results are the same sentence: probability minus target.

With $n$ options and the correct one at index $k$ , nothing changes. The loss is minus the correct score plus the logarithm of the sum of all the exponentials, and every option is differentiated the same way.

L = -z_k + \log\sum_j e^{z_j}, \qquad \frac{\partial L}{\partial z_i} = -\mathbb{1}[i = k] + \frac{e^{z_i}}{\sum_j e^{z_j}} = p_i - t_i

The matrix that gets cancelled

The reasonable expectation is that something unpleasant survives. Softmax does have an unpleasant derivative: each probability depends on every score, so the matrix of partial derivatives is full rather than diagonal, and a change to one score moves every output. Anyone who differentiates the two operations in turn and stops halfway has a right to expect a mess.

\frac{\partial p_i}{\partial z_j} = p_i\left(\delta_{ij} - p_j\right)

\bar z_j = \sum_i \frac{\partial L}{\partial p_i}\frac{\partial p_i}{\partial z_j} = \sum_i \left(-\frac{t_i}{p_i}\right)p_i\left(\delta_{ij} - p_j\right) = -t_j + p_j\sum_i t_i = p_j - t_j

Read the middle of that: the $p_i$ in front of the Jacobian cancels the $p_i$ in the denominator of the loss derivative exactly, in every term, which is why no fraction is left. What remains is $-t_j$ plus $p_j$ multiplied by the sum of the targets. Since the targets are one for a single option and zero elsewhere, that sum is one, and $p_j$ times one is just $p_j$ . The Jacobian was never absent. It was multiplied by the one thing that undoes it.

FIG 1What goes backwards at four levels of confidence

step	probability on the correct option	gradient there	largest wrong-option gradient	what happened
1	0.99	-0.01	0.01	Confidently right. Almost nothing travels back and the weights barely move.
2	0.7	-0.3	0.3	Right but unsure. A moderate push towards more confidence.
3	0.5	-0.5	0.5	Undecided between two options. A large correction.
4	0.02	-0.98	0.98	Confidently wrong, the largest signal the loss can produce.

4 steps

The correction is the error itself, nothing to tune, and it is bounded: no single example can send back more than one per option.

The numbers are the subtraction, done. A correct option sitting at probability $0.99$ sends back $0.99 - 1 = -0.01$ , and a wrong option at $0.01$ sends $0.01 - 0 = 0.01$ . A confident mistake at $0.02$ on the correct option sends $-0.98$ there and $0.98$ on the wrong one, close to the ceiling: with probabilities trapped in the unit interval no single entry of this gradient can exceed one in size.

That the algebra lands this cleanly is why libraries compute the two steps as one operation. Separating them means storing the probabilities for the backward pass, running a full matrix product where a subtraction would do, and taking the logarithm of a probability that may have underflowed to zero, which is not a number at all. The failure lands precisely on the confidently wrong examples, the ones carrying the most signal.

Where the difference stops appearing

Change either half of the pair and the cancellation goes with it. Put a squared error on the softmax output instead of a logarithm and the derivative against the scores keeps the whole Jacobian, $2\sum_i (p_i - t_i)\,p_i(\delta_{ij} - p_j)$ , which does not reduce to a difference. Keep the cross entropy but divide the scores by a temperature $T$ and the same argument leaves $(p_i - t_i)/T$ : the shape survives, the scale does not. The identity also leans on the targets summing to one. Normalised targets, smoothed ones included, are fine; a label vector summing to something else leaves $p_j\sum_i t_i - t_j$ with a term that should not be there.

Worth keeping

The gradient of softmax followed by cross entropy, with respect to the scores, is the predicted probability minus the target, one entry per option.
The softmax Jacobian is full, but the loss derivative carries a factor of one over the probability that cancels it term by term.
Because probabilities lie between zero and one, no entry of this gradient can exceed one in size, so a confident correct answer barely moves the weights.

Where this goes next

This question is one step of Backpropagation in Full, a course that works through its subject from the beginning. The page that teaches this part of it is The Layer Where Everything Cancels, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course