Why can a model get 100% on the training set and still be useless?

The gap $G$ is how much of the training performance did not transfer. Here $E_{\text{train}} = 0$ and $E_{\text{new}} = 0.90$, so $G = 0.90$. A training error of zero is therefore compatible with new error of 0.90. Neither number means much alone; the pair does. The lower the training error and the wider the gap, the more of the apparent success was fitting. A perfect training score alone cannot distinguish this from a model that has found a pattern that does transfer.

The training examples were used to pick the model. Every adjustment during training was made to improve performance on exactly those examples, including adjustments that fitted their particular accidents. Real examples carry accidents: a measurement that is a little off, an unusual case, or two features that happen to line up in this sample and not in general. A flexible model fits all of it because fitting it improves the number being optimised, and nothing in the procedure distinguishes an accident from a pattern. That is why the training number is contaminated rather than merely optimistic. It is computed on the very data whose accidents were fitted, so it cannot separate pattern from accident. Asking the training number whether the model learned is like asking a student to mark their own paper against answers they wrote.

Reading the pair is the practical skill. A low training error with a wide gap means the model fitted the specific examples; the first response is to reduce freedom or add data. A high training error with a narrow gap means the model was too rigid to fit anything, and the gap is narrow only because there was nothing to lose; the first response is more freedom or better inputs. A low training error with a narrow gap is the target. A high training error with a wide gap usually means the measurement is broken, not the model. The two situations call for opposite responses, and the pair is the fastest way to tell them apart.

Capacity is the freedom to fit. Regularisation is a deliberate reduction of that freedom. More data makes an accident harder to fit reliably, because an accident does not repeat; less freedom removes the ability to fit it at all. Those two responses act on different causes, which is why a wide gap is not automatically a request for more examples. The lookup table shows the limit: adding examples leaves it perfect on the enlarged training set and still guessing on new ones, because its failure is in what the method does with data, not in how much data there is. If the model is merely flexible, more data or less freedom can narrow the gap. If the method memorises exactly, neither changes the new-example error.

A held-out measurement is only informative if the held-out examples come from the same source as the training examples. Grant that and a sample can tell you about the population; deny it and a model fitted to the old process has no claim on the new one. This is why an honest held-out number can still fail in deployment: prices change, users change, the thing being measured changes, and sometimes the model itself changes the behaviour it is predicting. When the source moves, the old measurement stops being a promise, and more old examples do not repair it. No quantity of training examples repairs a source that has moved.