When a Model Is Sure and Wrong
Uncertainty More Data Would Remove, and Uncertainty It Would Not
Last timeOne Number That Fixes Most of It
Some uncertainty is in the question and cannot be trained away. Some is in the model and would vanish with the right examples. They look identical in the output and demand opposite responses.
Uncertainty in the question
Ask a model to predict whether a particular coin flip comes up heads. The best
possible answer is 0.5. A model that says 0.99 is not more confident, it is
wrong, and no quantity of additional coin flips in the training data will
improve the prediction, because the prediction is already as good as the
question permits.
This is uncertainty that lives in the question rather than in the model. It
shows up whenever the same input genuinely leads to different outcomes: two
customers with identical records, one of whom churns; a sentence with two
defensible translations; a medical presentation consistent with two conditions
at similar rates. In each case a perfect model reports a spread, and the spread
is the right answer.
The practical mark of this kind is that it does not shrink. Collect ten times
the data for the region in question and the reported spread is the same, because
the outcomes themselves are spread.
Uncertainty in the model
The second kind is ignorance. The model has not seen anything much like this
input, so it is extrapolating, and the answer it gives is weakly supported.
The crucial difference is that somewhere there exists data that would remove it.
A model that is unsure about contracts in a jurisdiction absent from its training
set is not facing an unanswerable question; it is facing a question it was never
taught. Three hundred examples would fix it.
The lesson stops here
3 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents