When a Model Is Sure and Wrong
The Threshold Comes From Your Costs, Not From a Round Number
Last timeAsking the Same Thing Twice
Answering beats refusing exactly when the expected cost of a wrong answer falls below the cost of a referral. That comparison gives a threshold, and the threshold is where calibration pays off.
The two costs
A system that can refuse has three outcomes rather than two: right, wrong, or
handed to a person. Choosing between them requires knowing what each one costs,
in the same units.
The cost of a wrong answer is whatever the organisation actually loses when the
system is confidently incorrect: a refund, a complaint handled, a contract
mispriced, a correction issued, occasionally something much larger. The cost of
a referral is the time of whoever picks it up, plus the delay. Both of these are
estimable to within a factor of two by anybody who has worked on the process,
and the factor of two is good enough, because the threshold moves slowly in
them.
Most arguments about where to set a threshold are arguments about these two
numbers conducted without writing either down. Writing them down usually ends
the argument, and occasionally ends the project, because sometimes the honest
answer is that a wrong answer costs so much that nothing the model offers is
worth having.
Where the threshold comes from
Suppose the model reports probability p for its answer, a wrong answer costs W,
and a referral costs R. Answering has expected cost (1 - p) times W, since a
right answer costs nothing. Referring costs R whatever happens.
Setting the two equal and solving gives the rule.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents