When the Right Answer Moves
Last timeWhen the Inputs Move
The inputs moving is visible and usually harmless; the relationship between input and answer moving is invisible without labels and is what silently ruins a model.
Two different things can move
Last lesson watched the inputs. This lesson is about the distinction that
makes that watching useful, and it is the central idea of the course.
Two things can change between training and today. Which cases arrive can
change: more young customers, larger amounts, a new region. Or, for exactly
the same case, the right answer can change: the customer who would have repaid
two years ago now defaults, with identical inputs, because interest rates
moved.
The first leaves the model relationship intact. It is being asked about a
different mixture, and if it saw cases like these in training it will be fine.
The second makes the relationship wrong, and no amount of the same inputs
helps.
| visible in the inputs | visible in the scores | the learned relationship | typical cause | |
|---|---|---|---|---|
| the mixture of cases cha | 1 | 1 | 1 | 1 |
| the answer for a given c | 0 | 0 | 0 | 2 |
| both at once | 1 | 1 | 0 | 3 |
Why the second one is worse
Picture a fraud model. The inputs are amounts, times, device types, merchant
categories, and their distributions are exactly what they were last month. The
score distribution is unchanged. Volumes are normal, response times are
normal, nothing has errored.
Meanwhile a new fraud technique has appeared that the model was never trained
on, and transactions that it scores at 0.1 are the ones worth worrying about.
Every signal on the panel is green. The model is wrong on the cases that
matter and will remain wrong until the disputes arrive, which is six to twelve
weeks.
There is a partial exception worth knowing. If the relationship moves in a way
that also changes the balance of answers, for instance if genuine fraud
becomes much more common, then the score distribution shifts and the panel
does say something. That is luck rather than coverage, and the cases where the
model is quietly wrong on a constant share of traffic produce nothing at all.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents