Answers · Putting a model into service
Can I judge my model's accuracy from the labels that come back first?
In shortNo. Early labels are the failures, not a random sample, so an early accuracy figure is biased and usually makes the model look worse than it is.
No. The labels that come back first are not a small random sample of the outcomes. They are the cases whose outcomes resolve fastest, and in most products that means the failures: a disputed transaction, a missed first payment, a claim that is filed at once. Judging accuracy on them is therefore not a noisy read of the model. It is a read of a different population, and the difference has a direction, usually making the model look worse than it is. The early figure can be useful, but only as a comparison with another cohort read at the same age, never as an estimate of the final accuracy of this cohort.
Outcomes do not arrive in random order. A fraudulent card transaction is disputed within days while a legitimate one is never disputed at all; a loan that goes wrong immediately misses its first payment within a month while a loan that repays takes the full term to say so. The early arrivals are the bad tail of the cohort. If you read only what has arrived, you are conditioning on speed of resolution, and speed is correlated with the outcome you are trying to measure.
Take a cohort where 30 per cent of labels have arrived, so . The bad rate among those early reporters is , but history says the whole cohort usually runs at . The remaining 70 per cent must then have bad rate , about 5.1 per cent. Nothing alarming has happened: the early read is high because the bad cases reported first, and the late reporters are expected to be much better. If instead you had treated 18 per cent as the cohort's bad rate, you would have overestimated it by a factor of two.
The error is not noise
The same cohort read at five moments shows the pattern. Nothing about the model changes between the readings; only the share of labels that has arrived changes. At day ten the early bad rate is 41 per cent; by day one hundred and twenty it is 8.9 per cent. The final figure is slightly better than the historical 9 per cent, but the day ten figure is more than four times the truth. A team acting on the day ten number would roll back a model that is working.
| step | days since the decisions | share of labels arrived | bad rate among arrived | what a naive read concludes | what happened |
|---|---|---|---|---|---|
| 1 | 10 | 8 per cent | 41 per cent | catastrophe, escalate immediately | day ten |
| 2 | 30 | 26 per cent | 19 per cent | much worse than the last model | day thirty |
| 3 | 60 | 58 per cent | 12 per cent | somewhat worse | day sixty |
| 4 | 90 | 87 per cent | 9.4 per cent | about the same | day ninety |
| 5 | 120 | 97 per cent | 8.9 per cent | the truth: slightly better | day one hundred and twenty |
The shape is predictable because it is driven by the arrival curve, not by the model. At day thirty the partial rate is 19 per cent against a final 8.9 per cent, so a naive read says 'much worse'. At day sixty it is 12 per cent; at day ninety 9.4 per cent. The sequence converges from above. The convergence is not evidence that the model improved during the cohort's life. It is evidence that the bad cases reported early and the good cases reported late.
Reading a cohort honestly
The clean fix is to group decisions by when they were made and read a cohort only when its labels are substantially complete, typically at the age where ninety per cent have reported. Every cohort is then read at the same point in its life, so cohort-to-cohort comparison is honest. The cost is delay. If labels take days and you need labelled cases at decisions per day, the measurable quality figure is days old. With , and , days. The newest reliable figure describes decisions made more than two months ago.
Waiting for full maturity is often unacceptable, so use an incomplete cohort only by comparing like with like. The bad rate of this cohort at thirty days of age is biased, but the bad rate of last quarter's cohort at thirty days of age carries the same bias, so the bias cancels in the difference. A change in that comparison is a real change, and it can appear two months before either cohort matures. This is an alarm, not a measurement: it tells you that something moved between cohorts, not what the final bad rate will be.
The device stops working if the arrival process itself changes. If a dispute process becomes easier, or a payment scheme changes when failures are reported, then the share arriving by thirty days changes and the same-age cancellation fails silently. Monitor the share reported at each age alongside the bad rate; if that share moves, the comparison must be rebuilt. A genuinely random missingness process would make the early labels an unbiased sample, but that is the exception, not the rule, and it is not something to assume without checking the arrival curve.
Worth keeping
- An early read makes the model look worse than it is, because the cases that resolve first are the failures.
- A cohort should be read at a fixed age and compared with another cohort at the same age, so the arrival-order bias cancels.
- If the share of labels arriving by that age changes, the same-age comparison no longer cancels and the early read is not usable.
Where this goes next
This question is one step of Watching a Model After Launch, a course that works through its subject from the beginning. The page that teaches this part of it is Labels That Arrive Weeks Later, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- Why does the model return JSON wrapped in extra text when I asked for JSON?A request for JSON does not force the model to return only JSON, so you need to validate the reply and repair it once rather than assume it will parse.
- Why do the numbers look good while users keep complaining?Because the score measures a fixed set of cases with a fixed rule, and the users are living through behaviour that the set and the rule never covered.
- How do I make the model answer from the documents I give it instead of its own memory?Fence the documents as labelled reference material, put them before the question, tell the model to prefer them over memory, and require a passage number for each claim.
- Why does the memory used by a model keep growing as the conversation gets longer?The model's weights do not grow. It keeps a cache of keys and values for every earlier token, so memory rises with conversation length and with how many conversations run at once.
- How many legitimate requests should I be willing to block in order to catch the bad ones?Block no more legitimate requests than one prevented harmful answer is worth: the break-even count is the harm cost divided by the refusal cost.
- What does a p-value actually tell me?A p-value is the chance of seeing a difference at least as large as yours if the change did nothing; it is not the chance the result was luck or that there is no effect.