What You Can Measure Without Labels

During training there is a number. Accuracy on data held back, computed

against true answers, as precise as anybody wants.

On the day the model ships that number stops being computable. The cases

arriving now have no true answers attached, and most of them never will. The

loan either defaults in two years or does not. The flagged transaction is

disputed or is not, weeks later, and only if somebody noticed. The

recommendation is clicked or ignored, which is information about something but

not about whether the recommendation was right.

So the first question is not how to monitor quality. It is what is actually

visible, and then what that visible set can and cannot establish.

Five things are free, in the sense that they require no labels and no extra

work beyond recording what already passes through.

The second row is the one teams build last and need most. Recording the

distribution of every input, column by column, costs a little storage and

catches an enormous share of real failures, because most production failures

are an input that changed rather than a model that decayed.

The single most informative free quantity is the distribution of the model

own output. Not the average, which hides almost everything, but the shape.

Three habits make the score distribution useful rather than decorative. Record

it in buckets rather than as an average, because a model that has started

answering half its cases with near certainty and the other half with near

zero has the same average as one answering everything with a shrug. Compare

against the distribution the model produced on its training data, which is the

only anchor available. And watch the extremes specifically, since a jump in

the share of cases scored above 0.95 is usually a broken input rather than a

sudden abundance of obvious cases.

Between the free signals and real labels sit the proxies: things people do

that correlate with the model being wrong.

None of these is a substitute for truth, and the alternative is cheaper than

teams expect. Take a sample of cases, have somebody establish the real answer,

and compute accuracy on the sample. A few hundred cases a week is usually

within reach, and it provides the one thing the whole free panel cannot.

A deliberate blind spot remains. Reviewing only the flagged cases says nothing

about the cases the model let through, and those contain the failures nobody

hears about. A small random sample of unflagged cases, however unexciting, is

the only thing that measures them.

Worth stating plainly, because the dashboards built from this material look

authoritative and are not.

Nothing observable without labels can establish that the answers are right. A

model can be badly wrong with a perfectly stable score distribution, perfectly

normal inputs, unchanged response times and no complaints, and the panel will

be entirely green. This happens whenever the relationship between the inputs

and the answer has changed while the inputs themselves have not, which is the

subject of the third lesson and the most dangerous failure in the course.

What the panel does establish is that something has changed, which is worth

having for two reasons. Most failures are operational rather than statistical,

and those it catches immediately. And when the delayed labels eventually

arrive and show that quality fell, the panel is the record that says when it

started and what else moved at the same time, which is the difference between

diagnosing the problem in a morning and arguing about it for a fortnight.

The next lesson takes the first of the free signals seriously: deciding

whether the inputs have genuinely moved, or whether this is what an ordinary

week looks like.