Two Mistakes, One Dial, and You Have to Choose
Last timeWhy a Prompt Is Not a Boundary
A check on the incoming request has exactly two ways to be wrong, and one threshold trading them against each other. Everything else about it is detail.
The check and its four forms
An input check is a function from the incoming request to a decision: allow,
block, or sometimes escalate. It runs before the model is called, which is
where its two advantages come from. The blocked request costs nothing, since no
generation happens. And it cannot be argued with by the text it is reading,
because it is classifying that text rather than continuing it.
There are four kinds in common use and they are not alternatives so much as
rungs.
| cost per thousand, hundr | delay, milliseconds | catch rate, per cent | wrongly blocked, per cen | |
|---|---|---|---|---|
| a list of patterns | 0 | 1 | 35 | 4 |
| a trained classifier | 2 | 12 | 68 | 3 |
| a small model asked to j | 30 | 90 | 84 | 2 |
| a large model asked to j | 220 | 400 | 91 | 1 |
The practical arrangement is two rungs in series. A pattern list runs on
everything and is free. Anything it is unsure about goes to a classifier.
Anything the classifier scores in a middle band, typically five to ten per cent
of traffic, goes to a model. Average cost stays near the classifier and worst
case accuracy approaches the model.
The two ways to be wrong
Every check has exactly two failure modes and they pull against each other.
It can block a request that was fine. The user sees a refusal they did not
deserve, concludes the product is broken or prudish, and some fraction of them
stop using it. You usually do not find out, because nobody writes in to report
a refusal.
It can allow a request that was not fine. You find out about this one, late,
and from somebody else.
One number moves both: the threshold at which a score becomes a block.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents