What's the difference between a model that refuses because of how it was trained and one that just gets blocked by a filter sitting in front of it?

One is a habit the model learned, held in its weights and applied as part of producing a reply. The other is a separate piece of software, sitting in front of or behind the model, that reads text as data and decides. The difference that matters is what somebody would have to control to remove each: a trained refusal goes away when you find a framing the training did not cover, and a filter goes away only when you get at the pipeline. Nothing you can type into the box removes the second one, and that is the whole distinction.

Take a request for a synthesis route for a named compound. The trained model declines it, because the alignment stage showed it many requests of that shape and it learned to attach a refusal to the shape rather than to the exact words. Now wrap the same request as a scene in a novel, where a character explains the process to another character. The same model answers. Nothing has broken. A tendency has no threshold at which it must fire, and the fiction moved the request into a region the training barely covered.

Put the same two requests through a check on the output instead. The check reads the reply. The reply contains the route, whether it arrived bare or wrapped in dialogue, because the framing lives in the request and the route lives in the answer. Both requests are stopped. The wrapper bought nothing, and that is not because the check is stricter. It is because the check was never reading the wrapper in the first place.

In the first expression, the rules $r$ and the request $x$ are both text in the same window, and the model produces $y$ by continuing them. There is nothing in that arrangement marking $r$ as more authoritative than $x$. So everything usually called jailbreaking is the same move: text arrives in the stream the model is continuing and competes with the rules on equal terms. A sentence beginning ignore your previous instructions works because the model is doing what it always does, which is continuing text, and the countermand is text.

A classifier is not continuing anything. It reduces the message to features and returns a number, and the number decides. To that function, ignore your previous instructions is a string with certain properties, and those properties may well push the score up rather than down, since adversarial phrasing correlates with adversarial intent. There is no path from the content of the message into the mechanism that judges it, because the mechanism is not following the message, it is measuring it. The natural guess, that a clever enough phrasing talks its way past both, holds for the first and has no way to begin on the second.

The costs are not symmetrical either, and the arithmetic is dull but decisive. At twenty requests a second, a day is $20 \times 86{,}400 = 1{,}728{,}000$ requests, and thirty days is $1{,}728{,}000 \times 30 = 51{,}840{,}000$. A check billed at two hundredths of a cent a call costs $51{,}840{,}000 \times 0.0002 = \$10{,}368$ a month. If that check takes 150 milliseconds, then $20 \times 0.15 = 3$ requests are inside it at any instant, so three of them must run concurrently just to keep pace with arrivals, before any queue forms. The trained refusal adds no call and no delay, so its serving cost is zero and its latency is zero.

Error rates have the same shape. If the check wrongly blocks one honest request in a hundred, that is $1{,}728{,}000 \div 100 = 17{,}280$ people a day told no for no reason. A trained tendency does not produce that particular number, but it produces the opposite failure: some requests that should have been refused get through, and there is no dial to trade one failure against the other.

Both stop being reliable in specific places, and the places differ. The check fails on coverage rather than on authority. One that matches strings can be walked past by base64, or by a translation into a language whose patterns were never collected, so it fails on unanticipated input in the same way a tendency does. It never fails by being talked out of its job.

The trained refusal fails in the other direction as well: it is fixed at training time. A rule that the product decides on next week is not in the weights and will not be until somebody retrains, whereas a check can be updated in an afternoon. And there is one scenario where the ordering inverts entirely. If the generating model is under somebody else's control, because they run their own deployment with their own weights and their own prompt, then both the training and the prompt are inside the thing that was captured. A check sitting after the model still reads the output and can still stop it. The last limit is streaming: text already shown to a user cannot be unsent, so a check placed at the end of a streamed reply protects only the tail of it, which is a property of where the check sits rather than of how good it is.