Nobody Writes In to Report a Refusal

Last timeWhat the Model Is Allowed to Touch

Wrongly refused requests are expensive and almost never counted, because the people affected leave quietly. Here is how to measure them and where the stricter setting stops paying.

The error nobody reports

Of the two ways a check can be wrong, one of them generates a conversation and

the other does not.

A harmful answer that got through produces an incident, a review, a thread, and

usually a change. A legitimate request wrongly refused produces a polite

decline on a screen, a user who concludes the product is either broken or

timid, and silence. They do not write in. They try once more, sometimes, with

different words, and then they go elsewhere.

That asymmetry in reporting becomes an asymmetry in the system, because teams

respond to what they hear about. Over months, every incident pushes the

threshold stricter and nothing pushes it back, and the system ends up tuned to

a point nobody chose.

FIG 1Four methods for measuring over-refusal

	hours to set up	repeatable each release	measures your real traff	usefulness out of 10
sample your own refusals	4	0	1	9
a set of safe requests t	20	1	1	7
watch what users do afte	2	1	0	6
ask refused users direct	60	1	1	10

The marked row is the one to do first: half a day, no new infrastructure, and it measures the traffic you actually have rather than a set somebody else assembled. The second row is the one to build next, because it is the only one that can run on every release.

Four ways to count it

The first method in detail, because it is the one almost nobody does. Take

three hundred refusals from last week, at random. Read them. For each, decide

whether a reasonable person would say the system should have answered. Count.

Two things happen every time. The rate comes back two to five times higher

than the team expected. And the refusals cluster into three or four recognisable

groups, which is more useful than the number, because a cluster is something you

can fix.

FIG 2Three hundred refusals, read by hand

step	cluster	how many	should have answered	what it was	what happened
1	clinical vocabulary	71	64	nurses and carers asking about medicatio	The largest cluster and almost entirely wrong. The words overlap with a harmful category and nothing else about the requests does.
2	historical and fictional framing	54	48	students, writers and teachers describin	The second cluster. Refusing to discuss a historical atrocity is not a safety outcome, it is a product that cannot be used for history.
3	security and administration	42	31	people asking how an attack works so the	Mixed, and genuinely harder. A third of these were correctly refused, which is why this cluster needs a rule rather than a loosened threshold.
4	actually harmful	36	0	requests the system should and did decli	Twelve per cent of the refusals. The other eighty-eight per cent landed on people who had done nothing wrong.

4 steps

Three clusters and a remainder. Notice the shape of the fix: the first two are a threshold and a category rule, the third needs a written policy, and only the fourth is the thing the filter was built for. Reading them is what makes those distinctions visible.

What one refusal costs

A wrongly refused request is not one lost answer. Price it properly.

Some share of the affected users stop using the product. Multiply that share by

what a user is worth over their lifetime. Add the support contacts, since some

fraction do write in and each contact has a cost. Add the ones who tell

somebody else, which is hard to measure and real.

For a consumer product with modest per-user value the figure usually lands

somewhere between a few cents and a couple of dollars per wrongly refused

request. For a business product where a single user represents a seat in an

account, it can be hundreds. The number matters less than having one, because

the next step needs it.

The lesson stops here

3 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents