How do I write a safety rule I can actually test, instead of just saying 'be harmless'?

A rule you can test names an action rather than a quality, and it comes with the cases it covers, the cases it deliberately leaves out, and what the system does when it fires. Whether you have written one is settled by two people and twenty requests: hand them the sentence and twenty borderline cases from your own traffic, ask each independently to label every case allowed or refused, and count how often the labels match.

The twenty requests matter more than the sentence does. Draw them from real traffic and put them near the line, because twenty obvious refusals will score perfect agreement for any sentence at all, including "be helpful but safe". On a borderline set that sentence scored 23 per cent, and $20 \times 0.23 = 4.6$, so four or five of the twenty came back the same and fifteen came back differently. That is not a strict rule. It is a sentence two people read as a mood.

The rewrite is mechanical once you can see what is missing. A policy said "do not explain how weapons are made". Two readers agreed on 62 per cent of twenty cases, and $20 \times 0.62 = 12.4$, so about twelve agreed and eight split. The eight were not exotic: a homework question about reaction conditions, a paragraph of a novel, a treaty's history. Naming the behaviour and the boundary, "do not return a synthesis route, quantity or supplier for a controlled substance", moved agreement to 91 per cent, and $20 \times 0.91 = 18.2$, so eighteen of the twenty.

Two cases still split at 91 per cent. Adding the exclusions and the action closed them: "refuse, with a stated reason, any request for a synthesis route, quantity or supplier for a controlled substance; pharmacology, history and policy discussion are in scope" reached 97 per cent, which is nineteen of the twenty. The remaining request was a toxicology researcher asking about a dose, which is an exception to log rather than an ambiguity to argue about.

The natural guess is that a sentence everyone approves of is a rule everyone will apply the same way, and those are two different measurements. A reviewer nods at "be harmless" because they agree with the sentiment, and the nod feels like a rule being ratified. Then each reader privately supplies the list of cases the author never wrote down, and the lists do not match. The sentence feels precise to whoever wrote it for exactly that reason: they can see their list. Nobody else can, and twelve disagreements out of twenty is what that invisibility looks like once you count it.

Write the rule long and boring. Every clause that makes a sentence applicable makes it less quotable, so a document edited for readability has been edited towards untestability, and a document that reads well is worth re-checking. The ceiling is a sentence like "never send email to an address outside the account holder domain", which scores twenty out of twenty because it names a comparison between two strings and no values at all. Nothing about it is warm, and that is the point.

The agreement test measures checkability and nothing else. A sentence can be perfectly agreed and still wrong: the email rule is testable and says nothing about harm, so twenty out of twenty on it tells you only that two people can apply it. There is also a floor that no rewrite crosses. An audit of a thirty-one sentence document gave seven testable as written, nine fixable in one pass, eleven that were intentions rather than rules, and four that contradicted another sentence in the same document. Those add to 31, so 11 of 31, about 35 per cent, could not be made into rules at all. They still do a job: they record what the organisation was trying to do for the engineer who meets an unanticipated case.

Keep that job and file it separately, labelled, so that nobody counts it as protection in a review. What an untestable sentence actually does in a live system is move the decision to whoever is on call when the hard case arrives, under time pressure, without the context the authors had. The rules section holds only sentences that passed, each with its test set, its action, an owner, a review date, and a log of every exception granted. A rule whose exceptions outnumber its firings was not a rule anybody believed, and those cases are the material for the next rewrite.