A Step Good 95 Times in 100 Is Not Good Enough to Be the Fourth of Four
Last timeWhy Split a Job at All
Where the product rule for a chain of model calls comes from, what four steps at 0.95 actually deliver, and the two things that can be done about it.
The order service has four steps. It sorts the request, finds the matching
order and policy, drafts the reply, and checks the draft against the policy.
Suppose each of those four is good 95 times in 100, which is a rate most people
would be pleased with and would describe as working.
The service is good 81.5 times in 100. Nearly one request in five has something
wrong with it, where a single call at the same 95 per cent would have had one in
twenty. What follows is where that number comes from, how to read it off a
table for any chain, and what the two available repairs are.
What a step being good has to mean
Before any arithmetic, the quantity has to be defined, and the usual definition
is the wrong one.
The rate that compounds is not how often a step is right. It is how often a step
hands on something the next step can work from. Those come apart in both
directions. The drafting step can produce a stiff, badly phrased reply, and the
checking step will pass it, and the customer gets a slightly awkward email that
nobody counts as a failure. That is a step being wrong without the chain being
damaged, because the fault was recoverable by what came after it.
Now suppose the second step picks the wrong order, one placed three weeks
earlier by the same customer. Every later step performs perfectly on the input it
was given. The draft is warm and accurate about the wrong delivery, and the check
confirms the draft matches the policy for the wrong delivery, because nothing
downstream holds the record that would contradict it. One bad handover, four
steps running correctly, and a wrong answer.
So let the per step rate be the chance that a step produces a handover its
successor cannot detect or repair. Measure that and the product rule below
applies to it. Measure how often each step is right and the product will be
pessimistic about the awkward emails and blind to the wrong orders.
Where the product comes from
Write for the chance the whole chain comes out good and for the chance
step is good. Nothing about the chain says these are independent, so start
from the thing that is always true.
The lesson stops here
10 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents