A Chain of Reliable Parts Is Not Reliable

Last timeWhere the Measurement Is Taken

Availabilities multiply along a chain, so a service standing on ten dependencies cannot be more reliable than their product, however good its own code is.

A service is usually written as though it were one thing. It is not. It is the

last link in a chain, and the chain has an arithmetic that does not care how

carefully the last link was written.

FIG 1A chain where every part is required

: the availability of one part, when all the parts are alike
: how many parts the request passes through

This is multiplication because the request needs all of them, and multiplication of numbers below one always goes down. Every dependency added to a request is a subtraction from the reliability of everything above it, and nothing about the quality of the new dependency changes that direction.

FIG 2What a chain of three-nines parts can promise

each part at three nineseach part at four nines

At thirty dependencies the upper curve is still near 99.7 percent and the lower one has fallen to about 97. The lesson is not that dependencies are bad, it is that the count of them is a reliability decision being made by whoever adds the next one, usually without noticing.

What a second copy buys

Redundancy works on the failure rather than on the availability, and that is why

it is powerful.

FIG 3Two independent copies, either of which is enough

: the availability of the pair, which is everything except the case where both are down
: the probability that both copies are unavailable at the same moment, each being down with probability one minus its availability

A failure probability of one in a thousand becomes one in a million. No amount of careful coding buys three orders of magnitude; a second copy does it in one step. This is the single highest-leverage intervention available, and everything in the next section is about why it underdelivers.

The word that does all the work

The squaring above is a consequence of one assumption, stated quietly:

independence. The two copies must fail for unrelated reasons.

The lesson stops here

4 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

A Chain of Reliable Parts Is Not Reliable

What a second copy buys

The word that does all the work

4 more paragraphs to go

This is the reading half

Read alongside