The Average Is Not the Experience
Last timeOne Queue or Several
Nobody experiences the average wait. What people notice is the slow end of the distribution, and a request made of many parts inherits the slow end of every one.
Why the average describes nobody
Waiting times are not spread evenly around their average. They have a short
floor, which is the service time itself, a common case just above it, and a long
thin upper end where a small share of requests take many times as long.
That shape has a consequence for every report. The average is pulled upward by
the long end, so it sits above the common case, and it is nowhere near the bad
case, so it describes neither. A service with an average of fifty milliseconds
might serve most requests in twenty and the unluckiest one in fifty in six
hundred, and all three of those numbers are true at once.
| waiting, in milliseconds | share of requests at lea | |
|---|---|---|
| the common case | 18 | 1000 |
| half of requests are und | 24 | 500 |
| nine in ten are under | 95 | 100 |
| ninety nine in a hundred | 420 | 10 |
| nine hundred and ninety | 1900 | 1 |
| the average | 52 | 260 |
The reason this matters more than it sounds is that the slow end is not a rare
curiosity visited by a few unlucky people. Somebody who makes twenty requests
loading a page meets the one in a hundred case about a fifth of the time, so the
tail of the service is the ordinary experience of the user.
A request made of many parts
The compounding is what turns a mild tail into a serious one. If a request
depends on five other calls and is finished only when all five return, it is
slow whenever any one of them is slow.
Suppose each of the five is slow four times in a thousand, which sounds like a
well behaved service. The chance that the whole request escapes being slow is
the chance that all five escape, which is nought point nine nine six raised to
the fifth power, and the chance that it does not is about two in a hundred. The
assembled request is five times more likely to be slow than any of its parts.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents