The Bill Arrives as Latency

Last timeThe Part Only One May Enter

Waiting at a lock does not grow smoothly with load. It is flat for a long time and then rises without limit, and the first thing to break is the slowest few percent of requests.

A lock is correct. The question in this lesson is what it costs, and the answer

is more interesting than a constant. The cost of waiting is not proportional to

load. It is flat, and flat, and flat, and then it is not.

One number predicts the waiting

Take the rate at which workers arrive at the door and multiply it by how long one

trip through the door takes. The product is the share of time the region is

occupied, and almost everything else follows from it.

FIG 1How long an arrival waits, from how busy the door is

: the average time an arrival spends waiting before it gets in
: how long one trip through the guarded region takes
: the share of time the region is occupied, which is the arrival rate times s
: the multiplier applied to the hold time, which is small when the door is quiet and unbounded as it saturates

The multiplier is the whole story. At thirty percent occupancy it is about 0.43, so waiting is less than half a hold. At ninety percent it is 9, so waiting is nine holds. At ninety nine percent it is 99.

FIG 2Waiting against how busy the door is

a forty microsecond holdan eight hundred nanosecond hold

Both curves have the same shape, which is the point. Shortening the hold moves the whole curve down but does not change where the wall is, and the wall is always at full occupancy regardless of how fast the guarded region is.

This explains a pattern that otherwise looks like bad luck. A system runs at

sixty percent occupancy for a year and is fine. Traffic grows twenty percent,

occupancy goes to seventy two percent, and waiting roughly doubles. Traffic grows

another twenty percent, occupancy reaches eighty six percent, and waiting is now

six times what it was. Nothing was deployed. No component failed. The curve

simply got steep.

The lesson stops here

6 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The Bill Arrives as Latency

One number predicts the waiting

6 more paragraphs to go

This is the reading half

Read alongside