Arrivals Come in Clumps

Last timeWhy Waiting Explodes Near the End

The same average arrival rate produces far longer queues when arrivals clump together, and the clumping is a measurable quantity rather than an adjective.

Clumps, not a smooth flow

The curve of the previous lesson assumed a particular pattern of arrivals. Real

arrivals are worse, and the reason is easy to see once stated.

Suppose a service receives sixty requests a minute. If they arrive one a second

the servers absorb them comfortably. If they arrive as six clumps of ten, each

clump arrives faster than the servers can take it, so the last of the ten waits

while the others are served. The nine seconds of idleness that follow cannot be

spent on work that has already gone through, so the spare capacity in the quiet

period is lost rather than banked.

That asymmetry is the whole of it. Idle time cannot be saved up, so any clump

that exceeds the instantaneous capacity creates waiting that the following

quiet period cannot undo.

FIG 1Sixty arrivals a minute, four ways

step	the pattern	longest queue seen	typical wait	utilisation reported
1	one every second, exactly	0	none	0.60
2	random, independently timed	3	about one service time	0.60
3	six clumps of ten	9	about five service times	0.60
4	one clump of sixty	59	about thirty service times	0.60

4 steps

The last column is identical in every row, which is the previous lesson's complaint made concrete. The four systems differ by a factor of thirty in what their users experience and by nothing at all in what their dashboards say. The second row is the usual assumption behind queueing formulas and is already worse than regular arrivals; most real traffic is somewhere between the second and third rows, and traffic shaped by retries or by timers can reach the fourth.

Measuring the clumping

The useful part is that clumping is a number. Take the gaps between successive

arrivals, compute their average and their variability, and the measure is the

variance of the gaps divided by the square of their mean.

Regular arrivals have no variability in the gaps, so the measure is nought.

Arrivals that are independent and randomly timed give a measure of one, which

is the standard assumption. Clumpy traffic gives more than one, and traffic

shaped by synchronised timers can give ten or more.

FIG 2Recovering the clumping from an observed wait

: the wait the regular arrival formula predicts
: the wait actually observed
: the clumping measure of the arrivals

The relation being rearranged is that the real wait is the predicted wait multiplied by one plus the clumping, all over two, which is the standard approximation with the service variability left out for clarity. The direction shown here is the diagnostic one: a service whose measured wait is three times what the formula predicts has arrivals with a clumping measure of about five, and that is a finding about the traffic rather than about the service. It is also a check on the model, since a measure far above anything plausible usually means the service time was mismeasured.

FIG 3How much the clumping multiplies the waiting

at sixty per cent loadat ninety per cent load

Both lines are straight in the clumping measure, so clumping multiplies rather than bends. The gap between them is the point: at sixty per cent load, going from regular arrivals to heavily clumped ones costs a modest amount of waiting, and at ninety per cent the same change costs several times as much. Clumping and high utilisation are therefore not two separate problems to be traded against each other, they compound, and a system that is both busy and bursty is in a far worse place than either figure alone suggests.

Waiting with clumps

Putting the two lessons together gives the calculation worth doing on a real

service. Take the utilisation, work out the wait the curve predicts, then

multiply by one plus the clumping measure, all over two.

A service at eighty per cent with a two hundred millisecond service time has a

predicted wait of eight hundred milliseconds under the standard assumption. If

its arrivals have a clumping measure of four, the wait is two and a half times

that, which is two seconds. That is the difference between a service that feels

responsive and one that does not, and the utilisation figure is the same in both

cases.

The lesson stops here

1 more paragraph to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents