Answers · The systems around the model

How can I tell if my traffic is bursty or just high volume?

In shortCompare the variance of the gaps between arrivals with the square of their average gap: high volume alone does not make traffic bursty.

High volume is an average; burstiness is a variance. The test is to take the gaps between successive arrivals and compare their variance with the square of their mean: a high average with steady gaps is high volume, a lower average with wildly uneven gaps is bursty.

c = \frac{\operatorname{Var}(G)}{(\mathbb{E}[G])^2}

Write $G$ for the gap between one arrival and the next, in seconds. Regular arrivals, one every second, have $G=1$ every time: the mean is $1$ and the variance is $0$ , so $c=0/1^2=0$ . Independent random arrivals with the same mean gap have exponential gaps, whose variance equals the square of their mean: with mean $1$ the variance is $1$ , so $c=1/1^2=1$ . That is the standard random case, not an especially bursty one. The number to compare against is $1$ , not $0$ .

Now keep the same mean gap of one second and make the arrivals clump. Suppose nine gaps in ten are zero and one gap in ten is ten seconds. The mean is $0.9\times0+0.1\times10=1$ second. The second moment is $0.9\times0^2+0.1\times10^2=10$ , so the variance is $10-1^2=9$ . The clumping measure is $9/1^2=9$ . The average rate is unchanged at sixty a minute, but the spacing is nine times as variable as independent random arrivals. A zero gap is an idealisation of arrivals that are very close together, and the arithmetic is what matters: the same mean can carry a very different variance.

FIG 1

pattern	mean gap (s)	variance (s)
Regular, one every second	1	0	0	0.5
Independent random	1	1	1	1
Nine zero gaps in ten, one ten-second gap	1	9	9	5

Checking the wait

If the raw gaps are not available, the same quantity can be recovered from waiting times. Let $k$ be the wait the standard random-arrival formula predicts at the observed utilisation and service time, and let $W$ be the wait actually observed. The relation $W=k(1+c)/2$ rearranges to the diagnostic form. The two routes measure the same gap variance. Use the first when you have timestamps of individual arrivals; use the second when you have only latency and load. If both are available they check each other: a gap-based $c$ of nine should appear as a wait about five times the random-arrival prediction, and a wait five times the prediction should not be explained by a gap-based $c$ near one.

c = \frac{2W}{k} - 1

At eighty per cent utilisation with a two hundred millisecond service time, the standard formula gives $k=0.8/(1-0.8)\times0.2=0.8$ seconds. If the measured wait is two seconds, then $c=2\times2/0.8-1=4$ . Check: $0.8\times(1+4)/2=2$ seconds. A service whose wait is three times the prediction has $c=2\times3-1=5$ . That is a statement about the spacing of arrivals, not about the servers being slow.

Volume and clumping compound

High volume raises the utilisation and therefore raises both $k$ and the wait produced by any fixed $c$ . Burstiness raises $c$ and multiplies whatever wait the utilisation already produces. The two are not alternatives. At sixty per cent load a clumping measure of nine gives a multiplier of five; at ninety per cent the same multiplier applies to a much larger $k$ , so the absolute wait is far larger. This is why a dashboard showing requests per second can make two systems look identical: the average rate is the same and the clumping is invisible.

The natural guess is to compare peak rate with average rate and call the difference burstiness. That can be misleading, because a single peak over a long window is diluted by the window, and a stream of small clumps can leave the peak low while making every clump wait. The variance of gaps catches the clumping at the scale that matters to the queue. A high average with $c$ near one is high volume, not bursty. A modest average with $c$ of nine is bursty, not high volume.

Where it stops being true

The measure assumes a stationary window. If the morning is smooth and noon is clumped, one $c$ averages two regimes; compute it over a window in which the mean gap is roughly stable, and preferably over several windows to see the spread. It also assumes the service time is known. A $c$ far above ten is possible for synchronised timers and retries, but it is also what a badly measured service time produces, so check that first. With few arrivals the variance estimate is noisy; a dozen gaps cannot distinguish $c=1$ from $c=3$ reliably.

Worth keeping

High volume is an average arrival rate; burstiness is the variance of the gaps between arrivals relative to the square of their mean.
The clumping measure is zero for regular arrivals, about one for independent random arrivals, and above one for bursty traffic.
If the measured wait is W and the standard formula predicts k, the clumping measure is 2W/k minus 1.

Where this goes next

This question is one step of Work That Waits in a Queue, a course that works through its subject from the beginning. The page that teaches this part of it is Arrivals Come in Clumps, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course