Running the Whole Process

Last timeMaking Each Sample Count

Some questions are about a system that evolves rather than a single draw. The answer is read off the run itself, which raises three problems that a one shot simulation never has.

The first lesson opened with a queue and then put it aside. Here it is again,

because it is the ordinary shape of a question that has no formula: things

arrive at random, take a random time to deal with, and the question is what

that feels like from the inside.

Nothing is averaged over single draws here. The simulation runs the system and

reads the answer off what happened.

FIG 1Six events of a single server queue

step	clock	event	in queue	server	recorded
1	0.00	start, empty and idle	0	idle
2	0.41	arrival	0	busy until 1.63
3	0.88	arrival	1	busy until 1.63
4	1.63	service completes	0	busy until 2.10	waited 0.75
5	2.10	service completes	0	idle	waited 0.00
6	2.55	arrival	0	busy until 3.31
7	3.31	service completes	0	idle	waited 0.00

7 steps

The whole simulation is this loop: draw the next arrival, draw the next completion, advance the clock to whichever comes first, update the state, and record anything the question needs. No formula appears anywhere, which is why the same loop survives changes that destroy any formula, such as customers abandoning the queue after waiting too long or a second class of work jumping ahead. Note the third recorded wait of zero: a customer who arrives to an idle server waits not at all, and those zeros are a large part of why the average wait is so much shorter than the longest wait.

FIG 2One run, and which part of it may be measured

The run begins empty because it has to begin somewhere, and empty is not what the system usually looks like, so the first customers enjoy conditions that will never recur. Averaging them in pulls the answer down. The discarded stretch is chosen by plotting the quantity against time and finding where it stops drifting, rather than by a rule of thumb, because the right length depends heavily on load: a lightly loaded system forgets its start quickly while a heavily loaded one can take many times longer, and the heavily loaded case is usually the one somebody is asking about.

The measurements lean on each other

Two consecutive customers face almost the same queue, so if one waits a long

time the next probably does too. Those recorded waits are not independent

draws, and the error formula from the earlier lesson does not apply to them.

Applied anyway, it gives an interval several times too narrow. That is worse

than reporting no interval, because a confident wrong number travels further

than an honest uncertain one.

FIG 3A queue simulation, and two error bars for the same answer

python

import numpy as np

def run(seed, n=20_000, warm=2_000, rate=0.9):
    rng = np.random.default_rng(seed)
    gaps = rng.exponential(1.0, n)            # time between arrivals
    service = rng.exponential(rate, n)        # time to serve each customer
    free, waits = 0.0, np.empty(n)
    clock = 0.0
    for i in range(n):
        clock += gaps[i]
        waits[i] = max(0.0, free - clock)     # wait only if the server is busy
        free = max(clock, free) + service[i]
    return waits[warm:]

w = run(0)
print(float(np.mean(w)), 2 * float(np.std(w, ddof=1) / np.sqrt(len(w))))

per_run = [float(np.mean(run(s))) for s in range(10)]
print(float(np.mean(per_run)), 2 * float(np.std(per_run, ddof=1) / np.sqrt(10)))

The function runs one queue and returns the average wait after the warm up. The first printed interval treats every recorded wait as an independent draw, and the second compares ten independent runs with different seeds. Both describe the same answer and the first is several times narrower, which is the whole point: the narrow one is not more precise, it is wrong, because it counts correlated measurements as if each were fresh information. The ten run interval is the one to report.

How to split the work

A single long run pays the warm up cost once and then measures efficiently,

and it cannot produce an honest error bar. Many short runs produce the

independent numbers an interval needs, and spend most of their time warming

up.

The usual answer is about ten reasonably long runs. The warm up overhead stays

near a tenth of the work, and ten independent averages is enough to tell a

settled answer from a shaky one, which is the decision actually in front of

anybody running this.

The lesson stops here

1 more paragraph to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

Running the Whole Process

The measurements lean on each other

How to split the work

1 more paragraph to go

This is the reading half

Read alongside