When a Simulation Lies

Last timeUsing the Data You Have Twice

A wrong simulation looks exactly like a right one and reports a healthy error bar. Three failures account for most of them, and each has a check that takes minutes.

Everything so far has been about getting an answer and knowing how precise it

is. This lesson is about the gap between precise and correct, which the error

bar cannot see.

A simulation that is wrong produces a number, a tight interval and a plot that

looks exactly like a simulation that is right. The only defence is knowing in

advance what makes one lie.

FIG 1How each failure shows itself, and what catches it

	error bar notices?	answer moves between see	caught by a known case?	caught by halving the sa
a poor source of randomn	0	1	1	0
an event too rare to sam	0	1	0	0
draws that are not indep	0	1	0	1
drawing from the wrong d	0	0	1	0

The first column is zero all the way down, which is the single most important fact in this lesson: the reported interval detects none of these, because it measures noise and every one of them is a failure of correctness. The first marked cell shows why the seed check is good but not sufficient, since a simulation drawing from the wrong distribution is perfectly stable across seeds and consistently wrong. The second marked cell is the one that justifies the whole lesson, since a defective randomness source still reports a tight interval around a wrong answer.

Rare events and the error bar of zero

An event that never happened in the sample gives an estimate of zero with an

interval of zero width. Nothing about that output signals a problem, and it is

simply the statement that the run was too short to see anything.

The rule to adopt is to count the successes, not the draws. Fewer than a few

dozen successes means no digit of the answer should be believed, and the two

honest responses are to use the importance sampling from the earlier lesson,

or to report an upper bound derived from having seen nothing at all.

FIG 2Four simulations, each wrong, each confident

step	what was built	what it reported	why it is wrong	the check that catches it
1	eight workers seeded from the clock	an estimate with a tight interval	several workers shared a stream, so the draws repeat	compare the spread across seeds with the interval
2	a million draws of a one in ten million event	a probability of zero, interval zero	the run was too short to see one	count the successes before quoting
3	waits recorded within one long queue run	an interval four times too narrow	consecutive waits lean on each other	compute the interval from ten whole runs
4	an arrival pattern with the wrong rate	a stable, repeatable, precise answer	the model does not match the system	run a simplified case with a known answer
5	the common thread	a healthy looking output	an error the interval cannot see	a check from outside the simulation

5 steps

Every row produced output that would pass an unsuspicious review, which is the reason to run the checks by habit rather than when something looks wrong. The last row is the pattern worth taking away: each check compares the simulation against something outside itself, whether a different seed, a count, a different unit of independence, or a case whose answer is already known. No check in the list looks only at the numbers the run reported, because those numbers are exactly what the failure corrupted.

FIG 3Four checks, each a few lines

python

import numpy as np

def est(seed, n=50_000):
    r = np.random.default_rng(seed)
    v = r.random(n) ** 2
    return float(np.mean(v)), float(np.std(v, ddof=1) / np.sqrt(n))

m, se = est(0)
print(abs(m - 1/3) < 2 * se)                       # recovers a known answer

runs = [est(s)[0] for s in range(10)]
print(float(np.std(runs, ddof=1)), est(0)[1])      # these two should agree

print(est(0, 50_000)[1], est(0, 12_500)[1])        # should roughly double

r = np.random.default_rng(4)
hits = int((r.random(1_000_000) < 1e-6).sum())
print(hits, 'successes: quote nothing below a few dozen')

The first check recovers a known answer, here an area whose exact value is a third, and confirms the interval covers it; most plumbing defects die here. The second compares the spread across ten seeds with the interval each run reported, and a spread much larger than the interval means the draws are not as independent as assumed. The third halves the sample and confirms the error grows by about forty per cent, since an error bar that does not move means draws are being reused. The fourth counts successes before any rare probability is quoted.

The habit

Run a known case first. Most defects are in the plumbing rather than the

mathematics, and a simplified version of the model with a closed form will

find them in a minute.

Then quote the answer the way the course has argued for throughout: an

estimate, an interval, the number of draws behind it, and for a probability

the number of successes. Those four numbers together are an answer somebody

else can check. Any one of them alone is a number somebody else has to trust.

The lesson stops here

1 more paragraph to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

When a Simulation Lies

Rare events and the error bar of zero

The habit

1 more paragraph to go

This is the reading half

Read alongside