The Average Is the Answer

Last timeMaking a Random Draw

The average of many draws settles on the quantity wanted. This lesson says why it does, what has to be true for it to, and what the settling looks like while it happens.

The previous two lessons set up a loop: draw, compute, average. This one

answers the question that loop raises, which is why the average should be the

number that was wanted.

FIG 1What is promised, and at what rate

: the estimate formed from n draws, which is itself random
: the true quantity, fixed and unknown
: the spread of a single draw, which the problem fixes and the method cannot change
: the number of draws taken
: the typical distance of a random quantity from its own average

Three separate claims, worth keeping apart. The first says the estimate is centred on the truth at every sample size, so a small run is noisy rather than biased. The second is the convergence guarantee, which licenses the method but gives no rate and so cannot say when to stop. The third supplies the rate and is the most practically important of the three: the typical error falls with the square root of the number of draws, which is the arithmetic the next lesson is entirely about.

FIG 2How the typical error shrinks as draws accumulate

a quantity with spread onea quantity with spread three

Both curves fall steeply at first and then almost flatten, which is the single most important shape in this course. Going from ten draws to a hundred is a large improvement; going from a thousand to two thousand is barely visible. The spread of the quantity being averaged scales the whole curve without changing its shape, which is why the upper curve is three times the lower one everywhere, and why reducing that spread is worth as much as taking more draws. The next two lessons develop exactly these two levers.

FIG 3One run of an estimate, as draws accumulate

step	draws so far	estimate	typical error at this point	comment
1	10	0.2900	0.158	swinging, and the first digit is not yet reliable
2	100	0.3510	0.050	in the right region
3	1 000	0.3367	0.016	two digits look stable
4	10 000	0.3340	0.005	third digit settling
5	100 000	0.33318	0.0016	four digits, at ten times the cost of the line above
6	the true value	0.33333	0	an exact third

6 steps

The estimate is the area under a squared curve, whose true value is one third. Each row costs ten times the row above and buys roughly one further reliable digit divided by three, since the error falls by a factor of about three per tenfold increase. Notice that the estimate is not always closer at a larger sample size, because every row is one draw of a random quantity; the typical error shrinks reliably while any individual run wanders inside it.

When it fails

Two conditions underlie all of this. The draws must be independent, and the

quantity must have a genuine average.

The second is easier to show than to describe. Some quantities have tails so

heavy that occasional enormous values dominate everything drawn before them,

and the running average jumps each time one arrives rather than settling.

Insurance losses, file sizes, city populations and wealth all behave this way

to some degree.

FIG 4A run that settles, and one that never does

python

import numpy as np
rng = np.random.default_rng(3)

x = rng.random(100_000)
for n in (10, 100, 1_000, 10_000, 100_000):
    print(n, round(float(np.mean(x[:n] ** 2)), 5))   # settles on 0.33333

heavy = 1.0 / rng.random(100_000)                    # a tail with no finite average
for n in (10, 100, 1_000, 10_000, 100_000):
    print(n, round(float(np.mean(heavy[:n])), 2))    # jumps, never settles

print(float(np.max(heavy)))                          # one draw larger than most sums

The first block prints the running estimate at five sample sizes and shows it tightening onto one third. The second draws from a quantity with a tail heavy enough that no average exists, and its running estimate jumps by large amounts at the biggest sample sizes rather than settling, with the jumps arriving whenever a single enormous draw lands. The lesson is in the comparison rather than in either number: the second run looks like an ordinary simulation from the inside, produces a confident figure, and that figure means nothing.

Independence fails more quietly. Draws that share a source of randomness, a

simulation that carries state from one run into the next, or a generator with

hidden structure all produce draws that lean the same way, and the average of

those converges to something, just not to the quantity wanted. The final

lesson of the course is about catching that.

The lesson stops here

1 more paragraph to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The Average Is the Answer

When it fails

1 more paragraph to go

This is the reading half

Read alongside