Get a usable number out of a problem too awkward to solve exactly by simulating it, and say honestly how far off that number is likely to be and how many more samples would be needed to halve the error

Answering With Random Samples

When the formula runs out, draw samples and average them. This course derives why that works, how wrong it is, and the handful of tricks that make each sample count for more.

8 lessons, written and corrected before you arrived. Reading them here needs no account. The first reads the whole way through; the others open and then stop, because a page nobody owns cannot tell who is reading it. Starting the course gives you your own copy, where every idea has problems standing under it and you can ask about any sentence.

Start reading

01When the Formula Runs OutMost quantities worth knowing have no formula. The move that rescues them is to write the quantity as an average and then produce that average by running the thing many times.
02Making a Random Drawopening onlyEverything starts from numbers drawn evenly between zero and one. Two standard moves turn those into draws from whatever distribution the problem actually has.
03The Average Is the Answeropening onlyThe average of many draws settles on the quantity wanted. This lesson says why it does, what has to be true for it to, and what the settling looks like while it happens.
04How Far Off Is Itopening onlyAn estimate without an error bar is not an answer. The error bar comes from the samples themselves, and it carries an uncomfortable consequence about the cost of precision.
05Making Each Sample Countopening onlyBrute force buys precision at a terrible exchange rate. Three standard tricks reduce the spread of what is being averaged instead, which is worth far more than extra draws.
06Running the Whole Processopening onlySome questions are about a system that evolves rather than a single draw. The answer is read off the run itself, which raises three problems that a one shot simulation never has.
07Using the Data You Have Twiceopening onlyThere is no formula for the error bar on most statistics worth computing. Resampling your own measurements supplies one, using nothing but the data already collected.
08When a Simulation Liesopening onlyA wrong simulation looks exactly like a right one and reports a healthy error bar. Three failures account for most of them, and each has a check that takes minutes.