Basis Lab

Answers · The mathematics underneath

What does expected value actually mean if I never actually get that number?

In shortIt is a weighted average over all possible outcomes, not a prediction of the outcome you will see; it can be a number no single result ever takes.

The expected value is a weighted average of every possible outcome, not a forecast of the outcome you will get. It can be a number no single trial ever produces, because it is the balance point of the whole list rather than an entry in it.

Take a request whose time is one millisecond with weight 0.94, eight with weight 0.04, thirty with weight 0.015, and two hundred with weight 0.005. Multiply each value by its weight: 1×0.94=0.941\times0.94=0.94, 8×0.04=0.328\times0.04=0.32, 30×0.015=0.4530\times0.015=0.45, and 200×0.005=1.00200\times0.005=1.00. Add them: 0.94+0.32=1.260.94+0.32=1.26, then 1.26+0.45=1.711.26+0.45=1.71, then 1.71+1.00=2.711.71+1.00=2.71 milliseconds.

E[X]=∑ipixi=(1)(0.94)+(8)(0.04)+(30)(0.015)+(200)(0.005)=2.71.\mathbb{E}[X]=\sum_i p_i x_i=(1)(0.94)+(8)(0.04)+(30)(0.015)+(200)(0.005)=2.71.

The natural guess is that 2.71 is the time to bet on. It is not. No outcome on the list equals 2.71, and the common request takes one millisecond. The number is nearly three times the common case because the rare two hundred millisecond request contributes a full millisecond by itself: 200×0.005=1.00200\times0.005=1.00, which is larger than the 0.940.94 contributed by the ninety four percent that take one millisecond. Expectation is pulled by rare extremes, so it is a summary of the weights and values, not a description of a typical request.

It is a centre, not a forecast

A fair six sided die has expectation 3.5, since (1+2+3+4+5+6)/6=21/6=3.5(1+2+3+4+5+6)/6=21/6=3.5. The die has no face showing 3.5 and never will. Three flips of a fair coin have expected number of heads 3×0.5=1.53\times0.5=1.5; the count is always 0, 1, 2 or 3, so 1.5 is not a possible count. The word expected invites the wrong reading: the expectation is where the distribution would balance if the values were masses placed along a line, with the weights as masses.

What the expectation promises is about repetition, not about one occasion. If the same request is made many times under the same weights, the running average of the observed times tends to settle near 2.71. A single request can be one millisecond or two hundred and will not be 2.71. Betting on the expectation as if it were the next outcome is a category error, not a conservative strategy.

The rules that make it usable

Most calculations never touch the full list. Scaling and shifting pass straight through: multiply every outcome by aa and add bb, and the expectation is multiplied by aa and increased by bb. Addition passes through as well: the expectation of X+YX+Y is the sum of the separate expectations, even when XX and YY are related.

E[aX+b]=a E[X]+b,E[X+Y]=E[X]+E[Y].\mathbb{E}[aX+b]=a\,\mathbb{E}[X]+b,\qquad \mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y].

For example, put a hundred letters into a hundred addressed envelopes at random. For each letter the probability it lands in its own envelope is 1/1001/100, so its indicator has expectation 0.010.01. Adding the hundred indicators gives expected number correct 100×0.01=1100\times0.01=1, exactly, even though the letters' fates are tangled.

One operation does not pass through. The expectation of a square is not the square of the expectation. If XX is 2 or 3 with equal weight, then E[X2]=(4+9)/2=6.5\mathbb{E}[X^2]=(4+9)/2=6.5, while (E[X])2=2.52=6.25(\mathbb{E}[X])^2=2.5^2=6.25. The average of the squared values is larger by 0.25. For a function that bends upward, averaging after the function lands at or above applying the function after averaging. For a downward bending function the inequality reverses, and for a straight line the two agree.

The definition requires the weighted sum to settle to a finite number. If the tail is heavy enough that the sum diverges, there is no finite expectation, and the long run average need not settle at all. And even when it does exist, the expectation alone says nothing about spread: two distributions can share 2.71 while one is almost always near it and the other is usually one millisecond with occasional two hundred millisecond waits. That missing spread is a separate number.

Worth keeping

  • An expectation is a weighted average of all possible values, so it need not be a value that can occur.
  • The expected value says nothing about a single trial; it describes where the average of many repetitions would settle.
  • A rare extreme can dominate an expectation, so the expected value can sit far from the common case.

Where this goes next

This question is one step of Chance, Spread and What to Expect, a course that works through its subject from the beginning. The page that teaches this part of it is The Number You Would Settle For, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. What is the difference between variance and standard deviation?They measure the same spread: variance is the average squared distance from the mean, standard deviation is its square root, so you report the standard deviation and calculate with the variance.
  2. Why does a test for a rare disease give so many false positives if the test is accurate?A positive result means the test fired, not that you are likely to be ill; when the disease is rare, healthy people vastly outnumber ill ones, so a small false-alarm rate on them swamps the few true cases.
  3. Why does variance square the deviations instead of just averaging how far each value is from the mean?Variance squares deviations because signed deviations cancel to zero, and squares remove the sign while allowing the algebra, scaling and addition rules that absolute distances do not.
  4. Why does the average of many random draws get closer to the expected value as I take more?Averaging divides by the number of draws, so the spread of the average shrinks with the square root of that number; no draw ever compensates for earlier ones.
  5. How do you make a vague claim precise enough to tell whether it's true?Fix what is being talked about, what is asserted, which cases are covered and the cut-off, then name an observation that would prove it false.
  6. How do you find the parabola that best fits a function near a point?Match the function's height, slope and bending at the chosen point; those three conditions single out exactly one parabola.

Read alongside