Basis Lab

Answers · The mathematics underneath

What is the difference between variance and standard deviation?

In shortThey measure the same spread: variance is the average squared distance from the mean, standard deviation is its square root, so you report the standard deviation and calculate with the variance.

They measure the same spread in two different units. The variance is the average squared distance from the expectation, so a quantity measured in milliseconds has a variance in squared milliseconds, a unit nobody can picture; the standard deviation is the square root of the variance, which brings the number back into milliseconds. Report the standard deviation, because it sits on the scale of the thing being described and can be compared to the mean directly. Calculate with the variance, because it is the one that behaves properly when quantities are combined.

Var⁡[X]=E[(X−μ)2],σ=Var⁡[X],μ=E[X]\operatorname{Var}[X] = \mathbb{E}\big[(X - \mu)^2\big], \qquad \sigma = \sqrt{\operatorname{Var}[X]}, \qquad \mu = \mathbb{E}[X]

A request takes 10 ms half the time, 20 ms three times in ten, and 60 ms the remaining one time in five. The expectation is 0.5(10)+0.3(20)+0.2(60)=5+6+12=230.5(10) + 0.3(20) + 0.2(60) = 5 + 6 + 12 = 23 ms. Measure every outcome against that 23, square each distance, and weight the squares by the same probabilities.

FIG 1
time (ms)probabilitydeviation from 23 mssquared deviationweight times squared deviation
100.5-1316984.5
200.3-392.7
600.2371369273.8
total10361

The last column sums to 84.5+2.7+273.8=36184.5 + 2.7 + 273.8 = 361, so the variance is 361 square milliseconds. Its square root is 19 exactly, so the standard deviation is 19 ms, a number on the same footing as the 23 ms mean: typical distances from the average are a little smaller than the average itself. The same 361 comes from the one pass form, 890−529=361890 - 529 = 361, where 890 is the weighted average of 100100, 400400 and 36003600 and 529 is 23223^2. For contrast, the mean distance without squaring is 0.5(13)+0.3(3)+0.2(37)=6.5+0.9+7.4=14.80.5(13) + 0.3(3) + 0.2(37) = 6.5 + 0.9 + 7.4 = 14.8 ms. That is a legitimate measure too, and smaller, because squaring makes the 37 ms deviation carry nine times the weight of the 13 ms one instead of under three times.

Squaring rather than taking absolute values is a genuine choice, and it buys the algebra: squares expand, differentiate and pass through expectations, which is what produces the shortcut form above and what makes variances add below. It also makes far outcomes count disproportionately, which is right when distance from the target is costly and wrong when it is merely unusual.

Why the square root is not decoration

The natural guess is that these are one number wearing two costumes, differing only by a change of units, so it makes no difference which you quote. That fails as soon as anything is combined, because the square root does not distribute over a sum. Two independent requests, each with variance 361 and standard deviation 19, have a combined variance of 361+361=722361 + 361 = 722 and a combined standard deviation of 722≈26.9\sqrt{722} \approx 26.9, not 19+19=3819 + 19 = 38. Anyone adding or averaging standard deviations of independent parts is quoting a figure that belongs to no distribution at all. The square root is not undoing work that should not have been done; the variance is what the arithmetic needs, and the root is what a reader needs.

Scaling and shifting show the same asymmetry. Double every time and the variance goes from 361 to 1444 while the standard deviation goes from 19 to 38, a factor of four against a factor of two. Add 100 ms to every time and neither moves, both staying at 361 and 19, because spread is about differences and is blind to location; only the mean shifts, from 23 to 123. So a quantity in different units has a variance that rescales by the square of the conversion factor and a standard deviation that rescales by the factor itself, which is another way of saying the standard deviation is the one in the original units.

Averaging is where the standard deviation earns its place in a report. The mean of nn unrelated copies of a quantity has variance divided by nn and standard deviation divided by n\sqrt{n}. Four independent requests, each with variance 361 and standard deviation 19, give an average with variance 361/4=90.25361/4 = 90.25 and standard deviation 9.5: four times the data halves the spread. In variances that reads as dividing by four; in standard deviations it reads as dividing by two, and the second sentence is the one anybody acting on the number wants to hear.

Where the pair stops working

Both numbers presume the squared deviations have a finite average. Distributions with heavy tails do not have one, and a variance computed from a sample of such a distribution measures the sample size rather than the quantity. The standard deviation is also dominated by the largest deviations, since a point twice as far out contributes four times as much, which distorts the picture when extreme values are merely rare rather than genuinely bad. And the familiar band, roughly two thirds of the weight within one standard deviation and nineteen twentieths within two, is a property of the bell shape and not of spread in general. For the 10, 20 and 60 ms example the interval 23±2(19)23 \pm 2(19) runs from −15-15 to 61 and covers everything while telling you almost nothing; without knowing more about the shape, the honest statement is only that the weight beyond kk standard deviations is at most 1/k21/k^2.

Worth keeping

  • A variance in squared milliseconds cannot be compared to a mean in milliseconds, but its square root can.
  • Variances of unrelated quantities add while their standard deviations do not: two independent spreads of 19 combine to about 26.9, not 38.
  • Averaging four independent measurements halves the standard deviation and quarters the variance.

Where this goes next

This question is one step of Chance, Spread and What to Expect, a course that works through its subject from the beginning. The page that teaches this part of it is How Far From Expected, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. Why does a test for a rare disease give so many false positives if the test is accurate?A positive result means the test fired, not that you are likely to be ill; when the disease is rare, healthy people vastly outnumber ill ones, so a small false-alarm rate on them swamps the few true cases.
  2. Why does variance square the deviations instead of just averaging how far each value is from the mean?Variance squares deviations because signed deviations cancel to zero, and squares remove the sign while allowing the algebra, scaling and addition rules that absolute distances do not.
  3. What does expected value actually mean if I never actually get that number?It is a weighted average over all possible outcomes, not a prediction of the outcome you will see; it can be a number no single result ever takes.
  4. Why does the average of many random draws get closer to the expected value as I take more?Averaging divides by the number of draws, so the spread of the average shrinks with the square root of that number; no draw ever compensates for earlier ones.
  5. How do you make a vague claim precise enough to tell whether it's true?Fix what is being talked about, what is asserted, which cases are covered and the cut-off, then name an observation that would prove it false.
  6. How do you find the parabola that best fits a function near a point?Match the function's height, slope and bending at the chosen point; those three conditions single out exactly one parabola.

Read alongside