Basis Lab

What a Computer Does to a Number

The Gap to the Next Representable Number, and How to Compute It

Last timeWhere the Point Floats

Representable values are evenly spaced between one power of two and the next, so the gap is two to the exponent minus 52, which is derived here and then used on a tenth and on a counter that stops.

Where the gap comes from

A 64 bit real is a sign, a 53 bit significand whose leading bit is implied, and

an exponent. Hold the exponent still at some value ee and ask what values are

available. The significand runs over every whole multiple of 2−522^{-52} from

1.000…01.000\ldots 0 up to 1.111…11.111\ldots 1, which is 2−2−522 - 2^{-52}. Multiplying by

2e2^{e}, the available values are

$2e,2e(1+2−52),2e(1+2⋅2−52),…2^{e}, \quad 2^{e}\left(1 + 2^{-52}\right), \quad 2^{e}\left(1 + 2 \cdot 2^{-52}\right), \quad \ldots$

and the difference between any two neighbours in that list is $2^{e} \cdot

2^{-52} = 2^{e-52}$. It is the same difference every time, because the only

thing changing is a whole number of units in the significand. So inside the

interval from 2e2^{e} to 2e+12^{e+1}, which is conventionally called a binade,

the representable values sit on an even grid, and the grid spacing is

2e−522^{e-52}.

FIG 1The distance to the next representable value
the gap: the distance from x to the representable value immediately above it
the value in question, anything other than zero
the exponent of the binade x lies in, which is the largest power of two not above the size of x
the number of fraction bits, which is 52 for the 64 bit form and 23 for the 32 bit form
Two things are worth noticing in this formula. The only appearance of x on the right is inside a floor of a logarithm, so the gap is flat in x and then jumps. And p sits in the exponent, so one extra fraction bit halves the gap everywhere at once: the 32 bit form, with 29 fewer fraction bits than the 64 bit form, has gaps two to the 29 times wider, which is 536,870,912 times.

The step everyone skips when doing this by hand is the floor. For x=0.1x = 0.1 the

logarithm is log⁡20.1=−3.3219\log_2 0.1 = -3.3219, and the floor of that is −4-4, not −3-3.

Rounding a negative logarithm towards zero gives the wrong binade and a gap

twice as large as the real one. The check that costs nothing is to confirm that

2e≤∣x∣<2e+12^{e} \le |x| < 2^{e+1} afterwards: 2−4=0.06252^{-4} = 0.0625 and 2−3=0.1252^{-3} = 0.125,

and 0.0625≤0.1<0.1250.0625 \le 0.1 < 0.125, so e=−4e = -4 is right.

FIG 2The power of two in the gap, against the magnitude
-53.00-51.25-49.50-47.75-46.001.08.816.524.332.0magnitude of the value
exponent of the gap
A staircase with a tread at every power of two. The gap is two to the minus 52 for every value from 1 up to just under 2, then two to the minus 51 from 2 to just under 4, then two to the minus 50 from 4 to just under 8. Each tread is twice as wide as the one before it in magnitude and each riser doubles the gap, which is what makes the relative gap almost constant: the values spread out at exactly the rate the magnitudes grow.

It doubles, and it doubles all at once

The practical consequence of the shape is the discontinuity rather than the

trend. Just below 2242^{24} in the 32 bit form the exponent is 23 and the gap is

223−23=12^{23-23} = 1. Just above 2242^{24} the exponent is 24 and the gap is

224−23=22^{24-23} = 2. Nothing gradual happens between those two statements. One

representable value has a neighbour one away and the very next one has a

neighbour two away.

The lesson stops here

4 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the planThe contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The rest of this course

  1. 01Why Eight Bits Hold 256 Values and Minus 128 Has No Positive Twin
  2. 02What Really Happens When a Sum Does Not Fitopening only
  3. 03Reading a Real Number Out of Its Bits, and Writing One Tenth Back Inopening only
  4. 04The Gap to the Next Representable Number, and How to Compute Ityou are here
  5. 05One Rounding Is Half a Gap, and a Tenth Plus Two Tenths Is Exactly a Tieopening only
  6. 06Error in a Long Sum Grows Like Root n, Not Like nopening only
  7. 07The Subtraction That Loses Every Correct Digit at Onceopening only
  8. 08Two Zeros, Two Infinities, and a Value Not Equal to Itselfopening only

Read alongside