Reading a Real Number Out of Its Bits, and Writing One Tenth Back In

Last timeWhat Happens at the Edge

The sign, exponent and fraction fields of a 32 bit and a 64 bit real, decoded by hand from one bit pattern, then a tenth normalised, expanded in binary, rounded, and quoted exactly.

Why the point is not simply nailed down

Thirty-two bits is $2^{32} = 4{,}294{,}967{,}296$ patterns, and nothing about

the bits decides what they mean. A rule for reading them does. The simplest

rule for fractions is to nail the binary point to one place and leave it there:

read the top 16 bits as a whole number and the bottom 16 as a fraction. That

layout is honest and easy, and its limits are arithmetic. The largest magnitude

it names is $2^{15} = 32{,}768$ , and the distance between neighbouring values

is $2^{-16} = 0.0000152587890625$ , the same distance everywhere on the scale.

Now consider a calculation holding Avogadro's number, about $6 \times 10^{23}$ ,

and the charge on an electron, about $1.6 \times 10^{-19}$ . The fixed layout

cannot name either one. Widening it does not help much, because a fixed point

layout spends bits on absolute resolution it does not need: to reach $10^{23}$

in steps of $10^{-19}$ takes over 140 bits, almost all of them recording

digits nobody measured.

So spend a few of the bits on where the point is instead. Take eight of the

thirty-two and read them as the power of two to scale by. Eight bits give

$2^8 = 256$ codes. Two of them are reserved, the all-zeros code and the

all-ones code, which is what the zeros, the infinities and the not-a-number

values are built out of. That leaves $256 - 2 = 254$ usable codes, and with a

bias of 127 they run from $1 - 127 = -126$ to $254 - 127 = 127$ .

FIG 1What eight exponent bits reach, in powers of two

The smallest normal magnitude is two to the minus 126, which is 0.0000000000000000000000000000000000000117549435, and the largest is two minus two to the minus 23, all times two to the 127, which is 340,282,346,638,528,859,811,704,183,484,516,925,440. The fixed point layout of the same width covers the 31 powers of two from minus 16 to 15 and nothing outside them. Eight bits of exponent buy 253 powers of two of extra reach and cost nothing but the eight bits.

The eight bits are not free. They come out of the fraction, which leaves 23

bits, and 23 bits of fraction is 23 bits of fraction at every magnitude. That

is the whole bargain, and it is worth saying plainly before any of the

decoding: the exponent field buys range, the fraction field buys precision, and

moving a bit from one to the other trades one for the other exactly.

Count what is left and finiteness stops being a warning and becomes a number.

Two signs, times 254 exponent codes, times $2^{23} = 8{,}388{,}608$ fractions,

is $2 \times 254 \times 8{,}388{,}608 = 4{,}261{,}412{,}864$ ordinary values.

Subtract that from the $4{,}294{,}967{,}296$ patterns and

$33{,}554{,}432$ remain, which is $2 \times 2 \times 2^{23}$ , the two reserved

exponent codes with both signs: the zeros, the very small values, the

infinities and the not-a-numbers, all of them together.

FIG 2The two formats, field by field

	sign bits	exponent bits	fraction bits	bias	decimal digits carried
32 bit	1.00	8.00	23.00	127.00	7.22
64 bit	1.00	11.00	52.00	1023.00	15.95

The bias in the marked column is two to the power of one less than the exponent width, minus one: two to the 7 minus 1 is 127, and two to the 10 minus 1 is 1,023. The last column is the base ten logarithm of two to the number of significand bits, which is 24 stored-plus-implied bits for the 32 bit form and 53 for the 64 bit form. Twenty-four times 0.30103 is 7.22 digits, and 53 times 0.30103 is 15.95, which is where the familiar seven digits and sixteen digits come from.

The three fields

Three fields, read from the top of the word down: one sign bit, then the

exponent field, then the fraction field. Two of the three are not what they

look like.

The exponent field does not hold a signed exponent. It holds the exponent plus

a bias, so the stored field is never negative and no sign convention is needed

inside it. The bias is $2^{k-1} - 1$ for a field of $k$ bits, so $2^7 - 1 = 127$

for the 32 bit form and $2^{10} - 1 = 1023$ for the 64 bit form. The reason for

that particular offset is worth knowing, because it looks arbitrary and is not.

With the bias, the exponent codes increase as the magnitude increases, and the

exponent field sits immediately below the sign bit, so reading the entire 32 bit

word as an unsigned integer puts the positive reals in ascending order. A

machine can therefore compare two positive reals with integer hardware, and

sorting them is sorting their bit patterns.

The fraction field does not hold the significand either. Any non-zero value can

be written as some number in $[1, 2)$ times a power of two, by shifting the

point until exactly one digit sits in front of it. In base two there is only one

non-zero digit available, so that leading digit is always a one. Storing it

would spend a bit on a constant. It is left out, and the 23 stored bits give 24

bits of significand. This is also why the all-zeros exponent code has to be

reserved: with no implied one there would be no way to write zero at all.

The lesson stops here

4 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

Reading a Real Number Out of Its Bits, and Writing One Tenth Back In

Why the point is not simply nailed down

The three fields

4 more paragraphs to go

This is the reading half

Read alongside