Basis Lab

Bytes, Text and File Formats

Numbers With Fractions In Them

Last timeWhole Numbers, and Where They Stop

A tenth cannot be written exactly in twos, so the usual fractional number is an approximation from the first value, which is why totals come out a hundredth of a penny wrong.

Why a tenth cannot be stored

In the usual base ten, a third cannot be written exactly. It is nought point

three recurring, and any attempt to write it down stops somewhere and is

therefore slightly too small. Nobody considers this a defect in decimal

notation.

The same thing happens in base two, with different fractions. A half, a quarter

and an eighth are exact, because each is one divided by a power of two. A tenth

is not. Written in twos it recurs forever, exactly as a third does in tens, so

whatever is stored for the value nought point one is a number very close to a

tenth and not equal to it.

The rule is the same in every base. A fraction is exact when its denominator

divides some power of the base. Ten has factors two and five, so fifths and

tenths are exact in base ten and thirds are not. Two has only itself, so only

halves, quarters, eighths and so on are exact, and everything else, including

a tenth and a fifth, is an approximation.

FIG 1Which fractions are exact in which base
exact in base tenexact in base two
a half11
a quarter11
a fifth10
a tenth10
a third00
Coded 1 for exact and 0 for recurring. The fourth row is the one that causes trouble in practice, because a tenth is the unit people write money and percentages in, and it is the most common inexact value in any system. The last row is the comforting one: a third is inexact in the notation everybody uses daily and nobody is confused by it, which is the right way to think about what follows.

The consequence is immediate. Add nought point one to itself three times and the

answer is not nought point three. It is slightly more, in about the seventeenth

digit, and an equality test against nought point three fails. Nothing has gone

wrong; three slightly wrong values were added and the result is slightly wrong.

How large the error is

The stored form keeps a fixed number of significant digits, about fifteen or

sixteen in the common eight byte size, and then an exponent that says where the

point goes. So the error is relative: the gap between neighbouring representable

values near one is about one part in ten thousand million million, and near a

million the gap is a million times larger.

The lesson stops here

5 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the planThe contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The rest of this course

  1. 01Everything Is a Run of Bytes
  2. 02Counting in Twos and Sixteensopening only
  3. 03Letters as Numbersopening only
  4. 04Whole Numbers, and Where They Stopopening only
  5. 05Numbers With Fractions In Themyou are here
  6. 06Dates, Times, and the Day It Movedopening only
  7. 07Rows, Columns and the Separator Problemopening only
  8. 08Making It Smalleropening only

Read alongside