Floating-point error
A Socratic walk-through of floating-point error — reasoned out one step at a time, not lectured.
The question we started with
THE QUESTION #Why can a computer calculate 0.1 + 0.2 without obtaining exactly 0.3?
Ask almost any programming language for 0.1 + 0.2 and it answers 0.30000000000000004. The natural reaction is that the machine got the addition wrong. But a computer that could not add would be useless for everything, and this one runs banks. So perhaps the addition is not where the trouble lies. Where else could it be?
Reasoning it through
REASONING #Consider what must happen before any addition occurs. You wrote "0.1" — a mark on a page, which the machine must first turn into a pattern of bits. So there are three places an error might enter, not one.
Here is a clue you already accept. Write one third in decimal: 0.333… and it never ends. Nobody blames arithmetic for that; base ten cannot finish writing one third, because 3 divides no power of ten. The rule that survives is general: a fraction terminates in a base only when its denominator's prime factors are factors of that base.
Now apply it to a machine working in base two, whose only prime factor is 2. Which fractions terminate? Halves, quarters, eighths — nothing else. One tenth? Ten factors into 2 and 5, and 5 is unavailable, so 0.1 in binary is 0.0001100110011… with "0011" repeating forever, exactly as one third repeats in decimal. Tenths are as awkward in base two as thirds are in base ten; our fondness for them is an accident of having ten fingers.
A double carries 53 bits of significand, so an endless expansion is cut there and rounded. The value stored for "0.1" is not one tenth but 0.10000000000000000555111512312578…, a hair too large, and "0.2" likewise. Then the addition happens — and it is exact. But the exact sum needs 54 bits, one more than fits, so it is rounded a second time, landing on 0.30000000000000004440892098500626. The closest double to the mark "0.3" is 0.29999999999999998889776975374843. Neighbours, one step apart, and not equal.
The analogy
THE ANALOGY #Imagine a ruler marked only in halves, quarters, eighths and so on down. Ask for one tenth and you must take the nearest tick — close, but not the mark you wanted. Ask for two tenths and take the nearest tick again. Lay the two lengths end to end and the total falls between ticks, so you read the nearest once more, one tick from the tick that best represents three tenths.
A ruler's ticks are evenly spaced; floating-point ticks crowd near zero and spread as numbers grow, which is why adding a small number to a large one can change nothing at all.
Clarifying the model
THE MODEL #Two temptations to resist. The first is to conclude that computers cannot do exact decimal arithmetic; they can, with decimal or fixed-point types that store money in whole cents. The second is to fix comparisons by rounding what is printed — the display was never the problem. The real repair is to stop asking whether two computed floats are equal and ask instead whether they agree within a tolerance suited to the scale of the values.
A picture of it
THE PICTURE #How to readRead the four boxes top to bottom as the life of a value, but read the arrow labels for the argument — each arrow is where information is lost. The first box is what you typed and can never be stored; the second is what the machine holds instead. The third is the sum, and note the addition itself is exact: the loss happens in the rounding after it. The last arrow says the two results differ by one representable step, which is why the equality test fails.
What became clearer
WHAT CLEARED #The addition is not wrong. One tenth has no finite binary expansion, so it is rounded on the way in, the exact sum is rounded again on the way out, and the result lands one representable step from where "0.3" lands. The error is in representation, not arithmetic.
Where to go next
ONWARD #- Why summing a long list of floats in a different order gives a different total.
- Why 0.5, 0.25 and 0.75 behave perfectly while 0.1, 0.2 and 0.3 do not.
- How decimal and fixed-point types trade speed and range for exact tenths.
Key terms
TERMS #| Term | What it means |
|---|---|
| Significand | the digits of a floating-point number, separate from its exponent; a double carries 53 bits of them. |
| Unit in the last place (ulp) | the gap between one representable number and the next, which widens as magnitude grows. |
| IEEE 754 | the standard defining binary floating-point formats and the rounding behaviour nearly all hardware implements. |
Every term the collection defines is gathered in the glossary.