The Finite Abilities of Mathematical Programming

By Minjoon Choi

Northwood High School

When you type in something like 0.1 + 0.2 into any programming language, the end result is precisely 0.30000000000000004, not 0.3. So, 0.1 + 0.2 is not 0.3.

With a finite amount of bits being able to store digits, only a finite amount of numbers can be represented.

As a compromise, computers calculate numbers through scientific notation, due to the ease of representing very small and very large numbers at the same time with minimal problems with storage shortages.

Due to how computers count in binary(instead of human’s base 10), their scientific notation is a value multiplied by 2 to a certain power, and not 10.

This scientific notation is great for representing numbers, but leads to inaccuracies for numbers that cannot be represented by base 2, or binary. This is very efficient at dealing with both astronomically large numbers, and numbers that are the size of atoms without needing too much storage.

Human Example: ⅓ + ⅓ + ⅓ should be 1. But if you were like a computer that cannot understand recursion, the answer would be something like 0.9999999999999999999999999999999999999999999999999999, but then you will stop, as you’ve ran out of digits to store(52 digits max for 64 bit computers). So if you divide 1 by 3 and multiply that by 3, the answer will no longer be 1.

Computer Example: Computers only see 2s, 4s, 8s, 16s, and halves, quarters, eighths, and sixteenths. To them, there is no easy way of representing 0.1, something that is a tenth of 1. In fact the decimal 0.1 in binary is 0.00011001100110011… with the “0011” part recurring. Since computers cannot understand recursion, they stop at a certain point(52 decimal points for 64 bit and 23 for 32bit), which turns out to be “0.1000000000000000055511151231257827021181583404541015625” for 64 bit computers, slightly more than what you started with.

This means that any number you tell to the computer that cannot be represented with a power of 2 is physically unable to be 100% precisely calculated.

So, why use this inefficient system?

In most cases, it doesn’t matter if your value is millionths or billionths off of what it is supposed to be, as that would almost certainly be close enough, and any kind of manufacturing tolerances will be unable to keep up anyways. But since what floating point essential does is store a fixed length of numbers and then trying to figure out where to place the decimal point, any kind of error will be very obvious to us humans.

Facts

Computers count in binary, and can only interpret Yes(1) and No(0), but nothing between.

0 -> 0

1 -> 1

2 -> 10

3 -> 11

4 -> 100

5 -> 101

And so on!