StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>This is a fundamental limitation of representing decimal numbers in binary form. Binary floating point numbers are expressed in powers of 2 while decimal numbers are expressed in powers of 10, and C's <code>float</code> is simply unable to exactly represent all the decimal numbers.</p> <p>Your example number, 96.1 can be written as:</p> <pre><code>96.1 = 9*10^1 + 9*10^0 + 1*10^-1 </code></pre> <p>To represent this in binary, you can get the integer 96 just fine:</p> <pre><code>96 = 1*2^6 + 1*2^5 </code></pre> <p>but representing the 0.1 is problematic in base 2. The place values of the first few fractional places in binary are:</p> <pre><code>2^-1 = 0.5 2^-2 = 0.25 2^-3 = 0.125 2^-4 = 0.0625 2^-5 = 0.03125 2^-6 = 0.015625 2^-7 = 0.0078125 2^-8 = 0.00390625 2^-9 = 0.001953125 ... and so on </code></pre> <p>So somehow you need to use a combination of these place values to add up to approximate 0.1 in decimal. So you would have to start with b0.0001 (d0.0625) as being the first place less than d0.1 and add some more of the smaller place values to get closer and closer to 0.1. For example:</p> <pre><code>b0.001 = d0.125 // too high, try lower b0.0001 = d0.0625 // too low, so add smaller places b0.00011 = d0.09375 // good, closer... rounding error is 0.0625 b0.000111 = d0.109375 // oops, a little high b0.00011001 = d0.09765625 // getting better - how close do you need? ... </code></pre> <p>And so on - you get the idea. So the binary values can only approximate decimals due to the fundamental representation. </p> <p>There are many articles on floating point rounding errors and representational limits. It is definitely worth doing some background reading on this topic.</p> <p>There are a few ways to solve this problem:</p> <ul> <li>Use <code>float</code> but remain aware of the limitations and carefully design the algorithm to minimise rounding errors</li> <li>Use an exact decimal representation such as BCD (binary coded decimal), which is used in financial systems to avoid rounding errors</li> <li>Use a fixed data type, where numbers are expressed as fractions of integers, and only convert to floating point at the end of the calculation to display the result.</li> </ul>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload