StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POThe Google Calculator Glitch, could float vs. double be a possible reason?
text
Body
copied!<p>I did this Just for kicks (so, not exactly a question, i can see the downmodding happening already) but, in lieu of Google's newfound <a href="http://www.google.com/search?hl=en&q=1999999999999999-1999999999999995&btnG=Search" rel="nofollow noreferrer">inability</a> to do <a href="http://www.google.com/search?hl=en&q=400000000000002-400000000000001&btnG=Search" rel="nofollow noreferrer">math</a> <a href="http://www.google.com/search?hl=en&q=10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000&btnG=Search" rel="nofollow noreferrer">correctly</a> (check it! according to google 500,000,000,000,002 - 500,000,000,000,001 = 0), i figured i'd try the following in C to run a little theory.</p> <pre><code>int main() { char* a = "399999999999999"; char* b = "399999999999998"; float da = atof(a); float db = atof(b); printf("%s - %s = %f\n", a, b, da-db); a = "500000000000002"; b = "500000000000001"; da = atof(a); db = atof(b); printf("%s - %s = %f\n", a, b, da-db); } </code></pre> <p>When you run this program, you get the following</p> <pre><code> 399999999999999 - 399999999999998 = 0.000000 500000000000002 - 500000000000001 = 0.000000 </code></pre> <p>It would seem like Google is using simple 32 bit floating precision (the error here), if you switch float for double in the above code, you fix the issue! Could this be it?</p> <p>/mp</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload