Databases
StackOverflow2013
Postsdbo
POOdd output behavior with Ruby on OS X
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POOdd output behavior with Ruby on OS X
    text
    copied!<p>I have a simple program written in Ruby. The code is:</p> <pre><code>#!/usr/bin/ruby # odd behavior here j = 1.11 while j &lt; 2 print "iteration #{j}\n" j += 0.01 end </code></pre> <p>I am running it on a MacBook Air with Mountain Lion. My version of Ruby is:</p> <pre><code>ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-darwin11.4.2] </code></pre> <p>The odd behavior I'm seeing is:</p> <pre><code>iteration 1.1 iteration 1.11 iteration 1.12 iteration 1.1300000000000001 iteration 1.1400000000000001 </code></pre> <p>Now if I change j to 1.13, I get the following resutls (I'm removing some of the output for the sake of brevity):</p> <pre><code>iteration 1.13 iteration 1.14 iteration 1.15 ... iteration 1.36 iteration 1.37 iteration 1.3800000000000001 iteration 1.3900000000000001 iteration 1.4000000000000001 iteration 1.4100000000000001 iteration 1.4200000000000002 </code></pre> <p>What is going on here? Initially, I thought maybe it had something to do with how the values were stored in j and that 1.13 has some special attribute. However, starting j at 1.13 turned that theory on its head. What's really getting me is that this doesn't seem to be consistent. In other words, it seems somewhat arbitrary (although I'm sure it's not) where the odd "0000000000"s come into play.</p> <p>To make matters worse, if I change the "j &lt; 2" to "j &lt; 5" I get even stranger behavior.</p> <pre><code>... iteration 4.85999999999994 iteration 4.86999999999994 iteration 4.8799999999999395 iteration 4.889999999999939 iteration 4.899999999999939 iteration 4.909999999999939 iteration 4.919999999999939 iteration 4.929999999999938 iteration 4.939999999999938 iteration 4.949999999999938 iteration 4.959999999999938 iteration 4.969999999999938 iteration 4.979999999999937 iteration 4.989999999999937 iteration 4.999999999999937 </code></pre> <p>I've tried Googling this but to be honest, I'm not really sure where to start. I've found a few threads where people are seeing odd behavior with to_d but nothing that specifically answers my question. Additionally, I don't know enough about Ruby to fully understand what's going on. It's got to be an issue with precision and how Ruby stores numbers, but I'm not sure where to look.</p> <p>Any nudges in the right direction are greatly appreciated! Thanks!</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload