Databases
StackOverflow2013
Postsdbo
POEmulate floating point string conversion behaviour of Linux on Windows
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POEmulate floating point string conversion behaviour of Linux on Windows
    primarykey
    data
    text
    <p>I've encountered an annoying problem in outputting a floating point number. When I format 11.545 with a precision of 2 decimal points on Windows it outputs "11.55", as I would expect. However, when I do the same on Linux the output is "11.54"!</p> <p>I originally encountered the problem in Python, but further investigation showed that the difference is in the underlying C runtime library. (The architecture is x86-x64 in both cases.) Running the following line of C produces the different results on Windows and Linux, same as it does in Python.</p> <pre><code>printf("%.2f", 11.545); </code></pre> <p>To shed more light on this I printed the number to 20 decimal places (<code>"%.20f"</code>):</p> <pre><code>Windows: 11.54500000000000000000 Linux: 11.54499999999999992895 </code></pre> <p>I know that 11.545 cannot be stored precisely as a binary number. So what appears to be happening is that Linux outputs the number it's actually stored with the best possible precision, while Windows outputs the simplest decimal representation of it, ie. tries to guess what the user most likely meant.</p> <p>My question is: <em>is there any (reasonable) way to emulate the Linux behaviour on Windows?</em></p> <p>(While the Windows behaviour is certainly the intuitive one, in my case I actually need to compare the output of a Windows program with that of a Linux program and the Windows one is the only one I can change. By the way, I tried to look at the Windows source of <code>printf</code>, but the actual function that does the float->string conversion is <code>_cfltcvt_l</code> and its source doesn't appear to be available.)</p> <p><strong>EDIT:</strong> the plot thickens! The theory about this being caused by an imprecise representation might be wrong, because <strong>0.125</strong> does have an exact binary representation and it's still different when output with <code>'%.2f' % 0.125</code>:</p> <pre><code>Windows: 0.13 Linux: 0.12 </code></pre> <p>However, <code>round(0.125, 2)</code> returns 0.13 on both Windows and Linux.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USEMP
    1. USEMP
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POEmulate floating point string conversion behaviour of Linux on Windows
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POEmulate floating point string conversion behaviour of Linux on Windows
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POEmulate floating point string conversion behaviour of Linux on Windows
      1. This table or related slice is empty.
      1. VTUpMod
    1. COComparing floats by converting to strings and comparing the strings is a fundamentally nonportable idea. Also, if you're doing math to come to these values, you could normally end up with different results anyway (for example, 32, 64 or 80-bit internal FPU precision; different precision of libm functions), before it even gets to formatting.
      singulars
      1. POEmulate floating point string conversion behaviour of Linux on Windows
      1. USGlenn Maynard
    2. COA double has about 15 decimal digits of precision - at least for MSVC. I'd have assumed that it would be similar for an x86 Linux build, but maybe they have some techniques to provide more (is a `double` on the Linux build 8 bytes? Is `DBL_DIG` in `float.h` something larger than 15?) What is happening on your Windows build is correct, since to the 15 digits of precision, the value does represent 11.545. Welcome to the weird and not always so wonderful world of floating point...
      singulars
      1. POEmulate floating point string conversion behaviour of Linux on Windows
      1. USMichael Burr
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload