Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Normally, I'm the one who comes in and explains to people that the number they entered is not representable as a floating-point number, and where the rounding errors are, blah blah blah.</p> <p>This question is much more fun than what we usually see, and it illustrates exactly what's wrong with the crowd wisdom of "floating point is inexact, read '<em>what every computer scientist should know...</em>' lolz".</p> <p><code>36.76662445068359375</code> is not just any 19-digit decimal number. It happens to be a 19 digit decimal number that is also exactly representable in double precision binary floating point. Thus, the initial conversion implicit in:</p> <pre><code>double wtf = 36.76662445068359375000; </code></pre> <p>is exact. <code>wtf</code> contains exactly <code>b100100.11000100010000011</code>, and no rounding has occurred. </p> <p>The spec for NSDecimalNumber says that it represents numbers as a 38 digit decimal mantissa and a decimal exponent in the range [-127,128], so the value in <code>wtf</code> is also exactly representable as an NSDecimalNumber. Thus, we may conclude that <code>numberWithDouble</code> is not delivering a correct conversion. Although I cannot find documentation that claims that this conversion routine is correctly rounded, there is no good reason for it not to be. <strong>This is a real bug</strong>, <a href="http://developer.apple.com/bugreporter/" rel="noreferrer">please report it</a>.</p> <p>I note that the string formatters on iPhoneOS seem to deliver correctly rounded results, so you can probably work around this by first formatting the double as a string with 38 digit precision and then using <code>decimalNumberWithString</code>. Not ideal, but it may work for you.</p>
    singulars
    1. This table or related slice is empty.
    1. POcocoa - I've discovered what I think is a bug with NSDecimalNumber
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USStephen Canon
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POcocoa - I've discovered what I think is a bug with NSDecimalNumber
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COYou might have a point if he were not using numberWithDouble. That message is not defined for NSDecimalNumber but for its super class NSNumber. There is no API for NSDecimalNumber that offers convenience creation of an instance directly from a double.
      singulars
      1. PO
      1. USDavid Schaefgen
    2. CONice detective work on that. If I can create a reproducible test case for this, I'll file a bug report.
      singulars
      1. PO
      1. USBrad Larson
    3. COThanks, I have already filled a bug and I have another proof of it. I will include in my original question.
      singulars
      1. PO
      1. USSpaceDog
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload