Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>The errors in floating point calculations are rooted in the way that floating point numbers are stored. Depending on which precision you are using (usually either single (float) or double-precision). Double-precision floats take more space, but are roughly twice as precise.</p> <p>Floating point numbers are typically stored in an IEEE format and thus only the most significant bits are included.</p> <p>In a string of bits stored in this way to represent a floating point number, there are different parts that comprise the number. First, there is a sign bit indicating whether the number is positive or negative. Next, the exponent portion of the number is stored (in single-precision floats, this takes 8 bits). After the exponent, all remaining bits represent the significant digits of the number. Thus, the bits 1 10000000 01000000000000000000000 represent -0.5 The sign bit (first one) signifies that the number is negative. The next eight bits are the exponent. Usually, the exponent is stored with a bias so that small numbers may be stored precisely in addition to large ones. To do this, the exponent used for calculations is -127 (if an 8 bit exponent) + the exponent bits interpreted as an integer (in our case 128). All remaining bits signify the significant digits of the number starting with the ones place and moving rightward from there we cut the value in half each time (1 is 1, but 01 is 1/2 and 001 is 1/4) in our case the total number is -0.5 (-1 for the sign bit * 2^1 for the exponent * 0.5 for the remaining bits)</p> <p>For further example, <a href="http://www.h-schmidt.net/FloatConverter/IEEE754.html" rel="nofollow">here</a> is a converter that uses checkboxes to indicate the bits. At worst, you could write a bot to test all of the combinations you care about.</p> <p>In general, if it cannot be described with a fraction that is not directly made of combinations of powers of two, it will be rounded. (If you can't get it with 1/2 + 1/4 + 1/8 etc. it will probably be rounded)</p>
    singulars
    1. This table or related slice is empty.
    1. POCalculate which numbers cause problems when converting decimal to floating point?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USPsetmaj
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COYes I am familiar with this page. I just want to find a way to document all of the numbers that have an issue for a certain range. I am using this for CNC programming, and when we enter 6.2, the machine won't actually run. So I want to document the problem #'s in a database/spreadsheet
      singulars
      1. PO
      1. USuser2039871
    2. COIt might be easier to find the numbers you can use rather than the ones you cannot. In that direction, you could automate this by trying each bit pattern for the numbers in the range you desire and storing the result as a number you can use. Otherwise, you could probably choose a scale at which you want to know which numbers will round (e.g. 1.1, 1.2, 1.3, 1.4, ... or 1.01, 1.02, 1.03, ...) and try them entering the ones that when converted to a String are not the same as the ones you entered as bad numbers.
      singulars
      1. PO
      1. USPsetmaj
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload