Databases
StackOverflow2013
Postsdbo
PORounding to specified absolute decimal precision in C90/99
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PORounding to specified absolute decimal precision in C90/99
    primarykey
    data
    text
    <p>I am working on software that, among other things, converts measured numbers between text and internal (<code>double</code>) representation. A necessary part of the process is to produce text representations with the correct decimal precision based on the statistical uncertainty of the measurement. The needed precision varies with the number, and the least-significant digit in it can be anywhere, including left of the (decimal) units place.</p> <p>Correct rounding is essential for this process, where "correct" means according to the floating-point rounding mode in effect at the time, or at least in a well-defined rounding mode. As such, I need to be careful about (read: avoid) performing intermediate arithmetic on the numbers being handled, because rounding can be sensitive even to the least-significant bit in the internal representation of a number.</p> <p>I think I can do almost all the needed formatting reasonably well with the printf family of functions if I first compute the number of significant digits in the required representation:</p> <pre><code>sprintf(buffer, "%.*e", num_sig_figs - 1, number); </code></pre> <p>There is one class of corner cases that has so far defeated me, however: the one where the most significant (decimal) digit in the measured number is one place right of the least significant digit of the desired-precision representation. In that case, rounding should yield the least (and only) significant digit in the desired result as either 0 or 1, but I haven't been able to devise a way to perform the rounding in a portable(*) way without risk of changing the result. This is similar to what the MPFR function <code>mpfr_prec_round()</code> could do, except that it works in binary precision, whereas I need to use decimal precision.</p> <p>For example, in the default rounding mode (round-to-nearest with ties rounded to even):</p> <ul> <li>0.5 expressed to unit (10^0) precision should be "0" or "0e+00"</li> <li>654 expressed to thousands (10^3) precision should be "1e+03"</li> <li>0.03125 expressed to tenths (10^-1) precision should be "0" or "0e-01" or even "0e+00"</li> </ul> <p>(*) "Portable" here means that the code accurately expresses the computation in standard, portable C99 (or better, C90). It is understood that the actual result may depend on machine details, and it <em>should</em> depend (and be consistent with) the floating-point rounding mode in effect.</p> <p>What options do I have?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USJohn Bollinger
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PORounding to specified absolute decimal precision in C90/99
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PORounding to specified absolute decimal precision in C90/99
      1. This table or related slice is empty.
      1. VTUpMod
    1. COrounding of decimal numbers and doubles doesn't go well together... (decimal fractions are periodic in binary) try to use some fixed point data type...
      singulars
      1. PORounding to specified absolute decimal precision in C90/99
      1. USV-X
    2. COIt would seem these would be handled with code such as `if (x < precision) if (x <= .5*precision) printf("0") else printf("%.0e", precision) else /* Not one of these cases, do other stuff. */…`. Is there a case where what you want differs from what this code would produce? (There is a problem with how `precision` is rounded, since .1 cannot be represented exactly, so you might have some issues exactly at the boundary. Also, negative `x` are not handled, but that is straightforward.)
      singulars
      1. PORounding to specified absolute decimal precision in C90/99
      1. USEric Postpischil
    3. COPlease explain better the case you think is problematic with your `sprintf` method. I think it is the correct method and I'm confused as to what problem you're having.
      singulars
      1. PORounding to specified absolute decimal precision in C90/99
      1. USR..
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload