StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I don't know anything about SQL, but my guess would be this line:</p> <pre><code> SET result = 0.339946499848118887e-4; </code></pre> <p>If MySQL is anything like other languages I know, then this will first evaluate the right-hand side, and <em>then</em> assign the value to <code>result</code>. No matter what type <code>result</code> is declared to be or what precision it's declared to have, it wouldn't matter if the right-hand side has already lost precision when being evaluated. This is almost surely what is happening here.</p> <p>I can reproduce your results, but If I change that line to </p> <pre><code> SET result = cast('0.339946499848118887e-4' as decimal(30, 25)); </code></pre> <p>(casting from a string instead of from a floating-point constant of unspecified precision) then I correctly get </p> <pre><code>+-----------------------------+ | test(0) | +-----------------------------+ | 0.0000339946499848118887000 | +-----------------------------+ 1 row in set (0.00 sec) </code></pre> <p>as desired. So that's your fix.</p> <hr> <p>BTW, the documentation that <code>scale</code> in <code>DECIMAL(precision, scale)</code> cannot be greater than 30 seems to be in section <a href="https://dev.mysql.com/doc/refman/5.6/en/precision-math-decimal-changes.html" rel="nofollow">12.19.2. DECIMAL Data Type Changes</a>:</p> <blockquote> <p>The declaration syntax for a DECIMAL column is DECIMAL(M,D). The ranges of values for the arguments in MySQL 5.6 are as follows:</p> <p>M is the maximum number of digits (the precision). It has a range of 1 to 65. (Older versions of MySQL permitted a range of 1 to 254.)</p> <p>D is the number of digits to the right of the decimal point (the scale). It has a range of 0 to 30 and must be no larger than M.</p> </blockquote>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload