Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Your code first computes <code>appdelegate.userRed/255.0</code> where <code>appdelegate.userRed</code> is 128. Mathematically, this is 128/255. However, since you are using floating-point arithmetic in a computer, the computer must return a value that is representable in that floating-point system rather than the exact mathematical value. In this case, <code>128/255.0</code> has type <code>double</code>, and the <code>double</code> that is closest to 128/255 is 0.5019607843137254832299731788225471973419189453125.</p> <p>Then your code assigns that value to <code>red</code>. Since <code>red</code> is a <code>float</code>, the value is converted to <code>float</code>. Again, the value must be rounded. The <code>float</code> value that is nearest to 0.5019607843137254832299731788225471973419189453125 is 0.501960813999176025390625.</p> <p>Then you compare <code>red</code> to <code>0.501961</code>. The compiler converts the source text <code>0.501961</code> to the nearest value representable as a <code>double</code>, which is 0.50196099999999999052846533231786452233791351318359375.</p> <p>Since 0.501960813999176025390625 is not equal to 0.50196099999999999052846533231786452233791351318359375, the comparison returns false.</p> <p>It is difficult to suggest alternatives for you because you have not explained why you are making this comparison. If you merely wish to check whether the <code>float</code> value of a pixel is the direct result of converting an integer 128 to <code>float</code> by dividing by 255, then I suggest you use <code>red==128/255.f</code>. (You may also wish to change <code>appdeletegate.userRed/255.0</code> to <code>appdelegate.userRed/255.f</code> so that you are nominally using <code>float</code>, rather than <code>double</code> throughout your code.)</p> <p>However, if you are doing anything more involved with floating-point arithmetic, there are additional considerations. In particular, if a <code>float</code> value is not the direct result of this conversion by dividing by 255 but is the result of additional arithmetic, then it is likely the value will contain additional rounding errors, and comparing it directly to <code>128/255.f</code> will not produce the desired result.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload