Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You are using the "spherical law of cosines" formula, which is susceptable to rounding error at small distances (like zero!) -- see <a href="http://en.wikipedia.org/wiki/Haversine_formula" rel="nofollow noreferrer">this discussion</a>. The long expression that is fed into acos() is evaluating to slightly more than 1.0, which is out of bounds.</p> <p>Here's the problem illustrated using Python to do the calculations:</p> <pre><code>&gt;&gt;&gt; from math import sin, cos, acos, radians &gt;&gt;&gt; lat = radians(45.266708) &gt;&gt;&gt; long_expression = sin(lat) * sin(lat) + cos(lat) * cos(lat) * cos(0.0) &gt;&gt;&gt; repr(long_expression) '1.0000000000000002' &gt;&gt;&gt; acos(long_expression) Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; ValueError: math domain error &gt;&gt;&gt; </code></pre> <p>It seems that MySQL is substituting <code>NULL</code> instead of raising an exception. I know little about MySQL, but you should be able to overcome that by doing something like <code>ifnull(acos(long_expression), 0.0)</code> or <code>coalesce(acos(long_expression), 0.0)</code>.</p> <p>Alternatively you could use the haversine formula, which shifts the rounding problem from your door-step to the opposite side of the earth.</p> <p><strong>Update:</strong> I've tested using that formula in Python to calculate the should-be-zero distance between a point and the same point, for each of the 37582 unique (lat, lon) 2-tuples in a <a href="http://www.boutell.com/zipcodes/" rel="nofollow noreferrer">file of US zip codes</a>. </p> <p>Of these:</p> <ul> <li>31591 (84.1%) produced a zero distance</li> <li>4244 (11.3%) produced a distance of 9.5 cm.</li> <li>831 (2.2%) produced a distance of 13.4 cm.</li> <li>916 (2.4%) produced a "cos" value of 1.0000000000000002 which would cause an exception in acos() if not detected and avoided.</li> </ul> <p>It appears that explicit testing for <code>lat1 = lat2 and lon1 = lon2</code> and avoiding the formula in that case (just use zero) might be a good idea -- it would give a consistent answer and avoid puzzlement.</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload