Databases
StackOverflow2013
Postsdbo
POmysql query - optimizing existing MAX-MIN query for a huge table
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POmysql query - optimizing existing MAX-MIN query for a huge table
    primarykey
    data
    text
    <p>I have a more or less good working query (concerning to the result) but it takes about 45seconds to be processed. That's definitely too long for presenting the data in a GUI.<br/> So my demand is to find a much faster/efficient query (something around a few milliseconds would be nice) My data table has something around <strike>3000</strike> ~2,619,395 entries and is still growing.</p> <p>Schema:</p> <pre><code>num | station | fetchDate | exportValue | error 1 | PS1 | 2010-10-01 07:05:17 | 300 | 0 2 | PS2 | 2010-10-01 07:05:19 | 297 | 0 923 | PS1 | 2011-11-13 14:45:47 | 82771 | 0 </code></pre> <p>Explanation</p> <ul> <li>the exportValue is always incrementing</li> <li>the exportValue represents the actual absolute value</li> <li>in my case there are 10 stations</li> <li>every ~15 minutes 10 new entries are written to the table</li> <li>error is just an indicator for a proper working station</li> </ul> <p>Working query:</p> <pre><code>select YEAR(fetchDate), station, Max(exportValue)-MIN(exportValue) from registros where exportValue &gt; 0 and error = 0 group by station, YEAR(fetchDate) order by YEAR(fetchDate), station </code></pre> <p>Output:</p> <pre><code>Year | station | Max-Min 2008 | PS1 | 24012 2008 | PS2 | 23709 2009 | PS1 | 28102 2009 | PS2 | 25098 </code></pre> <p>My thoughts on it:</p> <ol> <li>writing several queries with between statements like 'between 2008-01-01 and 2008-01-02' to fetch the MIN(exportValue) and between 2008-12-30 and 2008-12-31' to grab the MAX(exportValue) - Problem: a lot of queries and the problem with having no data in a specified time range (it's not guaranteed that there will be data)</li> <li>limiting the resultset to my 10 stations only with using order by MIN(fetchDate) - problem: takes also a long time to process the query</li> </ol> <p>Additional Info:<br/> I'm using the query in a JAVA Application. That means, it would be possible to do some post-processing on the resultset if necessary. (JPA 2.0)</p> <p>Any help/approaches/ideas are very appreciated. Thanks in advance.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USRonH
    1. USRonH
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POmysql query - optimizing existing MAX-MIN query for a huge table
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload