Databases
StackOverflow2013
Postsdbo
POOptimizing MySQL Aggregation Query
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POOptimizing MySQL Aggregation Query
    primarykey
    data
    text
    <p>I've got a very large table (~100Million Records) in MySQL that contains information about files. One of the pieces of information is the modified date of each file.</p> <p>I need to write a query that will count the number of files that fit into specified date ranges. To do that I made a small table that specifies these ranges (all in days) and looks like this:</p> <pre><code>DateRanges range_id range_name range_start range_end 1 0-90 0 90 2 91-180 91 180 3 181-365 181 365 4 366-1095 366 1095 5 1096+ 1096 999999999 </code></pre> <p>And wrote a query that looks like this:</p> <pre><code>SELECT r.range_name, sum(IF((DATEDIFF(CURDATE(),t.file_last_access) &gt; r.range_start and DATEDIFF(CURDATE(),t.file_last_access) &lt; r.range_end),1,0)) as FileCount FROM `DateRanges` r, `HugeFileTable` t GROUP BY r.range_name </code></pre> <p>However, quite predictably, this query takes forever to run. I think that is because I am asking MySQL to go through the HugeFileTable 5 times, each time performing the DATEDIFF() calculation on each file. </p> <p>What I want to do instead is to go through the HugeFileTable record by record only once, and for each file increment the count in the appropriate range_name running total. I can't figure out how to do that....</p> <p>Can anyone help out with this?</p> <p>Thanks.</p> <p><strong>EDIT</strong>: MySQL Version: 5.0.45, Tables are MyISAM</p> <p><strong>EDIT2</strong>: Here's the descibe that was asked for in the comments</p> <pre><code>id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE r ALL NULL NULL NULL NULL 5 Using temporary; Using filesort 1 SIMPLE t ALL NULL NULL NULL NULL 96506321 </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USZenshai
    1. USZenshai
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POOptimizing MySQL Aggregation Query
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POOptimizing MySQL Aggregation Query
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POOptimizing MySQL Aggregation Query
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhat version of MySQL are you using?
      singulars
      1. POOptimizing MySQL Aggregation Query
      1. USJordan S. Jones
    2. COcan you run DESCRIBE in front of your query and give us the output so we can see if here is anything in that to be improved
      singulars
      1. POOptimizing MySQL Aggregation Query
      1. UStristanbailey
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload