Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Thanks very much for the suggestions to check the error logs, I had somehow missed an out of memory error I was getting.</p> <p>The problem was in fact caused by the way I was using the <a href="http://www.yiiframework.com/doc/api/1.1/CActiveDataProvider/" rel="nofollow">CActiveDataProvider</a> from the Yii Framework. Reading straight from the DataProvider as I was doing in my question was reading each row into memory, as the script ran on this meant I eventually ran out of memory available to PHP.</p> <p>There are a couple of ways to fix this, one is to set pagination on the dataprovider to a smaller number of records and to manually iterate over the data, loading only the pagesize into memory each iteration.</p> <p>The option I went for is to use a <a href="http://www.yiiframework.com/doc/api/1.1/CDataProviderIterator" rel="nofollow">CDataProviderIterator</a> to handle this for me <code>$iterator = new CDataProviderIterator($dp);</code> this prevents memory filling up with the records I'm retrieving. </p> <p>Note that I also had to add an <code>ob_flush();</code> call to prevent the output buffer from filling up with the CSV contents iteslf.</p> <p>For reference I ended up with the following:</p> <pre><code>&lt;?php header('Content-type: text/csv'); header('Content-Disposition: attachment; filename="vacancies.csv"'); set_time_limit(240); $outstream = fopen("php://output", 'w'); $headings = array( $vacancy-&gt;getAttributeLabel('vacancy_id'), ... ); fputcsv($outstream, $headings, ',', '"'); $iterator = new CDataProviderIterator($dp); // create an iterator instead of just using the dataprovider foreach($iterator as $vacancy){ // use the new iterator here $row = array( $vacancy-&gt;vacancy_id, ... ); fputcsv($outstream, $row, ',', '"'); ob_flush(); // explicitly call a flush to avoid filling the buffer } fclose($outstream); ?&gt; </code></pre> <p>Would not have thought to go back and look at the logs again without the suggestion so many thanks :)</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload