Databases
StackOverflow2013
Postsdbo
POZend_DB whats the right strategy to export large amounts of data to csv? – chunks – fetchAll – fetchRow
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POZend_DB whats the right strategy to export large amounts of data to csv? – chunks – fetchAll – fetchRow
    primarykey
    data
    text
    <p>I have to export a huge amount of data. Also I have to transform every record a little bit through php. Whats the right strategy to export large amounts of data? </p> <ul> <li>Do I split Zend_Db requests in multiple chunked queries with limit(1000,x)? </li> <li>Do I use fetchAll or fetchRow? </li> <li>Which fetchrow or fetchall is performing better considering high performance?</li> </ul> <p>I cannot use SQL OUTFILE since I have to interpret the xml/html coming from one column. As far as I know mysql is not able to do this. This means for me, I either can use fetchrow or fetchall, but I cannot process on the mysql server. Since I'm fetching a huge amount of data fetchAll may leed to a out of memory of php. So I'm not sure if I can avoid this by using fetchrow or if I have to use chunks anyways? Is fetchrow slower than fetchall?</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USManuel
    1. USManuel
    plurals
    1. PL
      singulars
      1. LTDuplicate
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COjusty `query` with `SELECT whatever INTO OUTFILE '/tmp/result.csv'`
      singulars
      1. POZend_DB whats the right strategy to export large amounts of data to csv? – chunks – fetchAll – fetchRow
      1. USGordon
    2. COHey, this is not a duplicate question! I cannot use INTO OUTFILE since I have to process every record in php. Thus the performance of chucks with fetchAll vs. select all with fetchrow matters. I haven't found the answer yet :( links?
      singulars
      1. POZend_DB whats the right strategy to export large amounts of data to csv? – chunks – fetchAll – fetchRow
      1. USManuel
    3. COit depends on what you need to process. Since you will write the SQL Query in PHP, you could add any processing logic into the query and then let your database do the calculations and write the result into the outfile. But, okay, update your question please and I'll vote to reopen this. Although you could simply benchmark possible solutions. Will be more accurate to test against your own code than asking here anyway.
      singulars
      1. POZend_DB whats the right strategy to export large amounts of data to csv? – chunks – fetchAll – fetchRow
      1. USGordon
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload