Databases
StackOverflow2013
Postsdbo
POphp+odbc writing to files limitation
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POphp+odbc writing to files limitation
    primarykey
    data
    text
    <p>i got a function in PHP to read table from ODBC (to IBM AS400) and write it to a text file on daily basis. it works fine until it reach more than 1GB++. Then it just stop to some rows and didn't write completely.</p> <pre><code>function write_data_to_txt($table_new, $query) { global $path_data; global $odbc_db, $date2; if(!($odbc_rs = odbc_exec($odbc_db,$query))) die("Error executing query $query"); $num_cols = odbc_num_fields($odbc_rs); $path_folder = $path_data.$table_new."/"; if (!file_exists($path_folder)) mkdir ($path_folder,0777); $filename1 = $path_folder. $table_new. "_" . $date2 . ".txt"; $comma = "|"; $newline = chr(13).chr(10); $handle = fopen($filename1, "w+"); if (is_writable($filename1)) { $ctr=0; while(odbc_fetch_row($odbc_rs)) { //function for writing all field // for($i=1; $i&lt;=$num_cols; $i++) // { // $data = odbc_result($odbc_rs, $i); // if (!fwrite($handle, $data) || !fwrite($handle, $comma)) { // print "Cannot write to file ($filename1)"; // exit; // } //} //end of function writing all field $data = odbc_result($odbc_rs, 1); fwrite($handle,$ctr.$comma.$data.$newline); $ctr++; } echo "Write Success. Row = $ctr &lt;br&gt;&lt;br&gt;"; } else { echo "Write Failed&lt;br&gt;&lt;br&gt;"; } fclose($handle); } </code></pre> <p>no errors, just success message but it should be 3,690,498 rows (and still increase) but i just got roughly 3,670,009 rows</p> <p>My query is ordinary select like : </p> <pre><code>select field1 , field2, field3 , field4, fieldetc from table1 </code></pre> <p>What i try and what i assume :</p> <ol> <li>I think it was fwrite limitation so i try not to write all field (just write $ctr and 1st record) but it still stuck in same row.. so i assume its not about fwrite exceed limit..</li> <li>I try to reduce field i select and it can works completely!! so i assume it have some limitation on odbc.</li> <li>I try to use same odbc datasource with SQL Server and try to select all field and it give me complete rows. So i assume its not odbc limitation.</li> <li>Even i try on 64 bits machine but it even worse, it just return roughly 3,145,812 rows.. So i assume it's not about 32/64 bit infrastructure. </li> <li>I try to increase memory_limit in php ini to 1024mb but it didnt work also.. </li> </ol> <p>Is there anyone know if i need to set something in my PHP to odbc connection?? </p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USDaniel Robertus
    1. USDaniel Robertus
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COThis is buffered output, yeah? Are you flushing the buffer when you're done writing? That may be a dumb question, but I've never done this much file output in php before.
      singulars
      1. POphp+odbc writing to files limitation
      1. USmanchicken
    2. CO@manchicken i dont buffer the output.. i fwrite from each field it read on each line...
      singulars
      1. POphp+odbc writing to files limitation
      1. USDaniel Robertus
    3. COYeah, but isn't that a buffered stream? I believe that `fopen()` and `fwrite()` by default create and use buffered streams. You may want to have a look-see at the link below and mark your stream as unbuffered (which will have a performance cost), or simply make sure you flush it before you `fclose()`. http://www.php.net/manual/en/function.stream-set-write-buffer.php
      singulars
      1. POphp+odbc writing to files limitation
      1. USmanchicken
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload