Databases
StackOverflow2013
Postsdbo
POServer Error (Google Apps Engine) when executing custom bulkloader
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POServer Error (Google Apps Engine) when executing custom bulkloader
    text
    copied!<p>I had not succeeded in using the bulkloader option that is officially provided, so i wrote my own bulkloader script (actually post handler that would incrementally load entities from csv into the datastore).</p> <p>The solution worked as follows:<br> 1. I would copy data from a csv file<br> 2. Paste it into a text area in a form on the app<br> 3. Post the form<br> 4. The handler parses the incoming text for headers (column names)<br> 5. Stores rest of lines in list<br> 6. incrementally fetches 100 lines from the list and for each line creates and stores an entity (the type of entity is resolved from the type specified from a select field from the form) </p> <p>Now, this technique would work on my dev server for small inputs of up to 1000 lines, beyond that it would show the following error:</p> <blockquote> <p>Traceback (most recent call last):<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/ext/webapp/<strong>init</strong>.py", line 513, in <strong>call</strong><br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/myApps/ugvotes/ugvotes.py", line 241, in post<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/ext/db/<strong>init</strong>.py", line 893, in put<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore.py", line 291, in Put<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore.py", line 195, in _MakeSyncCall<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 499, in check_success<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 863, in MakeSyncCall<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/apiproxy_stub.py", line 80, in MakeSyncCall<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 933, in _Dynamic_Put<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 806, in __WriteDatastore<br> File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 836, in __WritePickled<br> IOError: [Errno 24] Too many open files: '/tmp/tmpOfgvm3' </p> </blockquote> <p>At first I thought this was an error due to the limitations of the development server, but when i tried to do the same task from the production server, i got the following error: </p> <blockquote> <p>Error: Server Error The server encountered an error and could not complete your request.</p> <p>If the problem persists, please report your problem and mention this error message and the query that caused it.</p> </blockquote> <p>Who knows what could have gone wrong, and what i can do about it?</p> <p>thanks.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload