Databases
StackOverflow2013
Postsdbo
POGoogle App Engine bulkloader issue when using yaml autogenerated configuration and entities with numeric ID
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POGoogle App Engine bulkloader issue when using yaml autogenerated configuration and entities with numeric ID
    primarykey
    data
    text
    <p>My application uses <a href="http://www.allbuttonspressed.com/projects/django-nonrel#documentation" rel="nofollow noreferrer">Django non-rel</a>. I don't have access to model. </p> <p>I have my <code>bulkloader.yaml</code> file autogenerated by <code>appcfg.py create_bulkloader_config</code>. </p> <p>Problem is entities numeric ID's are being imported as string key names. So if I export entity with int ID of, for example, <em>'62'</em>, it gets imported as entity with string key name of <em>'61'</em> which screws up Django.</p> <p><strong>Revelant bulkloader.yaml Fragment:</strong> </p> <pre><code> property_map: - property: __key__ external_name: key export_transform: transform.key_id_or_name_as_string </code></pre> <p>I'm trying to setup download/upload od data using bulkloader, and I want to have data as easy to understand format (like .csv) --- so using <code>bulkloader.py --dump (...)</code> is not a viable option since it gives me sqlite3 files that have entities contents pickled as a single row. </p> <h1>EDIT</h1> <p>I tried doing what @Nick suggested and I got an exception:</p> <pre><code> ErrorOnTransform: Numeric keys are not supported on input at this time. </code></pre> <p>Does this mean that I have to stick to bulkloader.py (that uses that werid sqlite format) or I messed something? ;)</p> <p><strong>Header of Transformer:</strong> </p> <pre><code>- kind: auth_user connector: csv connector_options: encoding: utf-8 skip_import_header_row: True print_export_header_row: True property_map: - property: __key__ external_name: key export_transform: transform.key_id_or_name_as_string import_transform: transform.create_foreign_key('auth_user', key_is_id=True) </code></pre> <p><strong>Whole Stacktrace:</strong> </p> <pre><code> Traceback (most recent call last): File "/opt/google/appengine/google/appengine/tools/adaptive_thread_pool.py", line 150, in WorkOnItems status, instruction = item.PerformWork(self.__thread_pool) File "/opt/google/appengine/google/appengine/tools/bulkloader.py", line 693, in PerformWork transfer_time = self._TransferItem(thread_pool) File "/opt/google/appengine/google/appengine/tools/bulkloader.py", line 848, in _TransferItem self.content = self.request_manager.EncodeContent(self.rows) File "/opt/google/appengine/google/appengine/tools/bulkloader.py", line 1269, in EncodeContent entity = loader.create_entity(values, key_name=key, parent=parent) File "/opt/google/appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 385, in create_entity return self.dict_to_entity(input_dict, self.bulkload_state) File "/opt/google/appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 131, in dict_to_entity instance = self.__create_instance(input_dict, bulkload_state_copy) File "/opt/google/appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 209, in __create_instance 'Numeric keys are not supported on input at this time.') </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USjb.
    1. USjb.
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POGoogle App Engine bulkloader issue when using yaml autogenerated configuration and entities with numeric ID
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POGoogle App Engine bulkloader issue when using yaml autogenerated configuration and entities with numeric ID
      1. USjotr
      1. VTFavorite
    3. VO
      singulars
      1. POGoogle App Engine bulkloader issue when using yaml autogenerated configuration and entities with numeric ID
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload