Databases
StackOverflow2013
Postsdbo
POExtremely slow db.get([keys]) on GAE HR datastore
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POExtremely slow db.get([keys]) on GAE HR datastore
    text
    copied!<p>I am using db.get([keys]) and experiencing extremely slow reads. It takes at least "9008cpu_ms 2125api_cpu_ms" for simple test. <strong>Keys array length is ~200</strong>. <strong>Is it normal?</strong> Entities are small:<br/><br/></p> <pre><code>p1 = db.StringProperty(indexed=False) - ~20 characters p2 = db.StringProperty(indexed=False, required=True) ~10 characters p3 = db.GeoPtProperty(indexed=False, required=True) p4 = db.StringListProperty(indexed=False) 10 items x ~10 characters </code></pre> <p>Total entities in HRD datastore: ~1000. Fetched: ~200.</p> <p>Appstats shows:</p> <pre><code>datastore_v3.RunQuery 9ms (29ms api) datastore_v3.Next 32ms (16ms api) datastore_v3.Next 11ms (16ms api) datastore_v3.Next 16ms (16ms api) datastore_v3.Next 86ms (16ms api) datastore_v3.Next 8ms (16ms api) datastore_v3.Next 84ms (16ms api) datastore_v3.Next 8ms (16ms api) datastore_v3.Next 92ms (16ms api) datastore_v3.Next 14ms (16ms api) datastore_v3.Next 82ms (16ms api) datastore_v3.Next 8ms (16ms api) datastore_v3.Next 86ms (16ms api) datastore_v3.Next 96ms (16ms api) datastore_v3.Next 7ms (16ms api) datastore_v3.Next 92ms (16ms api) datastore_v3.Next 92ms (16ms api) datastore_v3.Next 9ms (16ms api) datastore_v3.Next 89ms (16ms api) datastore_v3.Next 7ms (4ms api) datastore_v3.Get 5692ms (8ms api) datastore_v3.Get 5688ms (8ms api) datastore_v3.Get 5684ms (8ms api)&lt;/code&gt; </code></pre> <p>And hundreds of:</p> <pre><code>datastore_v3.Get ~ 5681ms (8ms api) </code></pre> <p>Source:</p> <pre><code>logging.debug('Fetching ' + str(len(m.keys())) + ' entities') items = db.get(m.keys()) logging.debug('Done fetching items') </code></pre> <p>Log:</p> <pre><code>D 2011-10-30 22:46:41.495 Fetching 238 entities D 2011-10-30 22:46:50.009 Done fetching items W 2011-10-30 22:46:54.407 Full proto too large to save, cleared variables. </code></pre> <p>Update 1 (Monday, October 31, 2011 at 23:33:42 UTC):</p> <p>While searching for possible solution, I have removed StringList property and recreated entities. No changes.</p> <p>Sample entity:</p> <pre><code>ID/Name|description|location|name id=804|Sample description|54.8968721,23.892426|Sample place </code></pre> <p>Update 2(Tuesday, November 01, 2011 at 12:27:31 UTC):</p> <p>Screenshot of Appstats output:</p> <p><img src="https://i.stack.imgur.com/xHtyA.png" alt="Screenshot of Appstats output"></p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload