Databases
StackOverflow2013
Postsdbo
POWhat is the best way to do AppEngine Model Memcaching?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhat is the best way to do AppEngine Model Memcaching?
    primarykey
    data
    text
    <p>Currently my application caches models in memcache like this: </p> <pre><code>memcache.set("somekey", aModel) </code></pre> <p>But Nicks' post at <a href="http://blog.notdot.net/2009/9/Efficient-model-memcaching" rel="noreferrer">http://blog.notdot.net/2009/9/Efficient-model-memcaching</a> suggests that first converting it to protobuffers is a lot more efficient. But after running some tests I found out it's indeed smaller in size, but actually <em>slower</em> (~10%).</p> <p>Do others have the same experience or am I doing something wrong?</p> <p>Test results: <a href="http://1.latest.sofatest.appspot.com/?times=1000" rel="noreferrer">http://1.latest.sofatest.appspot.com/?times=1000</a></p> <pre><code>import pickle import time import uuid from google.appengine.ext import webapp from google.appengine.ext import db from google.appengine.ext.webapp import util from google.appengine.datastore import entity_pb from google.appengine.api import memcache class Person(db.Model): name = db.StringProperty() times = 10000 class MainHandler(webapp.RequestHandler): def get(self): self.response.headers['Content-Type'] = 'text/plain' m = Person(name='Koen Bok') t1 = time.time() for i in xrange(int(self.request.get('times', 1))): key = uuid.uuid4().hex memcache.set(key, m) r = memcache.get(key) self.response.out.write('Pickle took: %.2f' % (time.time() - t1)) t1 = time.time() for i in xrange(int(self.request.get('times', 1))): key = uuid.uuid4().hex memcache.set(key, db.model_to_protobuf(m).Encode()) r = db.model_from_protobuf(entity_pb.EntityProto(memcache.get(key))) self.response.out.write('Proto took: %.2f' % (time.time() - t1)) def main(): application = webapp.WSGIApplication([('/', MainHandler)], debug=True) util.run_wsgi_app(application) if __name__ == '__main__': main() </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USDavid Underhill
    1. USKoen Bok
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWhat is the best way to do AppEngine Model Memcaching?
      1. USKoen Bok
      1. VTBountyStart
    2. VO
      singulars
      1. POWhat is the best way to do AppEngine Model Memcaching?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POWhat is the best way to do AppEngine Model Memcaching?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COI just tried it with really large and complex models too, but the result was about the same.
      singulars
      1. POWhat is the best way to do AppEngine Model Memcaching?
      1. USKoen Bok
    2. COMaybe there is http://docs.python.org/library/timeit.html on GAE? This should show more accurate results, but still - after reading the blog entry you linked to I would expect order of magnitude difference between performance of protobuffers and pickle - and this should be catched by time.time() anyway..
      singulars
      1. POWhat is the best way to do AppEngine Model Memcaching?
      1. USTomasz Zielinski
    3. COi'm using java appengine, so I'm too lazy to test this theory - is pickle() caching the results behind the scenes somewhere, whereas to_protobuf is not? Based on the article, i'm not sure i would expect a full order of magnitude increase in speed, as pickle is still being called even using the protobuf version. the space used could certainly be vastly smaller though.
      singulars
      1. POWhat is the best way to do AppEngine Model Memcaching?
      1. USPeter Recore
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload