StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>With this design, I'm afraid your Tag Entity could be a bottleneck, especially if you expect some tags to be very common. Three specific issues I can think of are efficiency of your gets and puts, write contention and exploding indexes. Let's look at stackoverflow for an example - there are 14,000 posts tagged "java" right now.</p> <ol> <li>That means every time you need to fetch your java tag entity you are pulling back 14k's worth of key data from the datastore. then you are sending it all back when you do a put. that could add up to a lot of bytes.</li> <li>In addition to the bytes going back and forth, each put will require indexes to be updated. each entry in the ListProperty maps to a separate index entry. so now you're doing lots of index updates. which leads us to number 3...</li> <li>Exploding Indexes. each entity has a limit on how many index entries it can have. I think the limit is 5000 per entity. so that is actually a hard limit on how many posts could ever have the same tag.</li> </ol> <p>Further Reading:</p> <ul> <li><p>this <a href="http://groups.google.com/group/google-appengine/browse_thread/thread/c2e9203028f26dc9/9efec6051d4e16c3?lnk=gst&q=size+of+list+property#9efec6051d4e16c3" rel="noreferrer">post</a> touches on some of the issues with large lists</p></li> <li><p><a href="http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes" rel="noreferrer">Exploding indexes</a></p></li> </ul> <p>The good news is, some of your requirements would be easily handled by just the Post entity. For example, you could easily find all the posts that have <em>all</em> of a list of tags with a query filter like this:</p> <pre><code>Query q = pm.newQuery(Post.class) q.setFilter("tags" == 'Java' && "tags == 'appengine'"); </code></pre> <p>For all posts with <em>either</em> java or appengine tags, you would need to do one query for each tag, then combine the results yourself. The datastore doesn't handle OR/IN type operations right now.</p> <p>Finding related posts sounds tricky. I'll think about that after some coffee.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload