StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHow to count both sides of many-to-many relationship in Google App Engine
text
Body
copied!<p>Consider a GAE (python) app that lets users comment on songs. The expected number of users is 1,000,000+. The expected number of songs is 5,000.</p> <p>The app must be able to:</p> <ul> <li>Give the number of songs a user has commented on</li> <li>Give the number of users who have commented on a song</li> </ul> <p>Counter management must be transactional so that they always reflect the underlying data.</p> <p>It seems GAE apps must keep these types of counts calculated at all times since querying for them at request time would be inefficient.</p> <p><strong>My Data Model</strong></p> <pre><code>class Song(BaseModel): name = db.StringProperty() # Number of users commenting on the song user_count = db.IntegerProperty('user count', default=0, required=True) date_added = db.DateTimeProperty('date added', False, True) date_updated = db.DateTimeProperty('date updated', True, False) class User(BaseModel): email = db.StringProperty() # Number of songs commented on by the user song_count = db.IntegerProperty('song count', default=0, required=True) date_added = db.DateTimeProperty('date added', False, True) date_updated = db.DateTimeProperty('date updated', True, False) class SongUser(BaseModel): # Will be child of User song = db.ReferenceProperty(Song, required=True, collection_name='songs') comment = db.StringProperty('comment', required=True) date_added = db.DateTimeProperty('date added', False, True) date_updated = db.DateTimeProperty('date updated', True, False) </code></pre> <p><strong>Code</strong><br> This handles the user's song count transactionally but not the song's user count.</p> <pre><code>s = Song(name='Hey Jude') s.put() u = User(email='me@example.com') u.put() def add_mapping(song_key, song_comment, user_key): u = User.get(user_key) su = SongUser(parent=u, song=song_key, song_comment=song_comment, user=u); u.song_count += 1 u.put() su.put() # Transactionally add mapping and increase user's song count db.run_in_transaction(add_mapping, s.key(), 'Awesome', u.key()) # Increase song's user count (non-transactional) s.user_count += 1 s.put() </code></pre> <p><strong>The question is:</strong> How can I manage both counters transactionally?</p> <p>Based on my understanding this would be impossible since User, Song, and SongUser would have to be a part of the same <a href="http://code.google.com/appengine/docs/python/datastore/keysandentitygroups.html#Entity_Groups_Ancestors_and_Paths" rel="nofollow noreferrer">entity group</a>. They can't be in one entity group because then all my data would be in one group and it could not be distributed by user.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload