Databases
StackOverflow2013
Postsdbo
POHow to find distinct set of tags from multiple entities in app engine
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to find distinct set of tags from multiple entities in app engine
    text
    copied!<p>I am writing a system on app engine that collects "samples" and provides services for querying and analyzing the samples. The data model for a sample looks similar to this:</p> <pre><code>class Sample(ndb.Model): category = ndb.StringProperty() name = ndb.StringProperty() data = ndb.JsonProperty() timestamp = ndb.DateTimeProperty() tags = ndb.StringProperty(repeated = True) </code></pre> <p>As you can see, for each sample there is a set of string tags. For example something like:</p> <pre><code>['CustomerA', '2.0.5', 'featureX', 'logTypeB', ...] </code></pre> <p>I have a handler that allows querying over all samples in the system based upon filters on the base properties and including a set of tags to require. Note: the results set can be very large, so the query supports paging/limits so I return data a bit at a time. That all works.</p> <p>Now when I am putting a user interface on top of this I would like a way to present the user with an autocomplete field for entering additional tags to further filter the results. So for example if they have restricted it down to samples with the following tags:</p> <pre><code>Sample(..., tags=['CustomerA', '2.0.5', 'featureX']) Sample(..., tags=['CustomerA', '2.0.5', 'featureY']) Sample(..., tags=['CustomerB', '2.0.5', 'featureX']) Sample(..., tags=['CustomerB', '2.0.5', 'featureX']) Sample(..., tags=['CustomerB', '2.0.5', 'featureY']) </code></pre> <p>then I want to show them an autocomple that includes:</p> <pre><code>['CustomerA', 'CustomerB', '2.0.5', 'featureX', 'featureY'] </code></pre> <p>In other words I need a handler that can return a unique list of tags that exist in the current set of results. The problem is that I can't see anyway to do this in App Engine without iterating over all result samples (potentially very large) and building up a set of unique tags to return.</p> <p>I could keep a separate set of entities for all tags in the system, but this doesn't solve the problem either. It would allow me to quickly find all the tags that exist over all the Samples in the system, but not restrict it to the set of Samples that pass the current filter.</p> <p>Any ideas on what I could do to implement this in a reasonable way?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload