Databases
StackOverflow2013
Postsdbo
POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
    primarykey
    data
    text
    <p>I'm trying to see if a specific algorithm can be translated to the kind of map-reduce index RavenDB/CouchDB uses, ie, "pre-computed" map-reduce (which means the indexes are refreshed on insertion and updates, not when performing the actual query).</p> <p>Let's say we have a typical online store with 50,000 products, grouped in categories. Every product has a collection of "Attribute Values", ie, something like "[Red, Round, Metal]".</p> <p>Since we have so much products on our website, and there's probably a lot of items in each of the categories, we want to give the user another way to "filter" the products he's currently seeing. </p> <p>For example, if a category is "Less than $20", there's a whole bunch of products in this category. But our user only need to see products which are less than $20 and Red. Unfortunately, there's no sub-category "Red" in the "Less than $20" category.</p> <p>Our algorithm would take the current list of products, and generate a list of "interesting" Attributes and Attribute Values, ie, given a list of products, it would output something like:</p> <pre><code>Color Red (40) Blue (32) Yellow (17) Material Metal (37) Plastic (36) Wood (23) Shape Square (56) Round (17) Cylinder (12) </code></pre> <p>Could this sort of algorithm be somehow pre-computed à la RavenDB/CouchDB map-reduce index? If not, why exactly (so I can identify that kind of algorithm in the future) and if yes, how?</p> <p>A <a href="http://wegel.ca/samples/MapReduceTest.zip" rel="nofollow">C# 4.0 Visual Studio Test Solution</a> is available that demonstrates the potential data structures and sample data, as well as a try at a map-reduce implementation (which doesn't seem to be pre-computable).</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USSimon Labrecque
    1. USSimon Labrecque
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
      1. USMikael Svenson
      1. VTFavorite
    3. VO
      singulars
      1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
      1. USVictor Nicollet
      1. VTFavorite
    1. COFor 50.000 items you could keep all data in memory in just about any structure and loop over it. Simple optimizations could also be implemented. That said, how many queries a second are you looking at?
      singulars
      1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
      1. USMikael Svenson
    2. COKeeping the products in memory is precisely what I'm doing right now, and it's working pretty good. I'm in a different context, however (relational DB with O/R mapper). Startup time is important in the application I'm working on, and currently it's starting to be a problem as most of the data has to be retrieved and put in memory during startup. To make this a good challenge, we're looking at any number of queries/sec that you'll find challenging ;) Ie, the solution has to scale.
      singulars
      1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
      1. USSimon Labrecque
    3. COIt's not quite clear what you're after, as <20$ isn't in the sample output. If the user wants <20$ AND Red, and there are no Red<$20, what do you show him? 'interesting' red things (even>$20) or what?
      singulars
      1. POCan a "pre-computed" map-reduce index (à la RavenDB/CouchDB) be used for this kind of algorithm?
      1. USsmirkingman
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload