Databases
StackOverflow2013
Postsdbo
POIn MongoDB, how to use the aggregation framework to have little docs
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POIn MongoDB, how to use the aggregation framework to have little docs
    primarykey
    data
    text
    <p>i'm trying to play with the aggregation framework but i have an issue. I need to known how many people in my database have purchase something during the last month.</p> <p>To do that i use this : </p> <pre><code>db.account.aggregate([ {$project : {civility : 1, 'purchase.date' : 1 }}, {$match: {civility : 1 ,'purchase.date': {$gte: new Date('02/02/2013'), $lt: new Date('02/03/2013')} }}, {$unwind: '$purchase'}, {$match: {civility : 1 ,'purchase.date': {$gte: new Date('02/02/2013'), $lt: new Date('02/03/2013')} }}, {$group: {_id: '$_id', total_buy : {$sum : 1}}}, {$match: {total_buy: {$gte: 2}}}, {$group: {_id: null, total_buyer : {$sum : 1}}} ]) </code></pre> <p>I have this response</p> <pre><code>{ "result" : [ { "_id" : null, "total_buyer" : 4443 } ], "ok" : 1 } </code></pre> <p>this query work because the range of date I use use is small but if i use the same query with a range of date wich is larger like this : </p> <pre><code>db.account.aggregate([ {$project : {civility : 1, 'purchase.date' : 1 }}, {$match: {civility : 1 ,'purchase.date': {$gte: new Date('02/01/2013'), $lt: new Date('03/01/2013')} }}, {$unwind: '$purchase'}, {$match: {civility : 1 ,'purchase.date': {$gte: new Date('02/01/2013'), $lt: new Date('03/01/2013')} }}, {$group: {_id: '$_id', total_buy : {$sum : 1}}}, {$match: {total_buy: {$gte: 2}}}, {$group: {_id: null, total_buyer : {$sum : 1}}} ]) </code></pre> <p>i have this : </p> <pre><code>{ "errmsg" : "exception: sharded pipeline failed on shard shard0000: { errmsg: \"exception: aggregation result exceeds maximum document size (16MB)\", code: 16389, ok: 0.0 }", "code" : 16390, "ok" : 0 } </code></pre> <p>is there something i do wrong or i can't do what i need to do ?</p> <p>thanks in advance</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USTouillettes
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POIn MongoDB, how to use the aggregation framework to have little docs
      1. USdg123
      1. VTFavorite
    2. VO
      singulars
      1. POIn MongoDB, how to use the aggregation framework to have little docs
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POIn MongoDB, how to use the aggregation framework to have little docs
      1. This table or related slice is empty.
      1. VTUpMod
    1. COprior to the first $group everything is done on the shards in parallel but to group results each shard's result is brought to mongos but it is limited to 16MB - if you can maximize reduction of the document prior to $group - for example, once you match on the date after $unwind you don't need it anymore, nor civility, right? You could try {$project:{_id:1}} before $group since that's all you are using and see if that gets you under the limit.
      singulars
      1. POIn MongoDB, how to use the aggregation framework to have little docs
      1. USAsya Kamsky
    2. COIn addition, if that doesn't work because you have huge number of documents in each shard, you can always split aggregation and aggregate by week at a time instead of by month and then add things on the application side.
      singulars
      1. POIn MongoDB, how to use the aggregation framework to have little docs
      1. USAsya Kamsky
    3. COi try all of your advice and i keep you post. Thanks
      singulars
      1. POIn MongoDB, how to use the aggregation framework to have little docs
      1. USTouillettes
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload