Databases
StackOverflow2013
Postsdbo
POCrunching Data in MongoDB. Aggregate or MR?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POCrunching Data in MongoDB. Aggregate or MR?
    primarykey
    data
    text
    <p>I have a collection with this kind of data</p> <pre><code>{"uid":"uirerb68624gdbnxcc", "location": "AA", "hit": "foodcat", "aggdone": "n"} {"uid": "hdbhdhu7347fhdufh", "location": "BB", "hit": "travelcat", "aggdone": "n"} {"uid": "uirerb68624gdbnxcc", "location": "CC", "hit": "travelcat", "aggdone": "n"} </code></pre> <p>So, you can see that there is one user who has more "hit"s than the other user. What do you suggest me to do to process these data above and push them in a different collection (i am free to do it once every week. I do not need it real time) like the following.</p> <pre><code>{"uid" : "uirerb68624gdbnxcc", "total_hits": 2} {"uid" : "hdbhdhu7347fhdufh", "total_hits": 1} </code></pre> <p>Also, one thing very important, if you see the first collection, there is a key called "aggdone". which should act as a flag and should be marked "y" with all the data processed. So that next week when I start doing the same process I do not deal with all the data.</p> <p>Please help me. Also, any suggestion/comments on the data design is welcome.</p> <p>Thanks in advance.</p> <p>Mongo version 2.4.8 </p> <p>In response to the comment ------ </p> <p>I have a collection with documents like the following. </p> <pre><code>{ "gpcId": "CGP_52400c5e301d5", "storageCreateTime": "1380270840", "autoUid": "1ea548e1-8970-4833-bc27-16871b0cba4a", "reportingTime": "1380271111", "userIP": "117.200.181.36" } </code></pre> <p>So the autoUid can repeat many times for a particular gpcId with different reporting time. So that means we can (logically) get a list of top users (based on autoUid). This is exactly what i want to get and store in a different collection. So, ideally the resultant collection should look like </p> <pre><code>{ "gpcId": "CGP_52400c5e301d5", "autoUid": "1ea548e1-8970-4833-bc27-16871b0cba4a" "total_count": 10 } { "gpcId": "CGP_52400c5e301d5", "autoUid": "06a02b72-8470-4492-af5f-96ea0d1e8235" "total_count": 9 } . . . </code></pre> <p>So, I would like to get some directions about how to achieve this. I have already mentioned my Mongo version. I am free to use PHP or Python at the back-end to do this.</p> <p>I am sorry if I was not very clear first time. </p> <p>Please tell me if something is still not clear.</p> <p>Also, I have been trying to use map-reduce but i think that code is pointless to give here because i could not achieve any result which is even near to what I want. </p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USSRC
    1. USSRC
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POCrunching Data in MongoDB. Aggregate or MR?
      1. This table or related slice is empty.
      1. VTDownMod
    2. VO
      singulars
      1. POCrunching Data in MongoDB. Aggregate or MR?
      1. This table or related slice is empty.
      1. VTDownMod
    1. CODo you need help with a particular implementation? If so, you'll need to provide what you've tried so we can help. Otherwise, questions asking for code like this are generally marked to be held/closed.
      singulars
      1. POCrunching Data in MongoDB. Aggregate or MR?
      1. USWiredPrairie
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload