StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POUse mapReduce on a 'logs' collection to generate HTTP stream
text
Body
copied!<p>MongoDB newbie issue:</p> <p>I have a lot of HTTP logs stored into a collection with the following data structure:</p> <pre><code>{ 'client': { 'ip_address': '1.2.3.4', 'referrer':"http://....", 'user_agent':'Mozilla..." }, 'request':{ "stream": "stream1", "method": "GET", "fragment_id": 97, "date": 13482181, 'response':{ 'status':200, 'size': 654 } } </code></pre> <p>Each document describes an HTTP request (from a client to a content streamer). As each stream is fragmented into smaller pieces, I would like to use "mapReduce" on my collection and then create a "generic stream request" document, as below:</p> <pre><code>{ 'client_ip': '1.2.3.4', 'user_agent': 'Mozilla', 'streams':[ { 'stream':"stream1", 'referrer':'http://...', 'requests':[ { 'fragment_id':97, 'status':200, 'date': 13482181, 'size': 654 ... }, { 'fragment_id':98, 'status':200, 'date': 13482192, 'size': 624 ... }, [...] ] }, [...] ] </code></pre> <p>Here is what I tried:</p> <pre><code>map = function(){ emit({client_ip:this.client.ip,user_agent:this.client.user_agent},{ stream:this.request.stream, referrer:this.client.referer, status:this.response.status, date:this.request.date, size:this.response.total_size, fragment_id:this.request.fragment_infos[1] }); } reduce = function(key,values){ r = {'count':0,'request':[]}; values.forEach(function(v){ r.count += 1; r.request.push(v); }); return r; } </code></pre> <p>but here is what I get as a result:</p> <pre><code>"_id" : { "client_ip" : "1.2.3.4", "user_agent" : "Mozilla\/4.0" }, "value" : { "client_ip" : "1.2.3.4", "user_agent" : "Mozilla\/4.0", "count" : 17, "request" : { "0" : { "client_ip" : "1.2.3.4", "user_agent" : "Mozilla\/4.0", "count" : 2, "request" : { "0" : { "stream" : "stream1.isml", "referrer" : null, "status" : 200, "date" : 1341706566, "size" : 456, "fragment_id" : null, "count" : 1 }, "1" : { "stream" : "stream1.isml", "referrer" : null, "status" : 200, "date" : 1341706566, "size" : null, "fragment_id" : null, "count" : 1 } } }, "1" : { "client_ip" : "1.2.3.4", "user_agent" : "Mozilla\/4.0", "count" : 3, "request" : { "0" : { "client_ip" : "1.2.3.4", "user_agent" : "Mozilla\/4.0", "count" : 2, "request" : { "0" : { "stream" : "stream1.isml", "referrer" : null, "status" : 200, "date" : 1341706568, "size" : null, "fragment_id" : null, "count" : 1 ......... </code></pre> <p>Where am I wrong?</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload