Databases
StackOverflow2013
Postsdbo
POHow can I utilize MongoDB's Map-Reduce while reducing the impact...staying as data store agnostic as possible?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow can I utilize MongoDB's Map-Reduce while reducing the impact...staying as data store agnostic as possible?
    primarykey
    data
    text
    <p>I am posing this question as it relates to a C# solution, however, I faced the same quandary in a RoR solution and simply opted to use Map-Reduce to its fullest, abandoning all hope of abstracting the data store.</p> <p>MongoDB Map-Reduce seems to be THE way to perform pivots as well as other reporting queries. An alternative, which is the typical document repository manner, such as is encouraged by typical EntityFramework (EF) folks, is to move the logic to the application layer.</p> <p>Without getting deep into arguments of the relative advantages of each approach, the amount of data within the data store is proven to be too large to fetch it all up into the application layer.</p> <p>The following code is a proof-of-concept (POC), which yields results, but begs the question I am asking here, is there a way to reduce the impact of using Map-Reduce within a C# (any .NET) solution?</p> <p>Data Models used throughout:</p> <pre><code>public class Call { [BsonId] [BsonRepresentation(BsonType.ObjectId)] public string Id { get; set; } public DateTime? StartTime { get; set; } public DateTime? EndTime { get; set; } public Agent Agent { get; set; } public Caller Caller { get; set; } } public class Agent : Person { public DateTime JoinedCompany { get; set; } } public class Caller : Person { } </code></pre> <p>Data Models used within the Map-Reduce POC:</p> <pre><code>public class AgentCallSummary { public ObjectId _id; public AgentCallAggregateValues value; public class AgentCallAggregateValues { public int count; public int totalTimeOnCall; } } </code></pre> <p>The following code depends upon CreateCollection() and an extension method Dump(this T, string) which are being used to represent abstractly that a document collection can be obtained from whatever document store, and any document may be dumped (like LINQPad provides):</p> <pre><code> private void DemostrateMapReduce() { var calls = CreateCollectionCall&lt;Call&gt;(); calls.Count().Dump("Call Count"); const string mapJavascript = @"function(){ var call = this; /* averageCallTime should be fetched, simplified here, averageCallTime is used as the timeOnCall for calls that are in progress */ var averageCallTime = 15.0; var calculateTotalTimeOnCall = function(startTime, endTime) { if ((!endTime) || (!startTime)) { return averageCallTime; } var diffMs = endTime - startTime; return (diffMs / 1000) * 60; }; emit(call.Agent._id, { count: 1, totalTimeOnCall: 1 }); }"; const string reduceJavascript = @"function(key, values) { var result = { count: 0, totalTimeOnCall: 0 }; values.forEach(function(value) { result.count += value.count; result.totalTimeOnCall += value.totalTimeOnCall; }); return result; }"; var mapReduceResult = calls.MapReduce(mapJavascript, reduceJavascript, MapReduceOptions.SetOutput(MapReduceOutput.Inline)); foreach (var item in mapReduceResult.GetInlineResultsAs&lt;AgentCallSummary&gt;()) { item.Dump(); } } </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USpaegun
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COI don't follow what you're looking for here. What does C# have to do with your use of MapReduce?
      singulars
      1. POHow can I utilize MongoDB's Map-Reduce while reducing the impact...staying as data store agnostic as possible?
      1. USWiredPrairie
    2. COThank you. The company's software development practice calls for remaining data store agnostic, ie they favor EntityFramework over direct ADO.NET. When MongoDB was brought in, the company is satisfying the goal by using only the 10-gen driver for C# .Find() and variant methods (such as FindOne()) for queries. I appreciate the desire to be data store agnostic, but to force larger sets than necessary to be shipped from MongoDB to the Application Layer does not make sense to me. I am looking for to reduce the impact of using MongoDB's Map-Reduce as it is exposed by the 10-gen C# driver.
      singulars
      1. POHow can I utilize MongoDB's Map-Reduce while reducing the impact...staying as data store agnostic as possible?
      1. USpaegun
    3. COI don't follow what you mean by "reduce the impact" though? It totally makes sense that you'd avoid sending large amounts of data to a middle-tier for processing if it could be done on the DB server, so it's hard to see what more you could do than what you've already done.
      singulars
      1. POHow can I utilize MongoDB's Map-Reduce while reducing the impact...staying as data store agnostic as possible?
      1. USWiredPrairie
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload