Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>(note: this might seem an over-complicated solution, but I'm assuming that the source data is large (lots of rows), so loading it into a <code>List&lt;T&gt;</code> might not be feasible)</p> <p>The file reading would be done quite well with an iterator block... if the data is large, you only want to handle one row at a time, not a 2D array.</p> <p>This actually looks like a good fit for <a href="http://www.pobox.com/~skeet/csharp/miscutil/" rel="nofollow noreferrer">MiscUtil</a>'s PushLINQ approach, which can perform multiple aggregates <em>at the same time</em> on a stream of data, without buffering...</p> <p>An example is below...</p> <p><strong>why is this useful?</strong></p> <p>Because it allows you to write multiple queries on a data source using standard LINQ syntax, but only read it once.</p> <p><strong>Example</strong></p> <pre><code>using System; using System.Collections.Generic; using System.IO; using System.Linq; using MiscUtil.Linq; using MiscUtil.Linq.Extensions; static class Program { static void Main() { // prepare a query that is capable of parsing // the input file into the expected format string path = "foo.txt"; var qry = from line in ReadLines(path) let arr = line.Split(',') select new { Name = arr[0].Trim(), Male = arr[1].Trim() == "male", Birth = int.Parse(arr[2].Trim()), M1 = int.Parse(arr[3].Trim()) // etc }; // get a "data producer" to start the query process var producer = CreateProducer(qry); // prepare the overall average var avg = producer.Average(row =&gt; row.M1); // prepare the gender averages var avgMale = producer.Where(row =&gt; row.Male) .Average(row =&gt; row.M1); var avgFemale = producer.Where(row =&gt; !row.Male) .Average(row =&gt; row.M1); // run the query; until now *nothing has happened* - we haven't // even opened the file producer.ProduceAndEnd(qry); // show the results Console.WriteLine(avg.Value); Console.WriteLine(avgMale.Value); Console.WriteLine(avgFemale.Value); } // helper method to get a DataProducer&lt;T&gt; from an IEnumerable&lt;T&gt;, for // use with the anonymous type static DataProducer&lt;T&gt; CreateProducer&lt;T&gt;(IEnumerable&lt;T&gt; data) { return new DataProducer&lt;T&gt;(); } // this is just a lazy line-by-line file reader (iterator block) static IEnumerable&lt;string&gt; ReadLines(string path) { using (var reader = File.OpenText(path)) { string line; while ((line = reader.ReadLine()) != null) { yield return line; } } } } </code></pre>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload