StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>The first thing I'd say is write some sample code <em>manually</em> that tells you what the absolute best case you can expect is - see if your current code is worth fixing.</p> <p>If you are using <code>PropertyInfo.SetValue</code> etc, then absolutely you can make it quicker, even with juts <code>object</code> - <a href="http://www.codeproject.com/KB/cs/HyperPropertyDescriptor.aspx" rel="nofollow noreferrer">HyperDescriptor</a> might be a good start (it is <em>significantly</em> faster than raw reflection, but without making the code any more complicated).</p> <p>For optimal performance, dynamic IL methods are the way to go (precompiled once); in 2.0/3.0, maybe <code>DynamicMethod</code>, but in 3.5 I'd favor <code>Expression</code> (with <code>Compile()</code>). Let me know if you want more detail?</p> <hr> <p>Implementation using <code>Expression</code> and <a href="http://www.codeproject.com/KB/database/CsvReader.aspx" rel="nofollow noreferrer"><code>CsvReader</code></a>, that uses the column headers to provide the mapping (it invents some data along the same lines); it uses <code>IEnumerable<T></code> as the return type to avoid having to buffer the data (since you seem to have quite a lot of it):</p> <pre><code>using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.Linq; using System.Linq.Expressions; using System.Reflection; using LumenWorks.Framework.IO.Csv; class Entity { public string Name { get; set; } public DateTime DateOfBirth { get; set; } public int Id { get; set; } } static class Program { static void Main() { string path = "data.csv"; InventData(path); int count = 0; foreach (Entity obj in Read<Entity>(path)) { count++; } Console.WriteLine(count); } static IEnumerable<T> Read<T>(string path) where T : class, new() { using (TextReader source = File.OpenText(path)) using (CsvReader reader = new CsvReader(source,true,delimiter)) { string[] headers = reader.GetFieldHeaders(); Type type = typeof(T); List<MemberBinding> bindings = new List<MemberBinding>(); ParameterExpression param = Expression.Parameter(typeof(CsvReader), "row"); MethodInfo method = typeof(CsvReader).GetProperty("Item",new [] {typeof(int)}).GetGetMethod(); Expression invariantCulture = Expression.Constant( CultureInfo.InvariantCulture, typeof(IFormatProvider)); for(int i = 0 ; i < headers.Length ; i++) { MemberInfo member = type.GetMember(headers[i]).Single(); Type finalType; switch (member.MemberType) { case MemberTypes.Field: finalType = ((FieldInfo)member).FieldType; break; case MemberTypes.Property: finalType = ((PropertyInfo)member).PropertyType; break; default: throw new NotSupportedException(); } Expression val = Expression.Call( param, method, Expression.Constant(i, typeof(int))); if (finalType != typeof(string)) { val = Expression.Call( finalType, "Parse", null, val, invariantCulture); } bindings.Add(Expression.Bind(member, val)); } Expression body = Expression.MemberInit( Expression.New(type), bindings); Func<CsvReader, T> func = Expression.Lambda<Func<CsvReader, T>>(body, param).Compile(); while (reader.ReadNextRecord()) { yield return func(reader); } } } const char delimiter = '\t'; static void InventData(string path) { Random rand = new Random(123456); using (TextWriter dest = File.CreateText(path)) { dest.WriteLine("Id" + delimiter + "DateOfBirth" + delimiter + "Name"); for (int i = 0; i < 10000; i++) { dest.Write(rand.Next(5000000)); dest.Write(delimiter); dest.Write(new DateTime( rand.Next(1960, 2010), rand.Next(1, 13), rand.Next(1, 28)).ToString(CultureInfo.InvariantCulture)); dest.Write(delimiter); dest.Write("Fred"); dest.WriteLine(); } dest.Close(); } } } </code></pre> <hr> <p>Second version (see comments) that uses <code>TypeConverter</code> rather than <code>Parse</code>:</p> <pre><code>using System; using System.Collections.Generic; using System.ComponentModel; using System.Globalization; using System.IO; using System.Linq; using System.Linq.Expressions; using System.Reflection; using LumenWorks.Framework.IO.Csv; class Entity { public string Name { get; set; } public DateTime DateOfBirth { get; set; } public int Id { get; set; } } static class Program { static void Main() { string path = "data.csv"; InventData(path); int count = 0; foreach (Entity obj in Read<Entity>(path)) { count++; } Console.WriteLine(count); } static IEnumerable<T> Read<T>(string path) where T : class, new() { using (TextReader source = File.OpenText(path)) using (CsvReader reader = new CsvReader(source, true, delimiter)) { string[] headers = reader.GetFieldHeaders(); Type type = typeof(T); List<MemberBinding> bindings = new List<MemberBinding>(); ParameterExpression param = Expression.Parameter(typeof(CsvReader), "row"); MethodInfo method = typeof(CsvReader).GetProperty("Item", new[] { typeof(int) }).GetGetMethod(); var converters = new Dictionary<Type, ConstantExpression>(); for (int i = 0; i < headers.Length; i++) { MemberInfo member = type.GetMember(headers[i]).Single(); Type finalType; switch (member.MemberType) { case MemberTypes.Field: finalType = ((FieldInfo)member).FieldType; break; case MemberTypes.Property: finalType = ((PropertyInfo)member).PropertyType; break; default: throw new NotSupportedException(); } Expression val = Expression.Call( param, method, Expression.Constant(i, typeof(int))); if (finalType != typeof(string)) { ConstantExpression converter; if (!converters.TryGetValue(finalType, out converter)) { converter = Expression.Constant(TypeDescriptor.GetConverter(finalType)); converters.Add(finalType, converter); } val = Expression.Convert(Expression.Call(converter, "ConvertFromInvariantString", null, val), finalType); } bindings.Add(Expression.Bind(member, val)); } Expression body = Expression.MemberInit( Expression.New(type), bindings); Func<CsvReader, T> func = Expression.Lambda<Func<CsvReader, T>>(body, param).Compile(); while (reader.ReadNextRecord()) { yield return func(reader); } } } const char delimiter = '\t'; static void InventData(string path) { Random rand = new Random(123456); using (TextWriter dest = File.CreateText(path)) { dest.WriteLine("Id" + delimiter + "DateOfBirth" + delimiter + "Name"); for (int i = 0; i < 10000; i++) { dest.Write(rand.Next(5000000)); dest.Write(delimiter); dest.Write(new DateTime( rand.Next(1960, 2010), rand.Next(1, 13), rand.Next(1, 28)).ToString(CultureInfo.InvariantCulture)); dest.Write(delimiter); dest.Write("Fred"); dest.WriteLine(); } dest.Close(); } } } </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload