Databases
StackOverflow2013
Postsdbo
PONeed a Better Way Than Reflection
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PONeed a Better Way Than Reflection
    primarykey
    data
    text
    <p>I'm reading a CSV file and the records are recorded as a string[]. I want to take each record and convert it into a custom object.</p> <pre><code>T GetMyObject&lt;T&gt;(); </code></pre> <p>Currently I'm doing this through reflection which is really slow. I'm testing with a 515 Meg file with several million records. It takes under 10 seconds to parse. It takes under 20 seconds to create the custom objects using manual conversions with <code>Convert.ToSomeType</code> but around 4 minutes to do the conversion to the objects through reflection.</p> <p>What is a good way to handle this automatically?</p> <p>It seems a lot of time is spent in the <code>PropertyInfo.SetValue</code> method. I tried caching the properties <code>MethodInfo</code> setter and using that instead, but it was actually slower.</p> <p>I have also tried converting that into a delegate like the great Jon Skeet suggested here: <a href="https://stackoverflow.com/questions/1027980/improving-performance-reflection-what-alternatives-should-i-consider">Improving performance reflection , what alternatives should I consider</a>, but the problem is I don't know what the property type is ahead of time. I'm able to get the delegate</p> <pre><code>var myObject = Activator.CreateInstance&lt;T&gt;(); foreach( var property in typeof( T ).GetProperties() ) { var d = Delegate.CreateDelegate( typeof( Action&lt;,&gt; ) .MakeGenericType( typeof( T ), property.PropertyType ), property.GetSetMethod() ); } </code></pre> <p>The problem here is I can't cast the delegate into a concrete type like <code>Action&lt;T, int&gt;</code>, because the property type of <code>int</code> isn't known ahead of time.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USJosh Close
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PONeed a Better Way Than Reflection
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PONeed a Better Way Than Reflection
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PONeed a Better Way Than Reflection
      1. USRobert Harvey
      1. VTFavorite
    1. COThose times don't sound bad to me for a 500GB file!
      singulars
      1. PONeed a Better Way Than Reflection
      1. USCocowalla
    2. COAgreed, the benchmark results you stated seem reasonable.
      singulars
      1. PONeed a Better Way Than Reflection
      1. USRobert Harvey
    3. COSorry for not being able to help but 10s to parse a 500GB file is unbelievably fast.
      singulars
      1. PONeed a Better Way Than Reflection
      1. USJonas Elfström
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload