StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>You mentioned that you found methods that tell you how much memory is allocated</p> <pre><code> GC.GetTotalMemory(false); </code></pre> <p>is one such method (which I presume you have already found).</p> <p>One thing that I would like to point out from the MSDN documentation.</p> <blockquote> <p>Retrieves the number of bytes currently thought to be allocated</p> </blockquote> <p>This is at the very top of the GC.GetTotalMemory method documentation. I would like to point out the word <strong>thought</strong> in the above phrase. Now I know you know how to find the allocated amount as it is mentioned in your question, however I bring this up to illustrate that C# is a managed language. Memory usage and consumption is abstracted away from you and even the GC methods are merely there to give you a vague idea of what is going on in your process. Working with memory levels manually sounds risky and unreliable to me.</p> <p>I would recommend going with your original approach but pulling the batch size way back to a level such that it is highly unlikely that you get an out of memory exception no matter how many columns you are working with. Think hundreds, perhaps a few thousand, rather than tens of thousands. Any performance gains you get by larger batches are likely to be outweighed by the risk of memory problems at these levels even if you try to detect that. The performance tools mentioned in another answer would be a great way to determine what the batch size should be and if it is even a problem.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload