Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>In general you should try to avoid having disk access on multiple threads. The disk is a bottleneck and will block, so might impact performance.</p> <p>If the size of the lines in the file is not an issue, you should probably read the entire file in first, and then process in parallel.</p> <p>If the file is too large to do that or it's not practical, then you could use <a href="http://msdn.microsoft.com/en-us/library/dd267312.aspx" rel="nofollow">BlockingCollection</a> to load it. Use one thread to read the file and populate the BlockingCollection and then Parallel.ForEach to process the items in it. BlockingCollection allows you to <a href="http://www.lovethedot.net/2009/02/parallel-programming-in-net-40-and.html" rel="nofollow">specify the max size of the collection</a>, so it will only read more lines from the file as what's already in the collection is processed and removed.</p> <pre><code> static void Main(string[] args) { string filename = @"c:\vs\temp\test.txt"; int maxEntries = 2; var c = new BlockingCollection&lt;String&gt;(maxEntries); var taskAdding = Task.Factory.StartNew(delegate { var lines = File.ReadLines(filename); foreach (var line in lines) { c.Add(line); // when there are maxEntries items // in the collection, this line // and thread will block until // the processing thread removes // an item } c.CompleteAdding(); // this tells the collection there's // nothing more to be added, so the // enumerator in the other thread can // end }); while (c.Count &lt; 1) { // this is here simply to give the adding thread time to // spin up in this much simplified sample } Parallel.ForEach(c.GetConsumingEnumerable(), i =&gt; { // NOTE: GetConsumingEnumerable() removes items from the // collection as it enumerates over it, this frees up // the space in the collection for the other thread // to write more lines from the file Console.WriteLine(i); }); Console.ReadLine(); } </code></pre> <p>As with some of the others, though, I have to ask the question: Is this something you really need to try optimizing through parallelization, or would a single-threaded solution perform well enough? Multithreading adds a lot of complexity and it's sometimes not worth it.</p> <p>What kind of performance are you seeing that you want to improve upon?</p>
    singulars
    1. This table or related slice is empty.
    1. POreading a csv file with a million rows in parallel in c#
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USRagoczy
    1. USRagoczy
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COCan you show an implementation of the above problem using BlockingCollection? I am a beginner and not an expert.
      singulars
      1. PO
      1. USNext Door Engineer
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload