StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>If I'm reading your question correctly, you're not looking to just speed up the parallel processing that creates the chars from the words - you would like your enumerable to produce each one <em>as soon as it's ready</em>. With the implementation you currently have (and the other answers I currently see), the <code>SplitItOut</code> will wait until all of the words have been sent to <code>GetCharacters</code>, and all results returned before producing the first one.</p> <p>In cases like this, I like to think of things as splitting my process into producers and a consumer. Your <em>producer</em> thread(s) will take the available words and call GetCharacters, then dump the results somewhere. The <em>consumer</em> will yield up characters to the caller of <code>SplitItOut</code> as soon as they are ready. Really, the consumer is the caller of <code>SplitItOut</code>.</p> <p>We can make use of the <a href="http://msdn.microsoft.com/en-us/library/dd267312.aspx" rel="nofollow"><code>BlockingCollection</code></a> as both a way to yield up the characters, and as the "somewhere" to put the results. We can use the <a href="http://msdn.microsoft.com/en-us/library/dd381779.aspx" rel="nofollow"><code>ConcurrentBag</code></a> as a place to put the words that have yet to be split:</p> <pre><code>static void Main() { var words = new List<string> { "abcd", "wxyz", "1234"}; foreach (var character in SplitItOut(words)) { Console.WriteLine(character); } } static char[] GetCharacters(string word) { Thread.Sleep(5000); return word.ToCharArray(); } </code></pre> <p>No changes to your <code>main</code> or <code>GetCharacters</code> - since these represent your constraints (can't change caller, can't change expensive operation)</p> <pre><code> public static IEnumerable<char> SplitItOut(IEnumerable<string> words) { var source = new ConcurrentBag<string>(words); var chars = new BlockingCollection<char>(); var tasks = new[] { Task.Factory.StartNew(() => CharProducer(source, chars)), Task.Factory.StartNew(() => CharProducer(source, chars)), //add more, tweak away, or use a factory to create tasks. //measure before you simply add more! }; Task.Factory.ContinueWhenAll(tasks, t => chars.CompleteAdding()); return chars.GetConsumingEnumerable(); } </code></pre> <p>Here, we change the <code>SplitItOut</code> method to do four things:</p> <ol> <li>Initialize a concurrentbag with all of the words we wish to split. (side note: If you want to enumerate over words on demand, you can start a new task to push them in rather than doing it in the constructor)</li> <li>Start up our char "producer" Tasks. You can start a set number, use a factory, whatever. I suggest not going task-crazy before you measure.</li> <li>Signal the <code>BlockingCollection</code> that we are done when all tasks have completed.</li> <li>"Consume" all of the produced chars (we make it easy on ourselves and just return an <code>IEnumerable<char></code> rather than foreach and yield, but you could do it the long way if you wish)</li> </ol> <p>All that's missing is our producer implementation. I've expanded out all the linq shortcuts to make it clear, but it's super simple:</p> <pre><code> private static void CharProducer(ConcurrentBag<string> words, BlockingCollection<char> output) { while(!words.IsEmpty) { string word; if(words.TryTake(out word)) { foreach (var c in GetCharacters(word)) { output.Add(c); } } } } </code></pre> <p>This simply</p> <ol> <li>Takes a word out of the ConcurrentBag (unless it's empty - if it is, task is done!)</li> <li>Calls the expensive method</li> <li>Puts the output in the BlockingCollection</li> </ol>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload