Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p><code>List&lt;T&gt;</code> has a built-in function called <code>GetRange()</code> which was made specifically for what you're trying to do. It's extremely fast and doesn't need Linq, casting, etc...</p> <pre><code>List&lt;string&gt; prodASINs = dc.aboProducts.Select(a =&gt; a.asin).ToList(); for(int i = 0; i &lt; prodASINs.Count; i += 10) { List&lt;string&gt; buffer = prodASINs.GetRange(i, 10); // do something with buffer } </code></pre> <p>That's it. Very simple.</p> <hr> <p>Test results: <code>GetRange</code> vs. <code>Slice</code> vs. <code>Linq</code> with 5000 strings in <code>List&lt;string&gt;</code> As you can clearly see, the Skip/Take approach using Linq is over 383 times slower than <code>Slice&lt;T&gt;()</code> and 4,736 times slower than <code>GetRange()</code></p> <p>==================================================================================</p> <pre><code>GetRange took on average 168 ticks Slice took on average 2073 ticks Linq took on average 795643 ticks </code></pre> <p>Test method used (try it yourself): </p> <pre><code>private static void GetRangeVsSliceVsLinq() { List&lt;string&gt; stringList = new List&lt;string&gt;(); for (int i = 0; i &lt; 5000; i++) { stringList.Add("This is a test string " + i.ToString()); } Stopwatch sw = new Stopwatch(); long m1 = 0, m2 = 0, m3 = 0; for (int x = 0; x &lt; 10; x++) { Console.WriteLine("Iteration {0}", x + 1); Console.WriteLine(); sw.Reset(); sw.Start(); for (int i = 0; i &lt; stringList.Count; i += 10) { List&lt;string&gt; buffer = stringList.GetRange(i, 10); } sw.Stop(); Console.WriteLine("GetRange took {0} msecs", sw.ElapsedMilliseconds); Console.WriteLine("GetRange took {0} ticks", sw.ElapsedTicks); m1 += sw.ElapsedTicks; sw.Reset(); sw.Start(); string[] sliceArray = stringList.ToArray(); for (int i = 0; i &lt; sliceArray.Length; i += 10) { List&lt;string&gt; buffer = sliceArray.Slice(i, 10).ToList(); } sw.Stop(); Console.WriteLine("Slice took {0} msecs", sw.ElapsedMilliseconds); Console.WriteLine("Slice took {0} ticks", sw.ElapsedTicks); m2 += sw.ElapsedTicks; sw.Reset(); sw.Start(); var count = stringList.Count(); for (var i = 0; i &lt; count; i++) { var buffer = stringList.Skip(i * 10).Take(10).ToList(); } sw.Stop(); Console.WriteLine("Skip/Take took {0} msecs", sw.ElapsedMilliseconds); Console.WriteLine("Skip/Take took {0} ticks", sw.ElapsedTicks); m3 += sw.ElapsedTicks; Console.WriteLine(); } Console.WriteLine(); Console.WriteLine("GetRange took on average {0} ticks", m1 / 10); Console.WriteLine("Slice took on average {0} ticks", m2 / 10); Console.WriteLine("Linq took on average {0} ticks", m3 / 10); } </code></pre>
    singulars
    1. This table or related slice is empty.
    1. POSplit results from DB into "chunks" of 10
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USChris Gessler
    1. USChris Gessler
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTDownMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTModeratorReview
    1. CO"It's extremely fast and doesn't need Linq, casting, etc..." - which is great... if you're working with lists. However, considering that this linq may, for example, be working with a DB driver with fast 'skip' and the act of accessing data might be the bottleneck - in which case you example may perform poorly. I note this because I was down-voted without reason at the same time you decided to post.
      singulars
      1. PO
      1. USNPSF3000
    2. CO"which is great... if you're working with lists" - `List<string> prodASIN` is a list, which is what this answer addresses. If the bottleneck is in the database layer, any attempt to retrieve the data will exibit the same performance issues and the only way around it would be to retrieve the data asynchronously, for example with PLINQ. With Parallelism the blocks of data could be retrieved simultaneously and processed immediately as each block is returned (provided that the blocks can be processed out of order) or wait until all blocks are returned, which would still be faster.
      singulars
      1. PO
      1. USChris Gessler
    3. CO"List<string> prodASIN is a list," prodASIN is the *RESULT* of the query. For example: Let us suppose that one element is 100MB in size [all we know is that it contains a string] and stored in a db or similar. Your version would first attempt to load all 1200 results [120GB] first, then split them. Others, such as mine, would request the data in say 1GB blocks. That's not to say your solution is bad, but that it has it's limitations - ones that don't appear in the down-voted answers.
      singulars
      1. PO
      1. USNPSF3000
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload