Databases
StackOverflow2013
Postsdbo
POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
    primarykey
    data
    text
    <p>It has been said that Scala For comprehensions are actually quite slow. The reason I was given was that due to a Java limitation, for comprehensions (such as "reduce", used below) need to generate a temporary object with each iteration in order to call the function passed in.</p> <p>IS...THIS...TRUE? The tests below seem to bear this out, but I do not completely understand why this is the case.</p> <p>This might make sense for "lambdas" or anonymous functions, but not for non-anonymous functions.</p> <p>In my test, I ran for loops against list.reduce (see code below), and found them to be over twice as fast, even when each iteration called the exact same function that was passed to reduce!</p> <p>I find this amazingly counter-intuitive (once would think that the Scala library would've been carefully created to be as optimal as possible).</p> <p>In a test I put together, I ran the same loop (sum up the numbers 1 to a million, irrespective of overflow) five different ways:</p> <ol> <li>for loop over the array of values</li> <li>for loop, but calling a function instead of inline arithmetic</li> <li>for loop, creating an object that contains an addition function</li> <li>list.reduce, passing i an anonymous function</li> <li>list.reduce, passing in an object member function</li> </ol> <p>Results were as follows: test: min/max/average (milliseconds)</p> <pre><code>1. 27/157/64.78 2. 27/192/65.77 &lt;--- note the similarity between tests 1,2 and 4,5 3. 139/313/202.58 4. 63/342/150.18 5. 63/341/149.99 </code></pre> <p>As can be seen, the "for comprehension" versions are on the order of "for with new for each instance", implying that a "new" may in fact be performed for both the anonymous and non-anonymous function versions.</p> <p>Methodology: code below (test call removed) was compiled into a single .jar file to ensure all versions ran the same library code. Each test in each iteration was called in a new JVM (i.e. scala -cp ... for each test) in order to remove heap size issues.</p> <pre><code>class t(val i: Int) { def summit(j: Int) = j + i } object bar { val biglist:List[Int] = (1 to 1000000).toList def summit(i: Int, j:Int) = i+j // Simple for loop def forloop: Int = { var result: Int = 0 for(i &lt;- biglist) { result += i } result } // For loop with a function instead of inline math def forloop2: Int = { var result: Int = 0 for(i &lt;- biglist) { result = summit(result,i) } result } // for loop with a generated object PER iteration def forloop3: Int = { var result: Int = 0 for(i &lt;- biglist) { val t = new t(result) result = t.summit(i) } result } // list.reduce with an anonymous function passed in def anonymousfunc: Int = { biglist.reduce((i,j) =&gt; {i+j}) } // list.reduce with a named function def realfunc: Int = { biglist.reduce(summit) } // test calling code excised for brevity. One example given: args(0) match { case "1" =&gt; { val start = System.currentTimeMillis() forloop val end = System.currentTimeMillis() println("for="+(end - start)) } ... } </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USMark Gerolimatos
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
      1. This table or related slice is empty.
      1. VTUpMod
    1. CO`.reduce` has nothing to do with "for comprehensions"
      singulars
      1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
      1. USNikita Volkov
    2. COAs a side note, there is scala compiler plugin that aims to remove this overhead for common cases: https://code.google.com/p/scalacl/wiki/ScalaCLPlugin. I have not tried it myself though.
      singulars
      1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
      1. USRégis Jean-Gilles
    3. COIn fact, your first 3 tests are using for-comprehensions, and you're comparing timing of those with `reduce`..
      singulars
      1. POWhy are Scala "for loop comprehensions" so very slow compared to FOR loops?
      1. USRichard Sitze
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload