Databases
StackOverflow2013
Postsdbo
POHow can I use Scala concurrent programming to do computations in parallel?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow can I use Scala concurrent programming to do computations in parallel?
    text
    copied!<p>I am trying to use Scala to find the parameter to a function that yields the largest return value, and I would like to do it in parallel. So for this function:</p> <pre><code>def f(i: Long): Double = { // do something with i and return a double } </code></pre> <p>I want to find the input parameter i over the range (0, x) that gives the maximum value when passed to the function f. This is what I have so far:</p> <pre><code>import scala.concurrent.ops._ def parMap(f: Long =&gt; (Double, Long), xs: List[Int]): Array[(Double, Long)] = { val results = new Array[(Double, Long)](xs.length) replicate(0, xs.length) { i =&gt; results(i) = f(xs(i)) } results } var results = parMap(i =&gt; (f(i), i), List.range(0, i)).max </code></pre> <p>It might work correctly but I get a java.lang.OutOfMemoryError: Java heap space error. For the problem I am working on the entire set of results will be too large to fit in memory, so it needs to discard results that are inferior to the best seen so far. If I make the list range small enough for it to all fit in memory, my results Array (before it calls the max method) looks kind of like this:</p> <pre><code>Array(null, null, (-Infinity,2), (-Infinity,3), null, (-Infinity,5), (-Infinity,6), (-Infinity,7), (-Infinity,8), (-22184.3237904591,9), null, (-22137.315048628963,11)... </code></pre> <p>The -Infinity values are normal for what I am doing but the nulls are not. I get different nulls each time I run it, so that is random. It is like the replicate method 'gives up' on some of the function calls and gives null instead.</p> <p>Note I am using Scala 2.8.1.</p> <p>Also, it seems to me accurate documentation on Scala and parallel computing is hard to come by. I would like to learn more, so I can figure out problems like this one on my own. Can anyone suggest a reliable resource I can learn from?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload