StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Dirk's comments are spot on w.r.t multicore/foreach/doMC. </p> <p>If you are doing thousands of simulations you may want to consider Amazon's Elastic Map Reduce (EMR) service. When I wanted to scale my simulations in R I started with huge EC2 instances and the multicore package (just like you!). It went well but I ran up a hell of an EC2 bill. I didn't really need all that RAM yet I was paying for it. And my jobs would finish at 3 AM then I would not get into the office until 8 AM so I paid for 5 hours I didn't need. </p> <p>Then I discovered that I could use the EMR service to fire up 50 cheap small Hadoop instances, run my simulations, and then have them automatically shut down! I've totally abandoned running my sims on EC2 and now use EMR almost exclusively. This worked so well that my firm is beginning to test ways to migrate more of our periodic simulation activity to EMR. </p> <p>Here's a <a href="http://www.cerebralmastication.com/2010/02/using-the-r-multicore-package-in-linux-with-wild-and-passionate-abandon/" rel="nofollow noreferrer">blog post</a> I wrote when I first started using multicore on EC2. Then when I discovered I could do this with Amazon EMR I wrote a <a href="http://www.cerebralmastication.com/2010/02/you-can-hadoop-it-its-elastic-boogie-woogie-woog-ie/" rel="nofollow noreferrer">follow up post</a>. </p> <p><strong>EDIT:</strong> since this post I've been working on a package for making it easier to use EMR with R for parallel apply functions. I've named the project <a href="http://code.google.com/p/segue/" rel="nofollow noreferrer">Segue and it's on Google Code</a>. </p> <p><strong>Further Update:</strong> I've since deprecated Segue because there are much better and more mature offerings for accessing Amazon's services from R. </p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload