StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>If you end up with several nested foreach loops, I'd rethink my approach. Using parallel versions of <code>tapply</code> can solve a lot of that hassle. In general, you shouldn't use nested parallelization, as that doesn't bring you anything. Parallelize the outer loop, and forget about the inner loop. </p> <p>Reason is simple : if you have 3 connections in your cluster, the outer dopar loop will use all three. The inner dopar loop will not be able to use any extra connections, as there are none available. So you don't gain a thing. Hence, the mock-up you give doesn't make sense at all from a programming point of view.</p> <p>Your second question is answered pretty easily by the function <code>getDoParRegistered()</code> which returns TRUE when a backend is registered, and FALSE otherwise. Pay attention though :</p> <ul> <li>it also returns TRUE if a sequential backend is registered (i.e. after calling registerDoSEQ). </li> <li>It will return TRUE as well after a cluster has been stopped, but in that case %dopar% will return an error.</li> </ul> <p>eg :</p> <pre><code>require(foreach) require(doSNOW) cl <- makeCluster(rep("localhost",2),type="SOCK") getDoParRegistered() [1] FALSE registerDoSNOW(cl) getDoParRegistered() [1] TRUE stopCluster(cl) getDoParRegistered() [1] TRUE </code></pre> <p>But now running this code :</p> <pre><code>a <- matrix(1:16, 4, 4) b <- t(a) foreach(b=iter(b, by='col'), .combine=cbind) %dopar% (a %*% b) </code></pre> <p>will return in an error :</p> <pre><code>Error in summary.connection(connection) : invalid connection </code></pre> <p>You could build an extra check. A (hideously ugly) hack you can use to check that the connection registered by <code>doSNOW</code> is valid, can be :</p> <pre><code>isvalid <- function(){ if (getDoParRegistered() ){ X <- foreach:::.foreachGlobals$objs[[1]]$data x <- try(capture.output(print(X)),silent=TRUE) if(is(x,"try-error")) FALSE else TRUE } else { FALSE } } </code></pre> <p>Which you could use as</p> <pre><code>if(!isvalid()) registerDoSEQ() </code></pre> <p>This will register the sequential backend if getDoParRegistered() returns TRUE but there is no valid cluster connection any longer. But again, this is a hack, and I have no idea if it works with other backends or even other types of cluster types (I use sockets mostly).</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload