Databases
StackOverflow2013
Postsdbo
POError calling serialize R function
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POError calling serialize R function
    primarykey
    data
    text
    <p>I am loading the following packages into R:</p> <pre><code>library(foreach) library(doParallel) library(iterators) </code></pre> <p>I "parallelize" code for a long time, but lately I am getting INTERMITTENT stops while code is running. The error is:</p> <pre><code>Error in serialize(data, node$con) : error writing to connection </code></pre> <p>My educated guess is that maybe the connection that I open using the commands below, has expired:</p> <pre><code>## Register Cluster ## cores&lt;-8 cl &lt;- makeCluster(cores) registerDoParallel(cl) </code></pre> <p>Looking at makeCluster man page I see that by default the connections expires only after 30 days! I could set options(error=recover) in order to check, on the fly, if the connection is opened or not when the code halts, but I decided to post this general question before.</p> <p><strong>IMPORTANT:</strong></p> <p>1) the error is really intermittent, sometimes I re-run the same code and get no errors. 2) I run everything on the same multi-core machine (Intel/8 cores). So it is not a communation (network) problem among the clusters. 3) I am a heavy user of CPU and GPU parallelization, on my laptop and desktop (64 cores) Unfortunately, it is the first time that I am getting this type of error.</p> <p>Is anybody having the same type of error?</p> <p>As requested I am providing my sessionInfo():</p> <pre><code>&gt; sessionInfo() R version 2.15.3 (2013-03-01) Platform: x86_64-w64-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C [5] LC_TIME=English_United States.1252 attached base packages: [1] parallel stats graphics grDevices utils datasets methods base other attached packages: [1] TTR_0.22-0 xts_0.9-3 doParallel_1.0.1 iterators_1.0.6 foreach_1.4.0 zoo_1.7-9 Revobase_6.2.0 RevoMods_6.2.0 loaded via a namespace (and not attached): [1] codetools_0.2-8 compiler_2.15.3 grid_2.15.3 lattice_0.20-13 tools_2.15.3 </code></pre> <p>@SeteveWeston, below the error in one of the calls (again it is intermittent):</p> <pre><code>starting worker pid=8808 on localhost:10187 at 15:21:52.232 starting worker pid=5492 on localhost:10187 at 15:21:53.624 starting worker pid=8804 on localhost:10187 at 15:21:54.997 starting worker pid=8540 on localhost:10187 at 15:21:56.360 starting worker pid=6308 on localhost:10187 at 15:21:57.721 starting worker pid=8164 on localhost:10187 at 15:21:59.137 starting worker pid=8064 on localhost:10187 at 15:22:00.491 starting worker pid=8528 on localhost:10187 at 15:22:01.855 Error in unserialize(node$con) : ReadItem: unknown type 0, perhaps written by later version of R Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted Error in unserialize(node$con) : error reading from connection Calls: &lt;Anonymous&gt; ... doTryCatch -&gt; recvData -&gt; recvData.SOCKnode -&gt; unserialize Execution halted </code></pre> <p>Adding a bit more information. I set options(error=recover) and it provided the following information:</p> <pre><code>Error in serialize(data, node$con) : error writing to connection Enter a frame number, or 0 to exit 1: #51: parallelize(FUN = "ensemble.prism", arg = list(prism = iis.long, instances = oos.instances), vectorize.arg = c("prism", "instances"), cores = cores, .export 2: parallelize.R#58: foreach.bind(idx = i) %dopar% pFUN(idx) 3: e$fun(obj, substitute(ex), parent.frame(), e$data) 4: clusterCall(cl, workerInit, c.expr, exportenv, obj$packages) 5: sendCall(cl[[i]], fun, list(...)) 6: postNode(con, "EXEC", list(fun = fun, args = args, return = return, tag = tag)) 7: sendData(con, list(type = type, data = value, tag = tag)) 8: sendData.SOCKnode(con, list(type = type, data = value, tag = tag)) 9: serialize(data, node$con) Selection: 9 </code></pre> <p>I tried to check if the connections were still available, and there are:</p> <pre><code>Browse[1]&gt; showConnections() description class mode text isopen can read can write 3 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 4 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 5 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 6 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 7 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 8 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 9 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" 10 "&lt;-www.007guard.com:10187" "sockconn" "a+b" "binary" "opened" "yes" "yes" Browse[1]&gt; </code></pre> <p>Since the connections are open and error 0 means R version (as pointed out by @SteveWeston), I really can;t figure out what is happening here.</p> <p><strong>EDIT 1:</strong></p> <p><strong>MY WORKAROUND TO THE PROBLEM</strong></p> <p>The code is fine in terms of arguments passed to the function. Thus, the answer provided by @MichaelFilosi haven't brought much to the table. In any manner, many thanks for your answer! </p> <p>I couldn't find exactly what was wrong with the call, but, at least, I could workaround the problem.</p> <p>The trick was to break the arguments of function call, for each parallel thread, into smaller blocks.</p> <p>Magically the error disappeared.</p> <p>Let me know if the same worked for you!</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USMSardelich
    1. USMSardelich
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POError calling serialize R function
      1. USTommy Levi
      1. VTFavorite
    2. VO
      singulars
      1. POError calling serialize R function
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POError calling serialize R function
      1. This table or related slice is empty.
      1. VTUpMod
    1. COCan you provide your sessionInfo()
      singulars
      1. POError calling serialize R function
      1. USTommy Levi
    2. COTry using the makeCluster `outfile=''` option so you will see any error messages that may be issued by the workers when this happens. You can specify a filename with outfile if you're using Rgui on Windows which I believe doesn't support `outfile=''`.
      singulars
      1. POError calling serialize R function
      1. USSteve Weston
    3. CO@TommyLevi, I just attached it! thanks for your prompt reply.
      singulars
      1. POError calling serialize R function
      1. USMSardelich
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload