Databases
StackOverflow2013
Postsdbo
POPython quit unexpectedly running parallel processes with scipy.linalg.inv()
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPython quit unexpectedly running parallel processes with scipy.linalg.inv()
    text
    copied!<p>I try to run some tasks with <code>multiprocessing</code>s <code>Pool</code> by Python. All seems to work as it should, but one line lets Python quit unexpectedly, it is the line, which computes the inverse of a matrix. (<code>scipy.linalg.inv()</code>) </p> <p>Apples error log prints</p> <pre><code>Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000108 </code></pre> <p>I am pretty sure, that it has something to do with the inverse, and not with some unknown variables, because the quit happens as well, if I just compute:</p> <pre><code>la.inv(np.eye(n)) </code></pre> <p>I think it could have something to do with the fact, that <code>la.inv()</code> uses multiple threads.</p> <p>Note: If I run the code without parallelization, it works of course. <em>AND</em> it also works, if I just comment the line above.</p> <p><em>BUT</em> if I run some test code, that minimalizes the problem:</p> <pre><code>import numpy as np import scipy.linalg as la from multiprocessing import Pool class ParClass(object): def __init__(self): super(ParClass, self).__init__() def compute(self,x): self.x = la.inv(np.random.randn(1000,1000)) return self def runPar(x): method = ParClassifier() method.compute(x) return method.x if __name__ == '__main__': pool = Pool(processes=4) result = pool.map(runPar, np.arange(1,100)) </code></pre> <p>The error does not occur. So why does a line of code that affects the hole process of my original problem does not affect a test problem? This is quite strange, however did anyone ever came across this before?</p> <p>EDIT: <code>np.linalg.solve()</code> does not do any better</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload