Databases
StackOverflow2013
Postsdbo
POParallel mapM on Repa arrays
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POParallel mapM on Repa arrays
    primarykey
    data
    text
    <p>In my recent <a href="https://github.com/bgamari/mixture-model" rel="noreferrer">work</a> with <code>Gibbs sampling</code>, I've been making great use of the <a href="http://hackage.haskell.org/packages/archive/rvar/0.2.0.1/doc/html/Data-RVar.html" rel="noreferrer"><code>RVar</code></a> which, in my view, provides a near ideal interface to random number generation. Sadly, I've been unable to make use of Repa due to the inability to use monadic actions in maps.</p> <p>While clearly monadic maps can't be parallelized in general, it seems to me that <code>RVar</code> may be at least one example of a monad where effects can be safely parallelized (at least in principle; I'm not terribly familiar with the inner workings of <code>RVar</code>). Namely, I want to write something like the following,</p> <pre><code>drawClass :: Sample -&gt; RVar Class drawClass = ... drawClasses :: Array U DIM1 Sample -&gt; RVar (Array U DIM1 Class) drawClasses samples = A.mapM drawClass samples </code></pre> <p>where <code>A.mapM</code> would look something like,</p> <pre><code>mapM :: ParallelMonad m =&gt; (a -&gt; m b) -&gt; Array r sh a -&gt; m (Array r sh b) </code></pre> <p>While clearly how this would work depends crucially on the implementation of <code>RVar</code> and its underlying <code>RandomSource</code>, in principle one would think that this would involve drawing a new random seed for each thread spawned and proceeding as usual.</p> <p>Intuitively, it seems that this same idea might generalize to some other monads.</p> <p>So, my question is: Could one construct a class <code>ParallelMonad</code> of monads for which effects can be safely parallelized (presumably inhabited by, at the least, <code>RVar</code>)? </p> <p>What might it look like? What other monads might inhabit this class? Have others considered the possibility of how this might work in Repa?</p> <p>Finally, if this notion of parallel monadic actions can't be generalized, does anyone see any nice way to make this work in the specific case of <code>RVar</code> (where it would be very useful)? Giving up <code>RVar</code> for parallelism is a very difficult trade-off.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USParas Joshi
    1. USbgamari
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POParallel mapM on Repa arrays
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POParallel mapM on Repa arrays
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POParallel mapM on Repa arrays
      1. This table or related slice is empty.
      1. VTUpMod
    1. COI guess the sticking point is "drawing a new random seed for each thread spawned" -- how should this step work, and how should the seeds be merged again once all the threads return?
      singulars
      1. POParallel mapM on Repa arrays
      1. USDaniel Wagner
    2. COThe RVar interface would almost certainly need some additions to accommodate spawning a new generator with a given seed. Admittedly, it's unclear how the mechanics of this work and it does seem quite `RandomSource` specific. My naive attempt at drawing a seed would be to do something simple and likely very wrong such as draw a vector of elements (in the case of `mwc-random`) and add 1 to each element to produce a seed for the first worker, add 2 for the second worker, etc. Woefully inadequate if you need cryptographic-quality entropy; hopefully fine if you just need a random walk.
      singulars
      1. POParallel mapM on Repa arrays
      1. USbgamari
    3. COI have been able to do something similar to what you're asking for using `fillChunkedIOP`.
      singulars
      1. POParallel mapM on Repa arrays
      1. USkosmikus
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload