Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>(1) Nested parallelism in OpenMP: <a href="http://docs.oracle.com/cd/E19205-01/819-5270/aewbc/index.html" rel="nofollow noreferrer">http://docs.oracle.com/cd/E19205-01/819-5270/aewbc/index.html</a></p> <p>You need to turn on nested parallelism by setting <code>OMP_NESTED</code> or <code>omp_set_nested</code> because many implementations turn off this feature by default, even some implementations didn't support nested parallelism fully. If turned on, whenever you meet <code>parallel for</code>, OpenMP will create the number of threads as defined in <code>OMP_NUM_THREADS</code>. So, if 2-level parallelism, the total number of threads would be N^2, where N = <code>OMP_NUM_THREADS</code>.</p> <p>Such nested parallelism will cause oversubscription, (i.e., the number of busy threads is greater than the cores), which may degrade the speedup. In an extreme case, where nested parallelism is called recursively, threads could be bloated (e.g., creating 1000s threads), and computer just wastes time for context switching. In such case, you may control the number of threads dynamically by setting <code>omp_set_dynamic</code>.</p> <p>(2) An example of matrix-vector multiplication: the code would look like:</p> <pre><code>// Input: A(N by M), B(M by 1) // Output: C(N by 1) for (int i = 0; i &lt; N; ++i) for (int j = 0; j &lt; M; ++j) C[i] += A[i][j] * B[j]; </code></pre> <p>In general, parallelizing inner loops while outer loops are possible is bad because of forking/joining overhead of threads. (though many OpenMP implementations pre-create threads, it still requires some to dispatch tasks to threads and to call implicit barrier at the end of parallel-for)</p> <p>Your concern is the case of where N &lt; # of CPU. Yes, right, in this case, the speedup would be limited by N, and letting nested parallelism will definitely have benefits. </p> <p>However, then the code would cause oversubscription if N is sufficiently large. I'm just thinking the following solutions:</p> <ul> <li>Changing the loop structure so that only 1-level loop exists. (It looks doable)</li> <li>Specializing the code: if N is small, then do nested parallelism, otherwise don't do that.</li> <li>Nested parallelism with <code>omp_set_dynamic</code>. But, please make it sure how <code>omp_set_dynamic</code> controls the number of threads and the activity of threads. Implementations may vary.</li> </ul>
    singulars
    1. This table or related slice is empty.
    1. POOpenMP: What is the benefit of nesting parallelizations?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USConor Taylor
    1. USminjang
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POOpenMP: What is the benefit of nesting parallelizations?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COThe way to deal with small N without putting the parallelism one level down is just to use collapse; `#pragma omp parallel for collapse`; `for (int i=0; i<N; ++i)`, etc. That notionally merges the i and j loops. As to the other issue, "However, then the code would cause oversubscription if N is sufficiently large." -- no it wouldn't. If you're putting the parallel for after the i, the main thread will do the i loop, and every i iteration you'll do a fork, divide up the j-work, and join. (Although as you say, most OpenMP implementations will use a common thread pool now.)
      singulars
      1. PO
      1. USJonathan Dursi
    2. COMaybe some confusion. I'm talking nested parallelism: Having nested parallel loop will mostly cause oversubscription. If N is equal or larger than the number of cores (say n), then n threads will be created at the outer for-i loop. Then, each thread will fork another n threads when the thread meets the for-j loop. So, n*n threads are working on n cores. You can easily check it by using system utilities.
      singulars
      1. PO
      1. USminjang
    3. COOk, fair enough, and that is what he asked. But of course one wouldn't do that; that's exactly what collapse is for -- parallelize over both loops without incurring the overhead.
      singulars
      1. PO
      1. USJonathan Dursi
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload