Databases
StackOverflow2013
Postsdbo
POMulti-Producer Single-Consumer Lazy Task Execution
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMulti-Producer Single-Consumer Lazy Task Execution
    primarykey
    data
    text
    <p>I am trying to model a system where there are multiple threads producing data, and a single thread consuming the data. The trick is that I don't want a dedicated thread to consume the data because all of the threads live in a pool. Instead, I want one of the producers to empty the queue when there is work, and yield if another producer is already clearing the queue.</p> <p>The basic idea is that there is a queue of work, and a lock around the processing. Each producer pushes its payload onto the queue, and then <strong>attempts</strong> to enter the lock. The attempt is non-blocking and returns either true (the lock was acquired), or false (the lock is held by someone else).</p> <p>If the lock is acquired, then that thread then processes all of the data in the queue until it is empty (including any new payloads introduced by other producers during processing). Once all of the work has been processed, the thread releases the lock and quits out.</p> <p>The following is C++ code for the algorithm:</p> <pre><code>void Process(ITask *task) { // queue is a thread safe implementation of a regular queue queue.push(task); // crit_sec is some handle to a critical section like object // try_scoped_lock uses RAII to attempt to acquire the lock in the constructor // if the lock was acquired, it will release the lock in the // destructor try_scoped_lock lock(crit_sec); // See if this thread won the lottery. Prize is doing all of the dishes if (!lock.Acquired()) return; // This thread got the lock, so it needs to do the work ITask *currTask; while (queue.try_pop(currTask)) { ... execute task ... } } </code></pre> <p>In general this code works fine, and I have never actually witnessed the behavior I am about to describe below, but that implementation makes me feel uneasy. It stands to reason that a race condition is introduced between when the thread exits the while loop and when it releases the critical section.</p> <p>The whole algorithm relies on the assumption that if the lock is being held, then a thread is servicing the queue.</p> <p>I am essentially looking for enlightenment on 2 questions:</p> <ol> <li>Am I correct that there is a race condition as described (bonus for other races)</li> <li>Is there a standard pattern for implementing this mechanism that is performant and doesn't introduce race conditions?</li> </ol>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USMranz
    1. USMranz
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMulti-Producer Single-Consumer Lazy Task Execution
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POMulti-Producer Single-Consumer Lazy Task Execution
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload