Databases
StackOverflow2013
Postsdbo
POCelery: Callback after task hierarchy
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POCelery: Callback after task hierarchy
    text
    copied!<p>I'm using Celery from a webapp to start a task hierarchy.</p> <h2>Tasks</h2> <p>I'm using the following tasks:</p> <ul> <li><code>task_a</code></li> <li><code>task_b</code></li> <li><code>task_c</code></li> <li><code>notify_user</code></li> </ul> <p>A Django view starts several <code>task_a</code> instances. Each of them does some processing and then starts several <code>task_b</code> instances. And each of those does some processing and then starts several <code>task_c</code> instances.</p> <p>To visualize:</p> <p><img src="https://i.stack.imgur.com/Kk7bc.png" alt="Tree"></p> <h2>Goals</h2> <p><strong>My goal is to execute all the tasks, and to run a callback function as soon as the entire hierarchy has finished. Additionally, I want to be able to pass data from the lowest tasks to the top level.</strong></p> <ol> <li>The view should just "kick off" the tasks and then return.</li> <li>Each subtask depends on the parent task. The parent task does not depend directly on the child task. After a parent task has started all the child tasks, it can be stopped.</li> <li>Everything can be parallelized, as long as the parent task runs before the child task is started.</li> <li>After <strong>all</strong> the tasks have finished, the <code>notify_user</code> callback function should be called.</li> <li>The <code>notify_user</code> callback function needs access to data from the <code>task_c</code>s.</li> </ol> <p>All the tasks should be non-blocking, so <code>task_b</code> should not wait for all the <code>task_c</code> subtasks to finish.</p> <p>What would be the right way to achieve the goal stated above?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload