Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I don't think you need queue unless you intend to get data out of the applications (Which if you do want data, I think it may be easier to add it to a database anyway)</p> <p>but try this on for size:</p> <p>put the contents of your create_graphs.py script all into a function called "create_graphs"</p> <pre><code>import threading from create_graphs import create_graphs num_processes = 64 my_list = [ 'XYZ', 'ABC', 'NYU' ] threads = [] # run until all the threads are done, and there is no data left while threads or my_list: # if we aren't using all the processors AND there is still data left to # compute, then spawn another thread if (len(threads) &lt; num_processes) and my_list: t = threading.Thread(target=create_graphs, args=[ my_list.pop() ]) t.setDaemon(True) t.start() threads.append(t) # in the case that we have the maximum number of threads check if any of them # are done. (also do this when we run out of data, until all the threads are done) else: for thread in threads: if not thread.isAlive(): threads.remove(thread) </code></pre> <p>I know that this will result in 1 less threads than processors, which is probably good, it leaves a processor to manage the threads, disk i/o, and other things happening on the computer. If you decide you want to use the last core just add one to it</p> <p><strong>edit</strong>: I think I may have misinterpreted the purpose of my_list. You do not need <code>my_list</code> to keep track of the threads at all (as they're all referenced by the items in the <code>threads</code> list). But this is a fine way of feeding the processes input - or even better: use a generator function ;)</p> <h2>The purpose of <code>my_list</code> and <code>threads</code></h2> <p><code>my_list</code> holds the data that you need to process in your function<br> <code>threads</code> is just a list of the currently running threads</p> <p>the while loop does two things, start new threads to process the data, and check if any threads are done running.</p> <p>So as long as you have either (a) more data to process, or (b) threads that aren't finished running.... you want to program to continue running. <strong>Once both lists are empty they will evaluate to <code>False</code> and the while loop will exit</strong></p>
    singulars
    1. This table or related slice is empty.
    1. POHow to spawn parallel child processes on a multi-processor system?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJiaaro
    1. USJiaaro
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COHi Jim. I have not seen this while syntax before. You have two lists in the while clause - threads and my_list. What is the logic behind this? Everywhere I have seen Python while loops they have a definitive less than or some other way to control the loop. That appears to be missing here - or am I just asking a dumb newbie question? Thanks.
      singulars
      1. PO
      1. UStatlar
    2. COedited answer to answer this question
      singulars
      1. PO
      1. USJiaaro
    3. COHi Jim - thanks for the clarification. In my code when I set up the which loops like this just makes my script return nothing (i.e the which loop never starts). If I change the while to a for loop it works: before: while threads and my_list now: for list_item in my_list I wonder what I am doing wrong. I am going to post the full working script I have below - maybe you could take a look? Thanks so much for all the help.
      singulars
      1. PO
      1. UStatlar
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload