Databases
StackOverflow2013
Postsdbo
POLarge Variations in Identical Python Process Run Times
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POLarge Variations in Identical Python Process Run Times
    primarykey
    data
    text
    <p>I just bought a new, machine to run python scripts for large scale modeling. It has two CPUs with 4 cores each (Xeon, 2.8GhZ). Each core has hyper-threading enabled for 4 logical cpu cores.</p> <p>Now for the problem: When I run identical python processes in 8 separate terminals, <code>top</code> command shows that each process is taking 100% of the cpu. However, the process in terminal 1 is running about 4 times slower than the process in terminal 8. This seems odd to me...</p> <p>I wonder if it has something to do with how the processes are scheduled on the various (logical?) cores? Does anyone have an idea of how I could all of the to run in about the same speed?</p> <p>EDIT (in response to larsmans): Good point. The script is a ginat loop that runs about 10,000 times. Each loop reads in a text file (500 lines) and runs some basic calculations on the quantities read in. While the loop runs, it uses about 0.2% of Memory. There is no writing to disk during the loop. I could understand that the read access could be a limiting factor but I am perplexed about the fact that it would be the first process that would be the slowest if that was the case. I would have expected that it would get slower as I start more processes...</p> <p>I timed the processes a couple of times using the <code>time</code> command in the terminal.</p> <p>EDIT2: I just found out that sometimes a single core is designated to handle all reading and writing - so multiple processes (even if they run on separate cores) will use one single core for all the I/O... This would however only affect one of the cores, not cause all to have various processing speeds...</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USnodapic
    1. USnodapic
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POLarge Variations in Identical Python Process Run Times
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POLarge Variations in Identical Python Process Run Times
      1. This table or related slice is empty.
      1. VTModeratorReview
    3. VO
      singulars
      1. POLarge Variations in Identical Python Process Run Times
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThis is impossible to answer without knowing what your script is doing. For starters, is it doing any disk I/O? Did you perform the timing once or did you repeat it?
      singulars
      1. POLarge Variations in Identical Python Process Run Times
      1. USFred Foo
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload