Databases
StackOverflow2013
Postsdbo
POStrange behaviour of bash script for running parallel subprocess in bash
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POStrange behaviour of bash script for running parallel subprocess in bash
    text
    copied!<p>This following script is used for running parallel subprocess in bash,which is slightly changed from <a href="https://stackoverflow.com/questions/6593531/running-a-limited-number-of-child-processes-in-parallel-in-bash">Running a limited number of child processes in parallel in bash?</a></p> <pre><code>#!/bin/bash set -o monitor # means: run background processes in a separate processes... N=1000 todo_array=($(seq 0 $((N-1)))) max_jobs=5 trap add_next_job CHLD index=0 function add_next_job { if [[ $index -lt ${#todo_array[@]} ]] then do_job $index &amp; index=$(($index+1)) fi } function do_job { echo $1 start time=$(echo "scale=0;x=$RANDOM % 10;scale=5;x/20+0.05" |bc);sleep $time;echo $time echo $1 done } while [[ $index -lt $max_jobs ]] &amp;&amp; [[ $index -lt ${#todo_array[@]} ]] do add_next_job done wait </code></pre> <p>The job is choosing a random number in 0.05:0.05:5.00 and sleep that much second.</p> <p>For example, with N=10, a sample out put is </p> <pre><code>1 start 4 start 3 start 2 start 0 start .25000 2 done 5 start .30000 3 done 6 start .35000 0 done 7 start .40000 1 done 8 start .40000 4 done 9 start .05000 7 done .20000 5 done .25000 9 done .45000 6 done .50000 8 done </code></pre> <p>which has 30 lines in total.</p> <p>But for big N such as 1000,the result can be strange.One run gives 2996 lines of ouput,with 998 lines with start ,999 with done ,and 999 with float number.644 and 652 is missing in start,644 is missing in done.</p> <p>These test are runned on an Arch Linux with bash 4.2.10(2).Similar results can be produced on debian stable with bash 4.1.5(1).</p> <p>EDIT:I tried parallel in moreutils and GNU parallel for this test.Parallel in moreutils has the same problem.But GNU parallel works perfect.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload