Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I think you may be examining the wrong problem. Certainly as Aaron says if you try to be both a producer to the beginning of a pipeline, and a consumer of the end of the pipeline, it is easy to get into a deadlock situation. This is the problem that communicate() solves.</p> <p>communicate() isn't exactly correct for you since stdin and stdout are on different subprocess objects; but if you take a look at the implementation in subprocess.py you'll see that it does exactly what Aaron suggested.</p> <p>Once you see that communicate both reads and writes, you'll see that in your second try communicate() competes with p2 for the output of p1:</p> <pre><code>p1 = Popen(["grep", "-v", "not"], stdin=PIPE, stdout=PIPE) p2 = Popen(["cut", "-c", "1-10"], stdin=p1.stdout, stdout=PIPE) # ... p1.communicate('data\n') # reads from p1.stdout, as does p2 </code></pre> <p>I am running on win32, which definitely has different i/o and buffering characteristics, but this works for me:</p> <pre><code>p1 = Popen(["grep", "-v", "not"], stdin=PIPE, stdout=PIPE) p2 = Popen(["cut", "-c", "1-10"], stdin=p1.stdout, stdout=PIPE) t = threading.Thread(target=get_output, args=(p2,)) t.start() p1.stdin.write('hello world\n' * 100000) p1.stdin.close() t.join() </code></pre> <p>I tuned the input size to produce a deadlock when using a naive unthreaded p2.read()</p> <p>You might also try buffering into a file, eg</p> <pre><code>fd, _ = tempfile.mkstemp() os.write(fd, 'hello world\r\n' * 100000) os.lseek(fd, 0, os.SEEK_SET) p1 = Popen(["grep", "-v", "not"], stdin=fd, stdout=PIPE) p2 = Popen(["cut", "-c", "1-10"], stdin=p1.stdout, stdout=PIPE) print p2.stdout.read() </code></pre> <p>That also works for me without deadlocks.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload