Databases
StackOverflow2013
Postsdbo
POHow to replicate tee behavior in Python when using subprocess?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to replicate tee behavior in Python when using subprocess?
    primarykey
    data
    text
    <p>I'm looking for a Python solution that will allow me to save the output of a command in a file without hiding it from the console.</p> <p>FYI: I'm asking about <a href="https://en.wikipedia.org/wiki/Tee_%28command%29" rel="noreferrer">tee</a> (as the Unix command line utility) and not the function with the same name from Python intertools module.</p> <h3>Details</h3> <ul> <li>Python solution (not calling <code>tee</code>, it is not available under Windows)</li> <li>I do not need to provide any input to stdin for called process</li> <li>I have no control over the called program. All I know is that it will output something to stdout and stderr and return with an exit code.</li> <li>To work when calling external programs (subprocess)</li> <li>To work for both <code>stderr</code> and <code>stdout</code></li> <li>Being able to differentiate between stdout and stderr because I may want to display only one of the to the console or I could try to output stderr using a different color - this means that <code>stderr = subprocess.STDOUT</code> will not work.</li> <li>Live output (progressive) - the process can run for a long time, and I'm not able to wait for it to finish.</li> <li>Python 3 compatible code (important)</li> </ul> <h3>References</h3> <p>Here are some incomplete solutions I found so far:</p> <ul> <li><a href="http://devlishgenius.blogspot.com/2008/10/logging-in-real-time-in-python.html" rel="noreferrer">http://devlishgenius.blogspot.com/2008/10/logging-in-real-time-in-python.html</a> (mkfifo works only on Unix)</li> <li><a href="http://blog.kagesenshi.org/2008/02/teeing-python-subprocesspopen-output.html" rel="noreferrer">http://blog.kagesenshi.org/2008/02/teeing-python-subprocesspopen-output.html</a> (doesn't work at all)</li> </ul> <p><a href="http://blog.i18n.ro/wp-content/uploads/2010/06/Drawing_tee_py.png" rel="noreferrer">Diagram http://blog.i18n.ro/wp-content/uploads/2010/06/Drawing_tee_py.png</a></p> <h3>Current code (second try)</h3> <pre><code>#!/usr/bin/python from __future__ import print_function import sys, os, time, subprocess, io, threading cmd = "python -E test_output.py" from threading import Thread class StreamThread ( Thread ): def __init__(self, buffer): Thread.__init__(self) self.buffer = buffer def run ( self ): while 1: line = self.buffer.readline() print(line,end="") sys.stdout.flush() if line == '': break proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdoutThread = StreamThread(io.TextIOWrapper(proc.stdout)) stderrThread = StreamThread(io.TextIOWrapper(proc.stderr)) stdoutThread.start() stderrThread.start() proc.communicate() stdoutThread.join() stderrThread.join() print("--done--") #### test_output.py #### #!/usr/bin/python from __future__ import print_function import sys, os, time for i in range(0, 10): if i%2: print("stderr %s" % i, file=sys.stderr) else: print("stdout %s" % i, file=sys.stdout) time.sleep(0.1) </code></pre> Real output <pre><code>stderr 1 stdout 0 stderr 3 stdout 2 stderr 5 stdout 4 stderr 7 stdout 6 stderr 9 stdout 8 --done-- </code></pre> <p>Expected output was to have the lines ordered. Remark, modifying the Popen to use only one PIPE is not allowed because in the real life I will want to do different things with stderr and stdout.</p> <p>Also even in the second case I was not able to obtain real-time like out, in fact all the results were received when the process finished. By default, Popen should use no buffers (bufsize=0).</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USPeter Mortensen
    1. USsorin
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow to replicate tee behavior in Python when using subprocess?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POHow to replicate tee behavior in Python when using subprocess?
      1. USunutbu
      1. VTFavorite
    3. VO
      singulars
      1. POHow to replicate tee behavior in Python when using subprocess?
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload