Databases
StackOverflow2013
Postsdbo
POHow can I print and display subprocess stdout and stderr output without distortion?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow can I print and display subprocess stdout and stderr output without distortion?
    primarykey
    data
    text
    <p>Maybe there's someone out in the ether that can help me with this one. (I have seen a number of similar questions to this on SO, but none deal with both standard out and standard error or deal with a situation quite like mine, hence this new question.)</p> <p>I have a python function that opens a subprocess, waits for it to complete, then outputs the return code, as well as the contents of the standard out and standard error pipes. While the process is running, I'd like to also display the output of both pipes as they are populated. My first attempt has resulted in something like this:</p> <pre><code>process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdout = str() stderr = str() returnCode = None while True: # collect return code and pipe info stdoutPiece = process.stdout.read() stdout = stdout + stdoutPiece stderrPiece = process.stderr.read() stderr = stderr + stderrPiece returnCode = process.poll() # check for the end of pipes and return code if stdoutPiece == '' and stderrPiece == '' and returnCode != None: return returnCode, stdout, stderr if stdoutPiece != '': print(stdoutPiece) if stderrPiece != '': print(stderrPiece) </code></pre> <p>There's a couple problems with this though. Because <code>read()</code> reads until an <code>EOF</code>, the first line of the <code>while</code> loop will not return until the subprocess closes the pipe.</p> <p>I could replace the <code>read()</code> in favor of <code>read(int)</code> but the printed output is distorted, cut off at the end of the read characters. I could <code>readline()</code> as a replacement, but the printed output is distorted with alternating lines of output and errors when there are many of both that occur at the same time.</p> <p>Perhaps there's a <code>read-until-end-of-buffer()</code> variant that I'm not aware of? Or maybe it can be implemented?</p> <p>Maybe it's best to implement a <code>sys.stdout</code> wrapper as suggested in this <a href="https://stackoverflow.com/questions/107705/python-output-buffering/107717#107717">answer to another post</a>? I would only want to use the wrapper in this function, however.</p> <p>Any other ideas from the community?</p> <p>I appreciate the help! :)</p> <p><em>EDIT</em>: The solution really should be cross-platform, but if you have ideas that aren't, please share them away to keep the brainstorming going.</p> <hr> <p>For another one of my python subprocess head scratchers, take a look at another of my questions on <a href="https://stackoverflow.com/q/7560708/608905">accounting for subprocess overhead in timing</a>.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USperden
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow can I print and display subprocess stdout and stderr output without distortion?
      1. USunutbu
      1. VTFavorite
    2. VO
      singulars
      1. POHow can I print and display subprocess stdout and stderr output without distortion?
      1. USNathan
      1. VTFavorite
    3. VO
      singulars
      1. POHow can I print and display subprocess stdout and stderr output without distortion?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COYou might want to look at something like pexpect.
      singulars
      1. POHow can I print and display subprocess stdout and stderr output without distortion?
      1. USThomas K
    2. COwhy not just create a StringIO and pass the same instance to the subprocess's stdout and stderr?
      singulars
      1. POHow can I print and display subprocess stdout and stderr output without distortion?
      1. USNathan Ernst
    3. CO@NathanErnst: Because that wouldn't work. `stdout` and `stderr` must be real, OS level file descriptors.
      singulars
      1. POHow can I print and display subprocess stdout and stderr output without distortion?
      1. USSven Marnach
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload