Databases
StackOverflow2013
Postsdbo
PONeed a better way to execute console commands from python and log the results
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PONeed a better way to execute console commands from python and log the results
    text
    copied!<p>I have a python script which needs to execute several command line utilities. The stdout output is sometimes used for further processing. In all cases, I want to log the results and raise an exception if an error is detected. I use the following function to achieve this:</p> <pre><code>def execute(cmd, logsink): logsink.log("executing: %s\n" % cmd) popen_obj = subprocess.Popen(\ cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = popen_obj.communicate() returncode = popen_obj.returncode if (returncode &lt;&gt; 0): logsink.log(" RETURN CODE: %s\n" % str(returncode)) if (len(stdout.strip()) &gt; 0): logsink.log(" STDOUT:\n%s\n" % stdout) if (len(stderr.strip()) &gt; 0): logsink.log(" STDERR:\n%s\n" % stderr) if (returncode &lt;&gt; 0): raise Exception, "execute failed with error output:\n%s" % stderr return stdout </code></pre> <p>"logsink" can be any python object with a log method. I typically use this to forward the logging data to a specific file, or echo it to the console, or both, or something else...</p> <p>This works pretty good, except for three problems where I need more fine-grained control than the communicate() method provides:</p> <ol> <li>stdout and stderr output can be interleaved on the console, but the above function logs them separately. This can complicate the interpretation of the log. How do I log stdout and stderr lines interleaved, in the same order as they were output?</li> <li>The above function will only log the command output once the command has completed. This complicates diagnosis of issues when commands get stuck in an infinite loop or take a very long time for some other reason. How do I get the log in real-time, while the command is still executing?</li> <li>If the logs are large, it can get hard to interpret which command generated which output. Is there a way to prefix each line with something (e.g. the first word of the cmd string followed by :).</li> </ol>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload