StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<h1>Edit:</h1> <p>This answer has been implemented in <a href="https://github.com/campos-ddc/cprofile_graph" rel="nofollow noreferrer">https://github.com/campos-ddc/cprofile_graph</a></p> <h1>Profiling with cProfile</h1> <p>Here's a post I wrote some time ago on profiling with cProfile with some graphical aid.</p> <p>cProfile is one of the most used python profilers out there, and although very powerful, the standard text output is somewhat lackluster. Here I'll show you how to use cProfile on your application in an easier way.</p> <p>There are two common ways to use cProfile, you can use it as a command in prompt to profile a given module, or you can use it inside your code, to profile specific snippets of code.</p> <h2>Profiling a module</h2> <p>To use cProfile to profile an entire module, simply use the following command in your prompt:</p> <pre><code>python -m cProfile -o output_filename.pstats path/to/script arg1 arg2 </code></pre> <p>This will run your module with the given arguments (they are optional) and dump the output in output_filename.pstats.</p> <p>There are <a href="http://docs.python.org/library/profile.html" rel="nofollow noreferrer">lots of ways</a> to read the data on that output file, but for the purpose of this post, let's not worry about those and focus on getting that graphical visualization.</p> <h2>Profiling from inside</h2> <p>Sometimes you don't want to profile an entire module, just a few lines of it.</p> <p>To do so, you are gonna have to add some code to your module.</p> <p>First of all:</p> <pre><code>import cProfile </code></pre> <p>And then, you can replace any segment of code with the following:</p> <pre><code>cProfile.runctx('Your code here', globals(), locals(), 'output_file') </code></pre> <p>For example, here is a test before and after profiling:</p> <pre><code>import unittest class Test(unittest.TestCase): def testSomething(self): self.DoSomethingIDontCareAbout() param = 'whatever' self.RunFunctionIThinkIsSlow(param) self.AssertSomeStuff() # This is after all, a test </code></pre> <p>After:</p> <pre><code>import unittest import cProfile class Test(unittest.TestCase): def testSomething(self): self.DoSomethingIDontCareAbout() param = 'whatever' cProfile.runctx( 'self.RunFunctionIThinkIsSlow(param)', globals(), locals(), 'myProfilingFile.pstats' ) self.AssertSomeStuff() # This is after all, a test </code></pre> <h2>Converting a pstats file to a graph</h2> <p>To convert your profiling file to a graph, you will need a couple of things:</p> <ul> <li><a href="http://code.google.com/p/jrfonseca/wiki/Gprof2Dot" rel="nofollow noreferrer">gprof2dot</a>: This module will convert your output into a <strong>dot</strong> file, a standard file format for graph descriptions.</li> <li><a href="http://www.graphviz.org/Download.php" rel="nofollow noreferrer">GraphViz</a>: It turns your <strong>dot</strong> file into an image.</li> </ul> <p>After you have downloaded <strong>gprof2dot</strong> and installed <strong>GraphViz</strong>, run this command in your prompt:</p> <pre><code>python gprof2dot -f pstats myProfileFile | dot -Tpng -o image_output.png </code></pre> <p>You might have to use a complete path for <strong>gprof2dot</strong> and/or <strong>dot</strong>, or you could add them to your PATH env variable.</p> <p>After all of this, you should have an image that looks kinda like this:</p> <p><img src="https://i.stack.imgur.com/JKcPb.png" alt="results example"></p> <ul> <li><p>Hotter colors (red, orange, yellow) indicate functions that take up more of the total runtime than colder colors (green, blue)</p></li> <li><p>On each node, you can see what percentage of the total runtime that function used and how many times it was called.</p></li> <li><p>Arrows between nodes indicate which function called other functions, and such arrows also have a caption indicating what percentage of the runtime came through there.</p></li> </ul> <p>Note: percentages won't always add up to 100%, especially on code sections that reference C++ code, which won't be profiled. cProfile also won't be able to determine what's called from inside an "eval" statement, so you might see some jumps in your graph.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload