StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POOptimizing Jinja2 Environment creation
text
Body
copied!<p>My application is running on Google App Engine and most of requests constantly gets yellow flag due to high CPU usage. Using profiler I tracked the issue down to the routine of creating <code>jinja2.Environment</code> instance.</p> <p>I'm creating the instance at module level:</p> <pre><code>from jinja2 import Environment, FileSystemLoader jinja_env = Environment(loader=FileSystemLoader(TEMPLATE_DIRS)) </code></pre> <p>Due to the Google AppEngine operation mode (CGI), this code can be run upon each and every request (their module import cache seems to cache modules for seconds rather than for minutes).</p> <p>I was thinking about storing the environment instance in memcache, but it seems to be not picklable. <code>FileSystemLoader</code> instance seems to be picklable and can be cached, but I did not observe any substantial improvement in CPU usage with this approach.</p> <p>Anybody can suggest a way to decrease the overhead of creating <code>jinja2.Environment</code> instance?</p> <p><strong>Edit</strong>: below is (relevant) part of profiler output.</p> <pre><code>222172 function calls (215262 primitive calls) in 8.695 CPU seconds ncalls tottime percall cumtime percall filename:lineno(function) 33 1.073 0.033 1.083 0.033 {google3.apphosting.runtime._apphosting_runtime___python__apiproxy.Wait} 438/111 0.944 0.002 2.009 0.018 /base/python_dist/lib/python2.5/sre_parse.py:385(_parse) 4218 0.655 0.000 1.002 0.000 /base/python_dist/lib/python2.5/pickle.py:1166(load_long_binput) 1 0.611 0.611 0.679 0.679 /base/data/home/apps/with-the-flow/1.331879498764931274/jinja2/environment.py:10() </code></pre> <p>One call, but as far I can see (and this is consistent across all my GAE-based apps), the most expensive in the whole request processing cycle.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload