Databases
StackOverflow2013
Postsdbo
POMemory leak in Python Twisted: where is it?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMemory leak in Python Twisted: where is it?
    primarykey
    data
    text
    <p>I have a Twisted server under load. When the server is under load, memory usage increases, and it is never reclaimed (even when there are no more clients). Next time it goes into high load, memory usage increases again. Here's a snapshot of the situation at that point:</p> <ul> <li>RSS memory is 400 MB (should be 200MB with usual max number of clients).</li> <li><strong>gc.garbage</strong> is empty, so there are no uncollectable objects.</li> <li>Using <strong>objgraph.py</strong> shows no obvious candidates for leaks (no notable difference between a normal, healthy process and a leaking process).</li> <li>Using <strong>pympler</strong> shows a few tens of MB (only) used by Python objects (mostly dict, list, str and other native containers).</li> <li><p><strong>Valgrind</strong> with leak-check=full enabled doesn't show any major leaks (only couple of MBs 'definitively lost') - so C extensions are not the culprit. The total memory also doesn't add up with the 400MB+ shown by <strong>top</strong>:</p> <p><code>==23072== HEAP SUMMARY:<br> ==23072== in use at exit: 65,650,760 bytes in 463,153 blocks<br> ==23072== total heap usage: 124,269,475 allocs, 123,806,322 frees, 32,660,215,602 bytes allocated</code></p></li> </ul> <p>The only explanation I can find is that some objects are not tracked by the garbage collector, so that they are not shown by objgraph and pympler, yet use an enormous amount of RAM.</p> <p>What other tools or solutions do I have? Would compiling the Python interpreter in debug mode help, by using sys.getobjects?</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USTommy
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMemory leak in Python Twisted: where is it?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POMemory leak in Python Twisted: where is it?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POMemory leak in Python Twisted: where is it?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAnother tool to look at is heapy. The answer on http://stackoverflow.com/questions/1331561/using-heapys-memory-profile-browser-with-twisted-web gives an example of one way you might want to use it.
      singulars
      1. POMemory leak in Python Twisted: where is it?
      1. USJean-Paul Calderone
    2. COI tried that, unfortunately Heapy crashes the process when I try to use it.
      singulars
      1. POMemory leak in Python Twisted: where is it?
      1. USTommy
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload