Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I assume you're getting the size of objects with something like this:</p> <pre><code>sum(sys.getsizeof(i) for i in gc.get_objects()) </code></pre> <p>Remember that the result of <code>gc.get_objects()</code> doesn't include memory consumed by the interpreter itself, only Python objects tracked by the garbage collector. Also, this function relies on objects returning accurate results from their <code>__sizeof__()</code> method, so if you're using any third party modules then you can't necessarily expect accurate results.</p> <p>You may have already done this, but you can have your application periodically call <code>gc.collect()</code> and then check <code>gc.garbage</code> to see if you've got any unreachable objects that the collector couldn't free. This will be the case if you have any classes with circular reference on which you've overridden <code>__del__()</code> (see <a href="http://docs.python.org/2/library/gc.html?highlight=gc#gc.garbage" rel="nofollow noreferrer">Python docs for gc</a>).</p> <p>I would also suggest adding this call to the start of your code:</p> <pre><code>gc.set_debug(gc.DEBUG_UNCOLLECTABLE | gc.DEBUG_INSTANCES | gc.DEBUG_OBJECTS) </code></pre> <p>This will print messages to <code>stderr</code> whenever objects are found with circular references and the like. This might give you useful information on where to look more closely.</p> <p>So far these sorts of problems would have likely been picked up by objgraph, though, but it might be worth putting them in anyway because you can leave them active in a long-running daemon to get a record over time of when such issues occurred.</p> <p>If you're using any C extensions (either that you've written yourself or third party ones) then carefully check the code for errors in handling reference counts - it's fairly easy to make a mistake there. Also, bear in mind that there's nothing to stop extensions allocating their own memory outside of Python's allocators - in this case nothing you do within the Python interpreter will detect it if these leak.</p> <p>All that said, if you're still seeing memory usage monotonically increasing and you can't find any Python objects that are the cause then it's probably time to check for leaks in lower-level C code or the Python interpreter itself - a good tool for this is <a href="http://valgrind.org/" rel="nofollow noreferrer">Valgrind</a>.</p> <p>To use Valgrind with Python you'll want to use <a href="http://hg.python.org/cpython/file/4617b7ac302a/Misc/valgrind-python.supp" rel="nofollow noreferrer">the Python suppression file</a>. You might find this is already installed - Ubuntu places a modified form of this file in <code>/usr/lib/valgrind/python.supp</code>, for example.</p> <p>To do this properly, you'll want to recompile Python as described in the <a href="http://hg.python.org/cpython/file/4617b7ac302a/Misc/README.valgrind" rel="nofollow noreferrer">README.valgrind</a> file in the Python distribution, but you might find you get some interesting results even without this.</p> <p>There's more discussion about running Python under Valgrind in <a href="https://stackoverflow.com/questions/3982036/how-can-i-use-valgrind-with-python-c-extensions">this stack overflow question</a>.</p>
    singulars
    1. This table or related slice is empty.
    1. POmemory leak, size of objects tracked by gc is far less than memory usage of process
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USCommunity
    1. USCartroo
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POmemory leak, size of objects tracked by gc is far less than memory usage of process
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThank you, Cartroo. After gc.collect(), gc.garbage() did get something unreachable before, and we have resolved that. The third party modules we used include "twisted". Ok, now we will try to find the problem in twisted and python itself.
      singulars
      1. PO
      1. USapporc
    2. COIt looks like [you're not the only person to see leaks in Twisted](http://stackoverflow.com/questions/4078084/memory-leak-in-python-twisted-where-is-it). In the past I've had to do staggered automated restarts of services (one redundant server at a time) when I've had issues like this in third party software and no time to track them down. At least that's a workaround that might reduce the impact of the problem.
      singulars
      1. PO
      1. USCartroo
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload