StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POGarbage collect a class with a reference to its instance?
text
Body
copied!<p>Consider this code snippet:</p> <pre class="lang-python prettyprint-override"><code>import gc from weakref import ref def leak_class(create_ref): class Foo(object): # make cycle non-garbage collectable def __del__(self): pass if create_ref: # create a strong reference cycle Foo.bar = Foo() return ref(Foo) # without reference cycle r = leak_class(False) gc.collect() print r() # prints None # with reference cycle r = leak_class(True) gc.collect() print r() # prints <class '__main__.Foo'> </code></pre> <p>It creates a reference cycle that cannot be collected, because the referenced instance has a <code>__del__</code> method. The cycle is created here:</p> <pre><code># create a strong reference cycle Foo.bar = Foo() </code></pre> <p>This is just a proof of concept, the reference could be added by some external code, a descriptor or anything. If that's not clear to you, remember that each objects mantains a reference to its class:</p> <pre class="lang-none prettyprint-override"><code> +-------------+ +--------------------+ | | Foo.bar | | | Foo (class) +------------>| foo (Foo instance) | | | | | +-------------+ +----------+---------+ ^ | | foo.__class__ | +--------------------------------+ </code></pre> <p>If I could guarantee that <code>Foo.bar</code> is only accessed from <code>Foo</code>, the cycle wouldn't be necessary, as theoretically the instance could hold only a weak reference to its class.</p> <p>Can you think of a practical way to make this work without a leak?</p> <hr> <p>As some are asking why would external code modify a class but can't control its lifecycle, consider this example, similar to the real-life example I was working to:</p> <pre><code>class Descriptor(object): def __get__(self, obj, kls=None): if obj is None: try: obj = kls._my_instance except AttributeError: obj = kls() kls._my_instance = obj return obj.something() # usage example # class Example(object): foo = Descriptor() def something(self): return 100 print Example.foo </code></pre> <p>In this code only <code>Descriptor</code> (a <a href="http://docs.python.org/2/reference/datamodel.html#invoking-descriptors" rel="nofollow">non-data descriptor</a>) is part of the API I'm implementing. <code>Example</code> class is an example of how the descriptor would be used.</p> <p>Why does the descriptor store a reference to an instance inside the class itself? Basically for caching purposes. <code>Descriptor</code> required this contract with the implementor: it would be used in any class assuming that</p> <ol> <li>The class has a constructor with no args, that gives an "anonymous instance" (my definition)</li> <li>The class has some behavior-specific methods (<code>something</code> here).</li> <li>An instance of the class can stay alive for an undefined amount of time.</li> </ol> <p>It doesn't assume anything about:</p> <ol> <li>How long it takes to construct an object</li> <li>Whether the class implements <strong>del</strong> or other magic methods</li> <li>How long the class is expected to live</li> </ol> <p>Moreover the API was designed to avoid any extra load on the class implementor. I could have moved the responsibility for caching the object to the implementor, but I wanted a standard behavior.</p> <p>There actually is a simple solution to this problem: make the default behavior to cache the instance (like it does in this code) but allow the implementor to override it if they have to implement <code>__del__</code>.</p> <p>Of course this wouldn't be as simple if we assumed that the class state <em>had</em> to be preserved between calls.</p> <hr> <p>As a starting point, I was coding a "weak object", an implementation of <code>object</code> that only kept a weak reference to its class:</p> <pre><code>from weakref import proxy def make_proxy(strong_kls): kls = proxy(strong_kls) class WeakObject(object): def __getattribute__(self, name): try: attr = kls.__dict__[name] except KeyError: raise AttributeError(name) try: return attr.__get__(self, kls) except AttributeError: return attr def __setattr__(self, name, value): # TODO: implement... pass return WeakObject Foo.bar = make_proxy(Foo)() </code></pre> <p>It appears to work for a limited number of use cases, but I'd have to reimplement the whole set of <code>object</code> methods, and I don't know how to deal with classes that override <code>__new__</code>.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload