Note that there are some explanatory texts on larger screens.

plurals
  1. POCreating a class within a function and access a function defined in the containing function's scope
    primarykey
    data
    text
    <blockquote> <p><strong>Edit</strong>:</p> <p>See my full answer at the bottom of this question.</p> <p><strong>tl;dr answer</strong>: Python has statically nested scopes. The <strong>static</strong> aspect can interact with the implicit variable declarations, yielding non-obvious results.</p> <p>(This can be especially surprising because of the language's generally dynamic nature).</p> </blockquote> <p>I thought I had a pretty good handle on Python's scoping rules, but this problem has me thoroughly stymied, and my google-fu has failed me (not that I'm surprised - look at the question title ;)</p> <p>I'm going to start with a few examples that work as expected, but feel free to skip to example 4 for the juicy part.</p> <p><strong>Example 1.</strong></p> <pre><code>&gt;&gt;&gt; x = 3 &gt;&gt;&gt; class MyClass(object): ... x = x ... &gt;&gt;&gt; MyClass.x 3 </code></pre> <p>Straightforward enough: during class definition we're able to access the variables defined in the outer (in this case global) scope.</p> <p><strong>Example 2.</strong></p> <pre><code>&gt;&gt;&gt; def mymethod(self): ... return self.x ... &gt;&gt;&gt; x = 3 &gt;&gt;&gt; class MyClass(object): ... x = x ... mymethod = mymethod ... &gt;&gt;&gt; MyClass().mymethod() 3 </code></pre> <p>Again (ignoring for the moment <em>why</em> one might want to do this), there's nothing unexpected here: we can access functions in the outer scope.</p> <p><strong>Note</strong>: as Frédéric pointed out below, this function doesn't seem to work. See Example 5 (and beyond) instead.</p> <p><strong>Example 3.</strong></p> <pre><code>&gt;&gt;&gt; def myfunc(): ... x = 3 ... class MyClass(object): ... x = x ... return MyClass ... &gt;&gt;&gt; myfunc().x Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; File "&lt;stdin&gt;", line 3, in myfunc File "&lt;stdin&gt;", line 4, in MyClass NameError: name 'x' is not defined </code></pre> <p><s>This is essentially the same as example 1: we're accessing the outer scope from within the class definition, just this time that scope isn't global, thanks to <code>myfunc()</code>.</s></p> <p><strong>Edit 5:</strong> As <a href="https://stackoverflow.com/a/20240755/260491">@user3022222 pointed out below</a>, I botched this example in my original posting. I believe this fails because only functions (not other code blocks, like this class definition) can access variables in the enclosing scope. For non-function code blocks, only local, global and built-in variables are accessible. A more thorough explanation is available in <a href="https://stackoverflow.com/questions/20246523/how-references-to-variables-are-resolved-in-python">this question</a></p> <p>One more:</p> <p><strong>Example 4.</strong></p> <pre><code>&gt;&gt;&gt; def my_defining_func(): ... def mymethod(self): ... return self.y ... class MyClass(object): ... mymethod = mymethod ... y = 3 ... return MyClass ... &gt;&gt;&gt; my_defining_func() Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; File "&lt;stdin&gt;", line 4, in my_defining_func File "&lt;stdin&gt;", line 5, in MyClass NameError: name 'mymethod' is not defined </code></pre> <p>Um...excuse me?</p> <p>What makes this any different from example 2?</p> <p>I'm completely befuddled. Please sort me out. Thanks!</p> <p>P.S. on the off-chance that this isn't just a problem with my understanding, I've tried this on Python 2.5.2 and Python 2.6.2. Unfortunately those are all I have access to at the moment, but they both exhibit the same behaviour.</p> <p><strong>Edit</strong> According to <a href="http://docs.python.org/tutorial/classes.html#python-scopes-and-namespaces" rel="noreferrer">http://docs.python.org/tutorial/classes.html#python-scopes-and-namespaces</a>: at any time during execution, there are at least three nested scopes whose namespaces are directly accessible:</p> <ul> <li>the innermost scope, which is searched first, contains the local names</li> <li>the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names</li> <li>the next-to-last scope contains the current module’s global names</li> <li>the outermost scope (searched last) is the namespace containing built-in names</li> </ul> <p>#4. seems to be a counter-example to the second of these.</p> <p><strong>Edit 2</strong></p> <p><strong>Example 5.</strong></p> <pre><code>&gt;&gt;&gt; def fun1(): ... x = 3 ... def fun2(): ... print x ... return fun2 ... &gt;&gt;&gt; fun1()() 3 </code></pre> <p><strong>Edit 3</strong></p> <p>As @Frédéric pointed out the assignment of to a variable of the same name as it has in the outer scope seems to "mask" the outer variable, preventing the assignment from functioning.</p> <p>So this modified version of Example 4 works:</p> <pre><code>def my_defining_func(): def mymethod_outer(self): return self.y class MyClass(object): mymethod = mymethod_outer y = 3 return MyClass my_defining_func() </code></pre> <p>However this doesn't:</p> <pre><code>def my_defining_func(): def mymethod(self): return self.y class MyClass(object): mymethod_temp = mymethod mymethod = mymethod_temp y = 3 return MyClass my_defining_func() </code></pre> <p>I still don't fully understand why this masking occurs: shouldn't the name binding occur when the assignment happens?</p> <p>This example at least provides some hint (and a more useful error message):</p> <pre><code>&gt;&gt;&gt; def my_defining_func(): ... x = 3 ... def my_inner_func(): ... x = x ... return x ... return my_inner_func ... &gt;&gt;&gt; my_defining_func()() Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; File "&lt;stdin&gt;", line 4, in my_inner_func UnboundLocalError: local variable 'x' referenced before assignment &gt;&gt;&gt; my_defining_func() &lt;function my_inner_func at 0xb755e6f4&gt; </code></pre> <p>So it appears that the local variable is defined at function creation (which succeeds), resulting in the local name being "reserved" and thus masking the outer-scope name when the function is called.</p> <p>Interesting.</p> <p>Thanks Frédéric for the answer(s)!</p> <p>For reference, from <a href="http://docs.python.org/tutorial/classes.html#python-scopes-and-namespaces" rel="noreferrer">the python docs</a>:</p> <blockquote> <p>It is important to realize that scopes are determined textually: the global scope of a function defined in a module is that module’s namespace, no matter from where or by what alias the function is called. On the other hand, the actual search for names is done dynamically, at run time — however, the language definition is evolving towards static name resolution, at “compile” time, so don’t rely on dynamic name resolution! (In fact, local variables are already determined statically.)</p> </blockquote> <p><strong>Edit 4</strong></p> <h2>The Real Answer</h2> <p>This seemingly confusing behaviour is caused by Python's <a href="http://www.python.org/dev/peps/pep-0227/" rel="noreferrer">statically nested scopes as defined in PEP 227</a>. It actually has nothing to do with <a href="http://www.python.org/dev/peps/pep-3104/" rel="noreferrer">PEP 3104</a>.</p> <p>From PEP 227:</p> <blockquote> <p>The name resolution rules are typical for statically scoped languages [...] [except] variables are not declared. If a name binding operation occurs anywhere in a function, then that name is treated as local to the function and all references refer to the local binding. If a reference occurs before the name is bound, a NameError is raised.</p> <p>[...]</p> <p>An example from Tim Peters demonstrates the potential pitfalls of nested scopes in the absence of declarations:</p> <pre><code>i = 6 def f(x): def g(): print i # ... # skip to the next page # ... for i in x: # ah, i *is* local to f, so this is what g sees pass g() </code></pre> <p>The call to g() will refer to the variable i bound in f() by the for loop. If g() is called before the loop is executed, a NameError will be raised.</p> </blockquote> <p>Lets run two simpler versions of Tim's example:</p> <pre><code>&gt;&gt;&gt; i = 6 &gt;&gt;&gt; def f(x): ... def g(): ... print i ... # ... ... # later ... # ... ... i = x ... g() ... &gt;&gt;&gt; f(3) 3 </code></pre> <p>when <code>g()</code> doesn't find <code>i</code> in its inner scope, it dynamically searches outwards, finding the <code>i</code> in <code>f</code>'s scope, which has been bound to <code>3</code> through the <code>i = x</code> assignment.</p> <p>But changing the order the final two statements in <code>f</code> causes an error:</p> <pre><code>&gt;&gt;&gt; i = 6 &gt;&gt;&gt; def f(x): ... def g(): ... print i ... # ... ... # later ... # ... ... g() ... i = x # Note: I've swapped places ... &gt;&gt;&gt; f(3) Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; File "&lt;stdin&gt;", line 7, in f File "&lt;stdin&gt;", line 3, in g NameError: free variable 'i' referenced before assignment in enclosing scope </code></pre> <p>Remembering that PEP 227 said "The name resolution rules are typical for statically scoped languages", lets look at the (semi-)equivalent C version offer:</p> <pre><code>// nested.c #include &lt;stdio.h&gt; int i = 6; void f(int x){ int i; // &lt;--- implicit in the python code above void g(){ printf("%d\n",i); } g(); i = x; g(); } int main(void){ f(3); } </code></pre> <p>compile and run:</p> <pre><code>$ gcc nested.c -o nested $ ./nested 134520820 3 </code></pre> <p>So while C will happily use an unbound variable (using whatever happens to have been stored there before: 134520820, in this case), Python (thankfully) refuses.</p> <p>As an interesting side-note, statically nested scopes enable what <a href="https://stackoverflow.com/questions/2933470/how-do-i-call-setattr-on-the-current-module/2933481#comment2987971_2933481">Alex Martelli has called</a> "the single most important optimization the Python compiler does: a function's local variables are not kept in a dict, they're in a tight vector of values, and each local variable access uses the index in that vector, not a name lookup."</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload