StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POAdd a subdirectory to a python namespace
text
Body
copied!<p>I would like to be able to import a python module which is actually located in a subdirectory of another module.</p> <p>I am developing a framework with plug-ins. Since I'm expecting to have a few thousands (there's currently >250 already) and I don't want one big directory containing >1000 files I have them ordered in directories like this, where they are grouped by the first letter of their name:</p> <pre><code>framework\ __init__.py framework.py tools.py plugins\ __init__.py a\ __init__.py atlas.py ... b\ __init__.py binary.py ... c\ __init__.py cmake.py ... </code></pre> <p>Since I would not like to impose a burden on developers of other plugins, or people not needing as many as I have, I would like to put each plugin in the 'framework.plugins' namespace. This way someone adding a bunch of private plugins can just do so by adding them in the folder framework.plugins and there provide a <code>__init__.py</code> file containing:</p> <pre><code>from pkgutil import extend_path __path__ = extend_path(__path__, __name__) </code></pre> <p>however, currently this setup is forcing them to also use the a-z subdirectories. Sometimes a plugin is extending another plugin, so now I have a </p> <pre><code>from framework.plugins.a import atlas </code></pre> <p>and I would like to have</p> <pre><code>from framework.pugins import atlas </code></pre> <p>Is there any way to declare a namespace where the full name space name actually doesn't map to a folder structure?</p> <p>I am aware of the pkg_resources package, but this is only available via setuptools, and I'd rather not have an extra dependency.</p> <pre><code>import pkg_resources pkg_resources.declare_namespace(__name__) </code></pre> <p>The solution should work in python 2.4-2.7.3</p> <p>update: Combining the provided answers I tried to get a list of all plugins imported in the <code>__init__.py</code> from plugins. However, this fails due to dependencies. Since a plugin in the 'c' folder tries to import a plugin starting with 't', and this one has not been added yet.</p> <pre><code>plugins = [ x[0].find_module(x[1]).load_module(x[1]) for x in pkgutil.walk_packages([ os.path.join(framework.plugins.__path__[0], chr(y)) for y in xrange(ord('a'), ord('z') + 1) ],'framework.plugins.' ) ] </code></pre> <p>I'm not sure If I'm on the right track here, or just overcomplicating things and better write my own PEP302 importer. However, I can't seem to find any decent examples of how these should work.</p> <p>Update: I tried to follow the suggesting of wrapping the <code>__getattr__</code> function in my <code>__init__.py</code>, but this seems to no avail.</p> <pre><code>import pkgutil import os import sys plugins = [x[1] for x in pkgutil.walk_packages([ os.path.join(__path__[0], chr(y)) for y in xrange(ord('a'), ord('z') + 1) ] )] import types class MyWrapper(types.ModuleType): def __init__(self, wrapped): self.wrapped = wrapped def __getattr__(self, name): if name in plugins: askedattr = name[0] + '.' + name else: askedattr = name attr = getattr(self.wrapped, askedattr) return attr sys.modules[__name__] = MyWrapper(sys.modules[__name__]) </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload