StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POpython import path: packages with the same name in different folders
text
Body
copied!<p>I am developing several Python projects for several customers at the same time. A simplified version of my project folder structure looks something like this:</p> <pre><code>/path/ to/ projects/ cust1/ proj1/ pack1/ __init__.py mod1.py proj2/ pack2/ __init__.py mod2.py cust2/ proj3/ pack3/ __init__.py mod3.py </code></pre> <p>When I for example want to use functionality from <code>proj1</code>, I extend <code>sys.path</code> by <code>/path/to/projects/cust1/proj1</code> (e.g. by setting <code>PYTHONPATH</code> or adding a <code>.pth</code> file to the <code>site_packages</code> folder or even modifying <code>sys.path</code> directly) and then import the module like this:</p> <pre><code>>>> from pack1.mod1 import something </code></pre> <p>As I work on more projects, it happens that different projects have identical package names:</p> <pre><code>/path/ to/ projects/ cust3/ proj4/ pack1/ <-- same package name as in cust1/proj1 above __init__.py mod4.py </code></pre> <p>If I now simply extend <code>sys.path</code> by <code>/path/to/projects/cust3/proj4</code>, I still can import from <code>proj1</code>, but not from <code>proj4</code>:</p> <pre><code>>>> from pack1.mod1 import something >>> from pack1.mod4 import something_else ImportError: No module named mod4 </code></pre> <p>I think the reason why the second import fails is that Python only searches the first folder in <code>sys.path</code> where it finds a <code>pack1</code> package and gives up if it does not find the <code>mod4</code> module in there. I've asked about this in an earlier question, see <a href="https://stackoverflow.com/questions/6630394/import-python-modules-with-the-same-name">import python modules with the same name</a>, but the internal details are still unclear to me.</p> <p>Anyway, the obvious solution is to add another layer of namespace qualification by turning project directories into super packages: Add <code>__init__.py</code> files to each <code>proj*</code> folder and remove these folders from the lines by which <code>sys.path</code> is extended, e.g.</p> <pre><code>$ export PYTHONPATH=/path/to/projects/cust1:/path/to/projects/cust3 $ touch /path/to/projects/cust1/proj1/__init__.py $ touch /path/to/projects/cust3/proj4/__init__.py $ python >>> from proj1.pack1.mod1 import something >>> from proj4.pack1.mod4 import something_else </code></pre> <p>Now I am running into a situation where different projects for different customers have the same name, e.g.</p> <pre><code>/path/ to/ projects/ cust3/ proj1/ <-- same project name as for cust1 above __init__.py pack4/ __init__.py mod4.py </code></pre> <p>Trying to import from <code>mod4</code> does not work anymore for the same reason as before:</p> <pre><code>>>> from proj1.pack4.mod4 import yet_something_else ImportError: No module named pack4.mod4 </code></pre> <p>Following the same approach that solved this problem before, I would add yet another package / namespace layer and turn customer folders into super super packages.</p> <p>However, this clashes with other requirements I have to my project folder structure, e.g.</p> <ul> <li>Development / Release structure to maintain several code lines</li> <li>other kinds of source code like e.g. JavaScript, SQL, etc.</li> <li>other files than source files like e.g. documents or data.</li> </ul> <p>A less simplified, more real-world depiction of some project folders looks like this:</p> <pre><code>/path/ to/ projects/ cust1/ proj1/ Development/ code/ javascript/ ... python/ pack1/ __init__.py mod1.py doc/ ... Release/ ... proj2/ Development/ code/ python/ pack2/ __init__.py mod2.py </code></pre> <p>I don't see how I can satisfy the requirements the python interpreter has to a folder structure and the ones that I have at the same time. Maybe I could create an extra folder structure with some symbolic links and use that in <code>sys.path</code>, but looking at the effort I'm already making, I have a feeling that there is something fundamentally wrong with my entire approach. On a sidenote, I also have a hard time believing that python really restricts me in my choice of source code folder names as it seems to do in the case depicted.</p> <p><strong>How can I set up my project folders and <code>sys.path</code> so I can import from all projects in a consistent manner if there are project and packages with identical names ?</strong></p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload