Most of the stuff is set up in Python's site.py which is automatically imported when starting the interpreter (unless you start it with the -S option). Few paths are set up in the interpreter itself during initialization (you can find out which by starting python with -S).
Additionally, some frameworks (like Django I think) modify sys.path upon startup to meet their requirements.
The site module has a pretty good documentation, a commented source code and prints out some information if you run it via python -m site.
Most of the stuff is set up in Python's site.py which is automatically imported when starting the interpreter (unless you start it with the -S option). Few paths are set up in the interpreter itself during initialization (you can find out which by starting python with -S).
Additionally, some frameworks (like Django I think) modify sys.path upon startup to meet their requirements.
The site module has a pretty good documentation, a commented source code and prints out some information if you run it via python -m site.
From Learning Python:
sys.path is the module search path. Python configures it at program startup, automatically merging the home directory of the top-level file (or an empty string to designate the current working directory), any PYTHONPATH directories, the contents of any .pth file paths you've created, and the standard library directories. The result is a list of directory name strings that Python searches on each import of a new file.
The site module documentation and Modifying Python's Search Path seem to be what you're looking for.
As far as I understand it, those entries are being added to sys.path by:
/usr/lib/python2.6/site.py/usr/lib/python2.6/dist-packages/site.py
(Change 2.6 to your version of Python.)
The easiest way to change it is to add a file /usr/local/lib/python2.6/dist-packages/site-packages.pth containing ../site-packages.
Alternatively, maybe you can teach the package to use site.getsitepackages()?
I'd like to summarize my findings about python's path modification. There are two ways to do it.
- .pth file
PYTHONPATH
Any .pth file which is found on the default path (see bellow) will get its content included into sys.path.
Format of said .pth file is simple: one (folder) path per line. Surprisingly, the paths can be absolute or relative to the .pth file.
Default path is where the interpreter resides and <some-prefix>/lib/python<version>/site-packages where <some-prefix> is usually /usr/.
PYTHONPATH is environmental variable of your operating system. On unix systems you list them by env. Global modification of such variables is done through .sh scripts inside /etc/profile.d/ folder as mentioned by @TestUser16418.
OK, I started to craft a solution that involved PEP 302 import hooks. Way too complicated.
I think you answered your own question with:
this "gateway" script could implement all manner of bootstrapping functionality - for example, ensuring that virtualenv is set up correctly and activated for the current branch.
if you are already using virtualenv, put the logic you have in the "~/venv/or-whatever-path/bin/activate" script that is already there. I'd suggest listing of possible entry points and prompting for a response (list with numbers and a raw_input() statement will do) if you are not specifying one.
I'd also put the name of your branch in to $PS1 so that is shows up in the tab name or window title of your terminal. I so something similar and change the color of the prompt so I don't do the right thing in the wrong place.
You have to use virtualenv. It create an isolated python environment. It is also recommended when you need different versions of third party packages depending of the version of your product.
How python finds its modules
Strictly taken, a module is a single python file, while a package is a folder containing python files, accompanied by a (can be empty) file named __init__.py, to tell python it is a package to import modules from. In both cases, modules need their .py extension, but importing them is done without (see further below).
By default, Python looks for its modules and packages in $PYTHONPATH.
To find out what is included in $PYTHONPATH, run the following code in python (3):
import sys
print(sys.path)
How to add a directory
Occasionally
From within a python file, you can add path(s) occasionally to the default path by adding the following lines in the head section of your python application or script:
import sys
sys.path.insert(0, "/path/to/your/package_or_module")
For example:
if I have a folder: /home/myname/pythonfiles, and I want to import the file module_1.py, located in that directory, I add this to the head section of my code:
import sys
sys.path.insert(0, "/home/myname/pythonfiles")
And I can simply import the file module_1.py by:
import module_1
When I create a package and want to import module(s) from the package, I need to create a folder in $PYTHONPATH, containing the modules, accompanied by a (can be empty) file called __init__.py
For example:
To import from a package (folder) called my_package in /home/myname/pythonfiles , add the /home/myname/pythonfiles path to your $PYTHONPATH, like in example 1, and import the module called module_2.py (inside the package folder) simply with: `
from <packagename> import module_2
Adding directories to $PYTHONPATH permanently:
Add the following line to your ~/.profile file.
export PYTHONPATH=$PYTHONPATH:/path/you/want/to/add
Subdirectories
From within a package, subdirectories are not included just like that; you need to "chain" the directories. To import a module module_3.py, inside folder subfolder inside folder packagename:
import packagename.subfolder.module_3
Given the fact that all subfolders in the package include their own __init__.py file.
When a module is in the same directory as the script or application
There is no need to insert the path to a module when it is in the same directory as the script or application, it is automatically added.
Example:
If I have a folder, containing script.py and module.py, I can simply import the module by:
import module
The correct way to use relative import is:
from ..Common import foo1
And you also need a __init__.py in all your folders.
You don't set PYTHONPATH, you add entries to sys.path. It's a list of directories that should be searched for Python packages, so you can just append your directories to that list.
import sys
sys.path.append('/path/to/whatever')
In fact, sys.path is initialized by splitting the value of PYTHONPATH on the path separator character (: on Linux-like systems, ; on Windows).
You can also add directories using site.addsitedir, and that method will also take into account .pth files existing within the directories you pass. (That would not be the case with directories you specify in PYTHONPATH.)
You can get and set environment variables via os.environ:
import os
user_home = os.environ["HOME"]
os.environ["PYTHONPATH"] = "..."
But since your interpreter is already running, this will have no effect. You're better off using
import sys
sys.path.append("...")
which is the array that your PYTHONPATH will be transformed into on interpreter startup.