Let's have MyClass in module.name module defined at /path/to/file.py. Below is how we import MyClass from this module

For Python 3.5+ use (docs):

import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = foo
spec.loader.exec_module(foo)
foo.MyClass()

For Python 3.3 and 3.4 use:

from importlib.machinery import SourceFileLoader

foo = SourceFileLoader("module.name", "/path/to/file.py").load_module()
foo.MyClass()

(Although this has been deprecated in Python 3.4.)

For Python 2 use:

import imp

foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()

There are equivalent convenience functions for compiled Python files and DLLs.

See also http://bugs.python.org/issue21436.

What is the entry point for your program? Usually the entry point for a program will be at the root of the project. Since it is at the root, all the modules within the root will be importable, provided there is an __init__.py file in them.

So, using your example:

my_project/
    main.py
    package1/
        __init__.py
        module1
        module2
    package2/
        __init__.py
        module1
        module2

main.py would be the entry point for your program. Because the file that is executed as main is automatically put on the PYTHONPATH, both package1 and package2 are available from the top level import.

# in main.py
from package1.module1 import *
from package1.module2 import *

# in package1.module1
import module2
from package2.module1 import *

# in package2.module1 import *
import module2
from package1.module1 import *

Note that in the above, package1 and package2 depend on each other. That should never be the case. But this is just an example of being able to import from anywhere.

main.py doesn't have to be anything fancy either. It can be very simple:

# main.py

if __name__ == '__main__':
    from package1.module1 import SomeClass
    SomeClass().start()

The point I'm trying to make, is that if a module needs to be accessible by other modules, that module should be available as a top level import. A module should not attempt to put itself as a top level import (directly on the PYTHONPATH).

It should be the responsibility of the project for ensuring that all imports can be satisfied if the module is included directly in the project. There are two ways to do this. The first is by creating a bootstrapper file such as main.py in the project folder. The other, is by creating a file that adds all relevant paths to PYTHONPATH, that is loaded by any entry points that may exist.

For example:

# setup.py
import sys

def load():
    paths = ['/path1/','/path2/','/path3/']
    for p in path:
        sys.path.insert(0, p)

# entrypoint.py
from setup import load
load()
# continue with program

The main thing to take away, is that a module is not supposed to put itself on the path. The path should be determined automatically by the entry point into the program, or defined explicitly by a setup script that knows where all the relevant modules are.

EDIT Nov 2014 (3 years later):

Python 2.6 and 3.x supports proper relative imports, where you can avoid doing anything hacky. With this method, you know you are getting a relative import rather than an absolute import. The '..' means, go to the directory above me:

from ..Common import Common

As a caveat, this will only work if you run your python as a module, from outside of the package. For example:

python -m Proj

Original hacky way

This method is still commonly used in some situations, where you aren't actually ever 'installing' your package. For example, it's popular with Django users.

You can add Common/ to your sys.path (the list of paths python looks at to import things):

import sys, os
sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'Common'))
import Common

os.path.dirname(__file__) just gives you the directory that your current python file is in, and then we navigate to 'Common/' the directory and import 'Common' the module.

How python finds its modules

Strictly taken, a module is a single python file, while a package is a folder containing python files, accompanied by a (can be empty) file named __init__.py, to tell python it is a package to import modules from. In both cases, modules need their .py extension, but importing them is done without (see further below).

By default, Python looks for its modules and packages in $PYTHONPATH.

To find out what is included in $PYTHONPATH, run the following code in python (3):

import sys
print(sys.path)

How to add a directory

Occasionally

From within a python file, you can add path(s) occasionally to the default path by adding the following lines in the head section of your python application or script:

import sys
sys.path.insert(0, "/path/to/your/package_or_module")

For example:

if I have a folder: /home/myname/pythonfiles, and I want to import the file module_1.py, located in that directory, I add this to the head section of my code:

import sys
sys.path.insert(0, "/home/myname/pythonfiles")

And I can simply import the file module_1.py by:

import module_1

When I create a package and want to import module(s) from the package, I need to create a folder in $PYTHONPATH, containing the modules, accompanied by a (can be empty) file called __init__.py

For example:

To import from a package (folder) called my_package in /home/myname/pythonfiles , add the /home/myname/pythonfiles path to your $PYTHONPATH, like in example 1, and import the module called module_2.py (inside the package folder) simply with: `

from <packagename> import module_2

Adding directories to $PYTHONPATH permanently:

Add the following line to your ~/.profile file.

export PYTHONPATH=$PYTHONPATH:/path/you/want/to/add

Subdirectories

From within a package, subdirectories are not included just like that; you need to "chain" the directories. To import a module module_3.py, inside folder subfolder inside folder packagename:

import packagename.subfolder.module_3

Given the fact that all subfolders in the package include their own __init__.py file.

When a module is in the same directory as the script or application

There is no need to insert the path to a module when it is in the same directory as the script or application, it is automatically added.

Example:

If I have a folder, containing script.py and module.py, I can simply import the module by:

import module

This will add a path to your Python process / instance (i.e. the running executable). The path will not be modified for any other Python processes. Another running Python program will not have its path modified, and if you exit your program and run again the path will not include what you added before. What are you are doing is generally correct.

set.py:

import sys
sys.path.append("/tmp/TEST")

loop.py

import sys
import time
while True:
  print sys.path
  time.sleep(1)

run: python loop.py &

This will run loop.py, connected to your STDOUT, and it will continue to run in the background. You can then run python set.py. Each has a different set of environment variables. Observe that the output from loop.py does not change because set.py does not change loop.py's environment.

A note on importing

Python imports are dynamic, like the rest of the language. There is no static linking going on. The import is an executable line, just like sys.path.append....