Because Python checks in the directories in sequential order, starting at the first directory in the sys.path list, until it finds the .py file it was looking for.

Normally, the current directory or the directory of the script is the first element in the list, unless you modify it (like you did). From documentation:

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.

So, most probably, you had a .py file with the same name as the module you were trying to import from, in the current directory (where the script was being run from).

Also, something to note about ImportErrors - Let's say the import error says: ImportError: No module named main. This doesn't mean main.py is overwritten - if it was overwritten it would still exist and we would not have issues trying to read it. It's some module above this that got overwritten with a .py or some other file.

For example, consider a directory structure that looks like this:

 - test
    - shared
       - __init__.py
       - phtest.py
    - testmain.py

Now from testmain.py, I call from shared import phtest, which works fine.

Now let's say I introduce a shared.py in test directory:

 - test
    - shared
       - __init__.py
       - phtest.py
    - testmain.py 
    - shared.py

Now when I try to do from shared import phtest from testmain.py, I get the error:

ImportError: cannot import name 'phtest'

As you can see above, the file that introduced the issue is shared.py, not phtest.py.

The correct way to use relative import is:

from ..Common import foo1

And you also need a __init__.py in all your folders.

If you have multiple versions of a package / module, you need to be using virtualenv (emphasis mine):

virtualenv is a tool to create isolated Python environments.

The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.

Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.

Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.

In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn’t share libraries with other virtualenv environments (and optionally doesn’t access the globally installed libraries either).

That's why people consider insert(0, to be wrong -- it's an incomplete, stopgap solution to the problem of managing multiple environments.

Pythonic Solution: Use packages and modules as they are intended to be used

Ask yourself, why did you want to create a src directory?

I would suggest that more than likely you wanted to follow a convention you knew from another language. (Maybe Java, maybe C, C++, or something else.)

However, if you use Python packages in the way they are intended to be used, there is a far simpler solution.

First lets review a few key points.

• To run the Python file main.py, you run it using the Python interpreter like so: python3 main.py.
• When the Python interpreter starts, it adds the current working directory to its path. (via sys.path)
• It will also add the directory containing the module to be run to its path
• More information can be found in this documentation page: https://docs.python.org/3/tutorial/modules.html
• One issue with that documentation page is it explains how to modify sys.path from within Python code. The issue with that is it gives developers the idea that this is possible and therefore should be used as a solution to import and path problems when it should not.
• The Python interpreter will search the sys.path and PYTHONPATH directories list for modules and packages to resolve when it sees an import statement
• A Python package is a directory with an __init__.py file
• A Python module is just a regular Python file
• The __init__.py file is there to signal to the Python interpreter that it needs to recursively search subdirectories for more Python packages and modules. This is why an __init__.py is usually empty.
• Without an __init__.py the Python interpreter will simply ignore a directory
• This rule is actually an optmization to prevent the interpreter from becoming slow to start up if there are a large number of subdirectories and files to search
• From this we conclude that all local source code should be resolvable from the same directory as the one used to run the target module (main.py)

With that information, you can re-structure your project:

sample_project/
    my_python_package/
        __init__.py
        sub_package_1/
            __init__.py
            utils.py
        sub_package_2/
            __init__.py 
            models.py
    app.py

Run app.py from the directory sample_project: python3 app.py

You can actually go further. If your project becomes very large, it sometimes makes sense to run modules within packages using python3 -m some_package.some_module. Then everything, including app.py becomes a package. I don't think you need this in this particular case, but if you have large numbers of "executable" Python files which are better grouped into a set of directories, then this is the approach to take.

Note that:

• This solution is simple (bordering on trivial, if not necessarily that obvious)
• There is no src directory. Forget about src. This works well in other languages, it doesn't fit into the Python model for how a project should be structured
• You did not need to modify PYTHONPATH
• You did not need to modify sys.path
• You did not need to write extra code to be able to resolve imports
• This solution is easy to understand, it is straight forward and has minimal complexity

An experiment to learn about PYTHONPATH and sys.path

You can find out what PYTHONPATH and sys.path are set to with a short experimental code:

$ cd ~
$ mkdir python-path-test
$ touch python-path-test/main.py 

# main.py

import os
import sys

print(f'PYTHONPATH:')
for string in os.environ.get('PYTHONPATH').split(';'):
    print(string)

print(f'sys.path:')
for string in sys.path:
    print(string)

$ export PYTHONPATH=`pwd`
$ python3 python-path-test/main.py
PYTHONPATH:
/home/username
sys.path:
/home/username
/home/username/python-path-test
/usr/lib/python311.zip
/usr/lib/python3.11
/usr/lib/python3.11/lib-dynload
/usr/local/lib/python3.11/dist-packages
/usr/lib/python3/dist-packages
/usr/lib/python3.11/dist-packages

Further explanation in regards to other answers

Let me address the issues with the other answers here. All of the answers provided will work, but none of them take the simplest and "most obviously correct" approach.

The reason for this is the "most obviously correct" approach is not that obvious, especially if you come to Python from other languages where things work differently.

Just to say as well - it took me a long time to figure out the solution to the exact same problem which is shown in the question and I only figured out the solution when I went to work for a firm where someone else had figured this out before me.

Also: None of this is really explained on any documentation page anywhere, so it is hardly surprising that most people get it wrong, or do something unneccessarily complex when it isn't needed.

Overview of other solutions:

So far several other solutions have been proposed:

1. Use setuptools and virtual environments to manage what is known as an "editable install".

I don't like this for two reasons: It is more work than is necessary, and you are pretending that some local source code is a PIP package, when it isn't. It just seems like a bizzare thing to do. (This is exactly what I used to do before realizing there is an easier way.)

2. Write Python code to modify the sys.path or PYTHONPATH environment variable

I don't like this because it is a hack:

• The PYTHONPATH environment variable is intended to be used to store the locations of installed packages on your system
• It should be a semi-permenant thing which doesn't change (often)
• This is similarly the case with sys.path
• The other reason modifying PYTHONPATH is bad is because you are embedding (hiding) some code within your project which does unexpected things
• PYTHONPATH should be managed by the Operating System, or at least by the user in a shell
• In my experience, twiddling things which should be managed by your operating system from within code frequently leads to hard to find bugs and hard to understand code

3. Modify PYTHONPATH from a shell

This is better than the above proposal of modifying it from with Python code, but it just isn't necessary, for the reasons I explained above.

Appendix:

To give a little further helpful information. Some languages (and corresponding build tools) are designed with maximum flexibility. Others contain built-in rules which constrain how files and folders should be arranged for the build system to work property. These rules are not always explicit or obvious.

cmake is a good example of a build system which offers maximum flexibility. Many projects contain a src directory, under which all the C/C++ code lives. The reason for this is cmake facilitates using explicit and arbitrary paths to configure the build.

On the other hand, the Rust module system is much more constrained. The existence of a directory "creates" a module (or submodule). Cargo and Rust require you to use the filesystem in a constrained way to get the modular structure you want.

Julia is more similar to C++ in that modules are explicit - there is a module keyword, and this is the only way to create a module. It also has include which can take an arbitrary path - although using the Julia build system in an arbitrary way is not recommended, just as it would not be recommended with cmake.

Finally, Python is a bit more tricky. Similarly to Julia, the build system needs to be told how and where modules can be loaded from. It is generally better not to add lots of arbitrary hard-coded paths to the code or build system. Rather, avoiding this and working with what the language offers natively is preferable.

In both the case of Julia and Python, this means that the interpreter/runtime should be able to load your code without adding additional paths.

With this constraint, you will write a much simpler project structure.

To answer some questions in the comments

Here is what one of my Python projects looks like.

python_project_root_directory/
  .vscode/
    settings.json
  .venv/
  lib_something/
    __init__.py
    lib_something_files.py
  the_main_module
    __main__.py
    __init__.py # might not be requried
    main.py # called from __main__.py
  tests/
    some_group/
      test_something.py
      test_another_thing.py
    another_group/
      test_more_things.py
  Dockerfile

Dockerfile

Note: Does not use .venv, because a Docker container is its own isolated environment. You can use a .venv if you want. Change the command to cmd ["./.venv/bin/python3", "-m", "the_main_module"].

from python:3.12-bookworm
... other stuff ...
run pip3 install --no-cache-dir --upgrade -r requirements.txt
cmd ["python3", "-m", "the_main_module"]

If rather than wanting to run a main module, you want to run a python file as "main", change to cmd ["python3", "main.py"].

settings.json

{
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
}