You seem to be misunderstanding how import searches for modules. When you use an import statement it always searches the actual module path (and/or sys.modules); it doesn't make use of module objects in the local namespace that exist because of previous imports. When you do:
import package.subpackage.module
from package.subpackage import module
from module import attribute1
The second line looks for a package called package.subpackage and imports module from that package. This line has no effect on the third line. The third line just looks for a module called module and doesn't find one. It doesn't "re-use" the object called module that you got from the line above.
In other words from someModule import ... doesn't mean "from the module called someModule that I imported earlier..." it means "from the module named someModule that you find on sys.path...". There is no way to "incrementally" build up a module's path by importing the packages that lead to it. You always have to refer to the entire module name when importing.
It's not clear what you're trying to achieve. If you only want to import the particular object attribute1, just do from package.subpackage.module import attribute1 and be done with it. You need never worry about the long package.subpackage.module once you've imported the name you want from it.
If you do want to have access to the module to access other names later, then you can do from package.subpackage import module and, as you've seen you can then do module.attribute1 and so on as much as you like.
If you want both --- that is, if you want attribute1 directly accessible and you want module accessible, just do both of the above:
from package.subpackage import module
from package.subpackage.module import attribute1
attribute1 # works
module.someOtherAttribute # also works
If you don't like typing package.subpackage even twice, you can just manually create a local reference to attribute1:
from package.subpackage import module
attribute1 = module.attribute1
attribute1 # works
module.someOtherAttribute #also works
You seem to be misunderstanding how import searches for modules. When you use an import statement it always searches the actual module path (and/or sys.modules); it doesn't make use of module objects in the local namespace that exist because of previous imports. When you do:
import package.subpackage.module
from package.subpackage import module
from module import attribute1
The second line looks for a package called package.subpackage and imports module from that package. This line has no effect on the third line. The third line just looks for a module called module and doesn't find one. It doesn't "re-use" the object called module that you got from the line above.
In other words from someModule import ... doesn't mean "from the module called someModule that I imported earlier..." it means "from the module named someModule that you find on sys.path...". There is no way to "incrementally" build up a module's path by importing the packages that lead to it. You always have to refer to the entire module name when importing.
It's not clear what you're trying to achieve. If you only want to import the particular object attribute1, just do from package.subpackage.module import attribute1 and be done with it. You need never worry about the long package.subpackage.module once you've imported the name you want from it.
If you do want to have access to the module to access other names later, then you can do from package.subpackage import module and, as you've seen you can then do module.attribute1 and so on as much as you like.
If you want both --- that is, if you want attribute1 directly accessible and you want module accessible, just do both of the above:
from package.subpackage import module
from package.subpackage.module import attribute1
attribute1 # works
module.someOtherAttribute # also works
If you don't like typing package.subpackage even twice, you can just manually create a local reference to attribute1:
from package.subpackage import module
attribute1 = module.attribute1
attribute1 # works
module.someOtherAttribute #also works
The reason #2 fails is because sys.modules['module'] does not exist (the import routine has its own scope, and cannot see the module local name), and there's no module module or package on-disk. Note that you can separate multiple imported names by commas.
from package.subpackage.module import attribute1, attribute2, attribute3
Also:
from package.subpackage import module
print module.attribute1
I have a project named ProjectA that I wrote that has a structure like this (__init__.py is present in all sub directories but not shown here):
-
main_file.py
-
libs/filea.py
-
libs/fileb.py
-
options/filec.py
-
classes/classa.py
-
classes/classb.py
-
functions/func1.py
-
functions/func2.py
The main_file.py will import or its imports will import part of every file listed above. I used absolute imports for each of those and it works great when I run python main_file.py.
Now I want to use what I wrote as a submodule for another project. It will be a simple script that imports a function from main_file.py (which in turn imports from all the files above).
What is the proper way to do the imports so that I can run both python main_file.py and use a function from main_file.py in another script? I know I can turn all of the imports in ProjectA into relative imports, is that the best way?
Python submodule imports using __init__.py - Stack Overflow
ModuleNotFoundError: No module named 'supermodule.submodule'; 'supermodule' is not a package
git - How to structure Python modules so they are accesible from different submodules - Software Engineering Stack Exchange
How to import a submodule in Python? (without `exec`) - Stack Overflow
You probably already understand that when you import a module, the interpreter creates a new namespace and executes the code of that module with the new namespace as both the local and global namespace. When the code completes execution, the module name (or the name given in any as clause) is bound to the module object just created within the importing namespace and recorded against its __name__ in sys.modules.
When a qualified name such as package.subpackage.module is imported the first name (package) is imported into the local namespace, then subpackage is imported into package's namespace and finally module is imported into package.subpackage's namespace. Imports using from ... import ... as ... perform the same sequence of operations, but the imported objects are bound directly to names in the importing module's namespace. The fact that the package name isn't bound in your local namespace does not mean it hasn't been imported (as inspection of sys.modules will show).
The __init__.py in a package serves much the same function as a module's .py file. A package, having structure, is written as a directory which can also contain modules (regular .py files) and subdirectories (also containing an __init__.py file) for any sub_packages. When the package is imported a new namespace is created and the package's __init__.py is executed with that namespace as the local and global namespaces. So to answer your problem we can strip your filestore down by omitting the top-level package, which will never be considered by the interpreter when test.py is run as a program. It would then look like this:
test.py
subpackage/
__init__.py
hello_world.py
Now, subpackage is no longer a sub-package, as we have removed the containing package as irrelevant. Focusing on why the do_something name is undefined might help. test.py does not contain any import, and so it's unclear how you are expecting do_something to acquire meaning. You could make it work by using an empty subpackage/__init__.py and then you would write test.py as
from subpackage.hello_world import do_something
do_something()
Alternatively you could use a subpackage/__init__.py that reads
from hello_world import do_something
which establishes the do_something function inside the subpackage namespace when the package is imported. Then use a test.py that imports the function from the package, like this:
from subpackage import do_something
do_something()
A final alternative with the same __init__.py is to use a test.py that simply imports the (sub)package and then use relative naming to access the required function:
import subpackage
subpackage.do_something()
to gain access to it in your local namespace.
With the empty __init__.py this could also be achieved with a test.py reading
import subpackage.hello_world
subpackage.hello_world.do_something()
or even
from subpackage.hello_world import do_something
do_something()
An empty __init__.py will mean that the top-level package namespace will contain only the names of any subpackages the program imports, which allows you to import only the subpackages you require. This gives you control over the namespace of the top-level package.
While it's perfectly possible to define classes and functions in the
__init__.py , a more normal approach is to import things into that namespace from submodules so that importers can just import the top-level package to gain access to its contents with a single-level attribute
reference, or even use from to import only the names you specifically want.
Ultimately the best tool to keep you straight is a clear understanding of how import works and what effect its various forms have on the importing namespace.
First, you have to understand how import alone work:
import test.submodule.do_something
Would try to load do_something from submodule itself loaded from test.
You want to load something from subpackage, so start with that:
import subpackage
Fine, subpackage/__init__.py is loaded.
Now, you want the do_something() function which is in the file (a "module") hello_world.py. Easy:
from subpackage.hello_world import do_something
And you are done! Just read this line loud, it does exactly what it says: import do_something from the module hello_world which is in the subpackage package.
Try that in test.py
from subpackage.hello_world import do_something
do_something()
It should work just fine.
Now, the second issue:
__init__.py won't be called in package/ since you don't use package/ as a package. __init__.py will be used if you do an import of package/ or anything in it, for eg:
from package import test
Otherwise, it won't be loaded at all.
However, if you want to load do_something() on the import of subpackage, put from submodule.hello_word import do_something in subpackage/__init__.py, and then, in you test.py, do a import subpackage.
Use importlib.import_module:
>>> import importlib
>>> importlib.import_module('os.path')
<module 'posixpath' from '/usr/lib/python2.7/posixpath.pyc'>
This should work in python2.7+ and 3.1+.
Note that if you want do: from A import B as C as a function call, importlib.import_module won't always work, since B may not be a module.
Heres a function which uses importlib and getattr.
def my_import_from(mod_name, var_name):
import importlib
mod = importlib.import_module(mod_name)
var = getattr(mod, var_name)
return var
So this:
from os.path import dirname as var
Can be replaced with this:
var = my_import_from("os.path", "dirname")
Which avoids exec and allows both submodules and any variables defined in the module.
Since my question explicitly says importing a submodule, the answer from @Bakuriu is correct, however including this for completeness and it may help others who run into the same problem.