python -O does the following currently:
- completely ignores asserts
- sets the special builtin name
__debug__to False (which by default is True)
and when called as python -OO
- removes docstrings from the code
I don't know why everyone forgets to mention the __debug__ issue; perhaps it is because I'm the only one using it :) An if __debug__ construct creates no bytecode at all when running under -O, and I find that very useful.
python -O does the following currently:
- completely ignores asserts
- sets the special builtin name
__debug__to False (which by default is True)
and when called as python -OO
- removes docstrings from the code
I don't know why everyone forgets to mention the __debug__ issue; perhaps it is because I'm the only one using it :) An if __debug__ construct creates no bytecode at all when running under -O, and I find that very useful.
It saves a small amount of memory, and a small amount of disk space if you distribute any archive form containing only the .pyo files. (If you use assert a lot, and perhaps with complicated conditions, the savings can be not trivial and can extend to running time too).
So, it's definitely not useless -- and of course it's being used (if you deploy a Python-coded server program to a huge number N of server machines, why ever would you want to waste N * X bytes to keep docstrings which nobody, ever, would anyway be able to access?!). Of course it would be better if it saved even more, but, hey -- waste not, want not!-)
So it's pretty much a no-brainer to keep this functionality (which is in any case trivially simple to provide, you know;-) in Python 3 -- why add even "epsilon" to the latter's adoption difficulties?-)
Stop ignoring asserts when running in optimized mode - Ideas - Discussions on Python.org
optimization - What are the implications of running python with the optimize flag? - Stack Overflow
What does Python optimization (-O or PYTHONOPTIMIZE) do? - Stack Overflow
Usage of assert statements in optimized mode
Videos
Another use for the -O flag is that the value of the __debug__ builtin variable is set to False.
So, basically, your code can have a lot of "debugging" paths like:
if __debug__:
# output all your favourite debugging information
# and then more
which, when running under -O, won't even be included as bytecode in the .pyo file; a poor man's C-ish #ifdef.
Remember that docstrings are being dropped only when the flag is -OO.
On stripping assert statements: this is a standard option in the C world, where many people believe part of the definition of ASSERT is that it doesn't run in production code. Whether stripping them out or not makes a difference depends less on how many asserts there are than on how much work those asserts do:
def foo(x):
assert x in huge_global_computation_to_check_all_possible_x_values()
# ok, go ahead and use x...
Most asserts are not like that, of course, but it's important to remember that you can do stuff like that.
As for stripping docstrings, it does seem like a quaint holdover from a simpler time, though I guess there are memory-constrained environments where it could make a difference.
assert statements are completely eliminated, as are statement blocks of the form if __debug__: ... (so you can put your debug code in such statements blocks and just run with -O to avoid that debug code).
With -OO, in addition, docstrings are also eliminated.
From the docs:
- You can use the
-Oor-OOswitches on the Python command to reduce the size of a compiled module. The-Oswitch removes assert statements, the-OOswitch removes both assert statements and __doc__ strings. Since some programs may rely on having these available, you should only use this option if you know what you’re doing. “Optimized” modules have anopt-tag and are usually smaller. Future releases may change the effects of optimization.- A program doesn’t run any faster when it is read from a
.pycfile than when it is read from a.pyfile; the only thing that’s faster about.pycfiles is the speed with which they are loaded.
So in other words, almost nothing.
Hi Pythonistas,
I'm interested in learning what optimization techniques you know for python code. I know its a general statement, but I'm interested in really pushing execution to the maximum.
I use the following -
-
I declare
__slots__in custom classes -
I use typing blocks for typing imports
-
I use builtins when possible
-
I try to reduce function calls
-
I use set lookups wherever possible
-
I prefer iteration to recursion
Edit: I am using a profiler, and benchmarks. I'm working on a library - an ASGI Api framework. The code is async. Its not darascience. Its neither compatible with pypy, nor with numba..
What else?
You can set the PYTHONOPTIMIZE environment variable if you really understand what you are doing.
# e.g.
# same as -O
export PYTHONOPTIMIZE=1
# same as -OO
export PYTHONOPTIMIZE=2
Reference: Python doc: Command line and environment
PYTHONOPTIMIZE
If this is set to a non-empty string it is equivalent to specifying the -O option. If set to an integer, it is equivalent to specifying -O multiple times.
But normally you should never do it !!!
Deestan's answer for another SO question "Best practice for Python Assert" is really great:
Asserts should be used to test conditions that should never happen.
The purpose is to crash early in the case of a corrupt program state.
Usually a django application uses many other libraries. When something critical happens, and those libraries believe that the application should crash immediately, they follow the best practice above and use asserts. You don't want to break that.
This works, though it is not particularly elegant.
python -O `which gunicorn` 'module.wsgi:application'