There is a big difference between exec in Python 2 and exec() in Python 3. You are treating exec as a function, but it really is a statement in Python 2.
Because of this difference, you cannot change local variables in function scope in Python 3 using exec, even though it was possible in Python 2. Not even previously declared variables.
locals() only reflects local variables in one direction. The following never worked in either 2 or 3:
def foo():
a = 'spam'
locals()['a'] = 'ham'
print(a) # prints 'spam'
In Python 2, using the exec statement meant the compiler knew to switch off the local scope optimizations (switching from LOAD_FAST to LOAD_NAME for example, to look up variables in both the local and global scopes). With exec() being a function, that option is no longer available and function scopes are now always optimized.
Moreover, in Python 2, the exec statement explicitly copies all variables found in locals() back to the function locals using PyFrame_LocalsToFast, but only if no globals and locals parameters were supplied.
The proper work-around is to use a new namespace (a dictionary) for your exec() call:
def execute(a, st):
namespace = {}
exec("b = {}\nprint('b:', b)".format(st), namespace)
print(namespace['b'])
The exec() documentation is very explicit about this limitation:
Answer from Martijn Pieters on Stack OverflowNote: The default locals act as described for function
locals()below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after functionexec()returns.
Behavior of exec function in Python 2 and Python 3 - Stack Overflow
How can I use exec in functions in python3.6 similar to python2.7?
Using a function defined in an exec'ed string in Python 3 - Stack Overflow
Why is it advised to not use exec?
Videos
There is a big difference between exec in Python 2 and exec() in Python 3. You are treating exec as a function, but it really is a statement in Python 2.
Because of this difference, you cannot change local variables in function scope in Python 3 using exec, even though it was possible in Python 2. Not even previously declared variables.
locals() only reflects local variables in one direction. The following never worked in either 2 or 3:
def foo():
a = 'spam'
locals()['a'] = 'ham'
print(a) # prints 'spam'
In Python 2, using the exec statement meant the compiler knew to switch off the local scope optimizations (switching from LOAD_FAST to LOAD_NAME for example, to look up variables in both the local and global scopes). With exec() being a function, that option is no longer available and function scopes are now always optimized.
Moreover, in Python 2, the exec statement explicitly copies all variables found in locals() back to the function locals using PyFrame_LocalsToFast, but only if no globals and locals parameters were supplied.
The proper work-around is to use a new namespace (a dictionary) for your exec() call:
def execute(a, st):
namespace = {}
exec("b = {}\nprint('b:', b)".format(st), namespace)
print(namespace['b'])
The exec() documentation is very explicit about this limitation:
Note: The default locals act as described for function
locals()below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after functionexec()returns.
I'd say it's a bug of python3.
def u():
exec("a=2")
print(locals()['a'])
u()
prints "2".
def u():
exec("a=2")
a=2
print(a)
u()
prints "2".
But
def u():
exec("a=2")
print(locals()['a'])
a=2
u()
fails with
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in u
KeyError: 'a'
--- EDIT --- Another interesting behaviour:
def u():
a=1
l=locals()
exec("a=2")
print(l)
u()
def u():
a=1
l=locals()
exec("a=2")
locals()
print(l)
u()
outputs
{'l': {...}, 'a': 2}
{'l': {...}, 'a': 1}
And also
def u():
l=locals()
exec("a=2")
print(l)
print(locals())
u()
def u():
l=locals()
exec("a=2")
print(l)
print(locals())
a=1
u()
outputs
{'l': {...}, 'a': 2}
{'l': {...}, 'a': 2}
{'l': {...}, 'a': 2}
{'l': {...}}
Apparently, the action of exec on locals is the following:
- If a variable is set within
execand this variable was a local variable, thenexecmodifies the internal dictionary (the one returned bylocals()) and does not return it to its original state. A call tolocals()updates the dictionary (as documented in section 2 of python documentation), and the value set withinexecis forgotten. The need of callinglocals()to update the dictionary is not a bug of python3, because it is documented, but it is not intuitive. Moreover, the fact that modifications of locals withinexecdon't change the locals of the function is a documented difference with python2 (the documentation says "Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns"), and I prefer the behaviour of python2. - If a variable is set within
execand this variable did not exist before, thenexecmodifies the internal dictionary unless the variable is set afterwards. It seems that there is a bug in the waylocals()updates the dictionary ; this bug gives access to the value set withinexecby callinglocals()afterexec.
Note: exec was just a Simple statement in Python 2.x, whereas it is a function in Python 3.x.
Python 2.7
Let us check the changes made by executing a.
class Test:
def __init__(self):
l, g = locals().copy(), globals().copy()
exec a # NOT a function call but a statement
print locals() == l, globals() == g
x()
t = Test()
Output
False True
42
It means that, it has changed something in the locals dictionary. If you print locals().keys() before and after the exec, you will see x, after exec. As per the documentation of exec,
In all cases, if the optional parts are omitted, the code is executed in the current scope.
So, it does exactly what the documentation says.
Python 3.x:
When we execute the same in Python 3.x, we get similar result, except we get that error.
class Test:
def __init__(self):
l, g = locals().copy(), globals().copy()
exec(a) # Function call, NOT a statement
print(locals() == l, globals() == g)
x()
Output
False True
NameError: name 'x' is not defined
Even the documentation of exec function says,
In all cases, if the optional parts are omitted, the code is executed in the current scope.
But it also includes a note at the bottom,
Note: The default locals act as described for function
locals()below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after functionexec()returns.
So, we curiously check the locals() documentation and find
Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
So, interpreter doesn't honor the changes made to the locals() object. That is why it is not recognizing x as defined in the local scope.
But when we do
def __init__(self):
exec(a, globals())
x()
it works, because we add it to the globals dictionary. Python tries to find x in local scope first and then in class scope and then in global scope and it finds it there. So it executes it without any problem.
Using exec(f, globals()) from a function (2.x and 3.x python compatible)
NOTE: If you don't like to pass globals() every time you can refactor you code by wrapping `exec(a, globals())` into a function and using it across modules, but it will require some additional coding, because globals() is per-module and you need those globals() which are from the calling module:
# macros.py
import sys
def execfunc(func_as_str):
print("(debug) calling module: %s" % sys._getframe(1).f_code.co_filename)
# See https://github.com/python/cpython/blob/main/Lib/traceback.py
exec(func_as_str, sys._getframe(1).f_globals)
# main script
import macros
macros.execfunc("def foo(x):\n print(x)")
foo('bar')
Python3 exec also takes globals and locals optional arguments of mapping type, which act as a context for the given code execution:
exec(object[, globals[, locals]])
By default the local scope gets passed in for both. The executed code can use it, can also modify the dict, but it will have no effect on the actual local scope. See locals() and example:
a = '''
print(t)
def x():
print(42)
'''
class Test:
def __init__(self):
t = 'locals are accessible'
# same as calling exec(a, locals())
exec(a)
print(locals())
x()
t = Test()
Output:
locals are accessible
{'x': <function x at 0x6ffffd09af0>,
'self': <__main__.Test object at 0x6ffffce3f90>,
't': 'locals are accessible'}
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 13, in __init__
NameError: global name 'x' is not defined
If you want the x to be available after the exec call you either need to pass in a global or a custom scope:
# global scope
class Test:
def __init__(self):
exec(a, globals())
x()
# custom scope
class Test:
def __init__(self):
scope = {}
exec(a, scope)
scope['x']()
I asked a question a few days ago on stack overflow about doing exec("func = lambda x:x**2") (or dynamically generating other functions for that matter) and someone said that I should not use exec in my code and instead create a function with def that returns a lambda function.
And it's something that I see in quite a few forums.