Both are true
- functions are objects: do a
dir(f)on a function to view its attributes - objects can be used as functions: just add
__call__(self, ...)method and use the object like a function.
In general things that can be called using a syntax like whatever(x, y, z) are called callables.
What the example is trying to show is that methods are just object attributes that are also callables. Just like you can write obj.x = 5, you can also write obj.f = some_function.
Videos
Both are true
- functions are objects: do a
dir(f)on a function to view its attributes - objects can be used as functions: just add
__call__(self, ...)method and use the object like a function.
In general things that can be called using a syntax like whatever(x, y, z) are called callables.
What the example is trying to show is that methods are just object attributes that are also callables. Just like you can write obj.x = 5, you can also write obj.f = some_function.
Yes, that example indeed shows that functions are object. You can assign an object to the callback attribute and this tries to show that the object you assign to callback can be a function.
class TimedEvent:
def __init__(self, endtime, callback):
self.endtime = endtime
self.callback = callback
What is missing to make it clear is the initialization. You could, for example, do this:
def print_current_time():
print(datetime.datetime.now().isoformat())
event = TimedEvent(endtime, print_current_time)
event.callback()
This will actually call print_current_time, because event.callback is print_current_time.
You are looking for the __call__ method. Function objects have that method:
>>> def foo(): pass
...
>>> foo.__call__
<method-wrapper '__call__' of function object at 0x106aafd70>
Not that the Python interpreter loop actually makes use of that method when encountering a Python function object; optimisations in the implementation jump straight to the contained bytecode in most cases.
But you can use that on your own custom class:
class Callable(object):
def __init__(self, name):
self.name = name
def __call__(self, greeting):
return '{}, {}!'.format(greeting, self.name)
Demo:
>>> class Callable(object):
... def __init__(self, name):
... self.name = name
... def __call__(self, greeting):
... return '{}, {}!'.format(greeting, self.name)
...
>>> Callable('World')('Hello')
'Hello, World!'
Python creates function objects for you when you use a def statement, or you use a lambda expression:
>>> def foo(): pass
...
>>> foo
<function foo at 0x106aafd70>
>>> lambda: None
<function <lambda> at 0x106d90668>
You can compare this to creating a string or an integer or a list using literal syntax:
listobject = [1, 'two']
The above creates 3 objects without ever calling a type, Python did that all for you based on the syntax used. The same applies to functions.
Creating one yourself can be a little more complex; you need to have a code object and reference to a global namespace, at the very least:
>>> function_type = type(lambda: None)
>>> function_type
<type 'function'>
>>> function_type(foo.__code__, globals(), 'bar')
<function bar at 0x106d906e0>
Here I created a function object by reusing the function type, taking the code object from the foo function; the function type is not a built-in name but the type really does exist and can be obtained by calling type() on an existing function instance.
I also passed in the global namespace of my interpreter, and a name; the latter is an optional argument; the name is otherwise taken from the code object.
One simple way to see this is to create a function in the Python interpreter def bar(x): return x + 1 and then use dir(bar) to see the various magic attributes including __class__.
Yes, python functions are full objects.
For another approach, objects are functions if they have a magic __call__() method.
You already extracted the code objects, now all you have to do is to create the functions objects using types.FuncitonType. It accepts:
- Code object
- Reference to the global module
- Name
from types import FunctionType
code_string = """\
def f1(p1, p2, p3):
print("a{}, b{}, c{}".format(p1, p2, p3))
def f2():
print("text")
"""
code = compile(code_string, "<string>", "exec")
f1_code_object = code.co_consts[0]
f2_code_object = code.co_consts[1]
f1 = FunctionType(f1_code_object, globals(), "f1")
f2 = FunctionType(f2_code_object, globals(), "f2")
print(f1)
print(f2)
f1(10, 20, 30)
f2()
output:
<function f1 at 0x1042a2980>
<function f2 at 0x1042a2a20>
a10, b20, c30
text
All compile() does is translate the Python source code (think contents of *.py files) into a compiled version (think *.pyc files). The source code must be syntactically correct, but need not be valid. In the case of your code_string, this means it has compiled the two def <name>: <body> statements into a binary form, but it has not executed those def statements to actually create the functions.
You need to execute the compiled code object in order to actually execute the def statement that creating the functions.
However, as norok2 mentions, simply executing code in arbitrary strings is a security risk. It can be done somewhat safer by controlling the environment the code is executed in. We can create an environment where the only function that exists is the print function:
code_string = """\
def f1(p1, p2, p3):
print("a{}, b{}, c{}".format(p1, p2, p3))
def f2():
print("text")
"""
code = compile(code_string, '<string>', 'exec')
builtins = {
'print': print,
}
code_globals = {'__builtins__': builtins}
exec(code, code_globals)
f1 = code_globals['f1']
f_two = code_globals['f2']
f1(1, 2, 3)
f_two()
Now it is much more difficult for the code to break out of the sandbox. For example, if you add import sys inside the code_string, you'll get an NameError: name '__import__' is not defined., and so on.
By careful construction of the available builtins, you can allow as much functionality as needed.
Of course, since we are no longer using the caller's global context to create the definitions, you need to extract the created functions out of the code_globals dictionary, as demonstrated above.
Why compile?
You generally use compile() when you want to execute the resulting code multiple times. That's not the case here. You will likely be executing the code exactly once to create the f1 and f2 functions. (Once created, they can be called many times, but they only need to be created once.) Therefore, we should use exec() instead of compile(), so we don't get (and accidentally keep) a reference to the intermediate code object:
code_string = """\
def f1(p1, p2, p3):
print("a{}, b{}, c{}".format(p1, p2, p3))
def f2():
print("text")
"""
builtins = {
'print': print,
}
code_globals = {'__builtins__': builtins}
exec(code_string, code_globals)
f1 = code_globals['f1']
f_two = code_globals['f2']
f1(1, 2, 3)
f_two()
Why code.co_consts is fragile
S.B shows how you can directly create functions from code.co_consts[...], and indirectly demonstrates how fragile this is.
Consider the OP's code:
f2 = code.co_consts[2]
compared with S.B's:
f2_code_object = code.co_consts[1]
Apparently, something changed (Python version? An edit to the code_string?) and f2 moved from index 2 to index 1. There is no obvious mapping from the indices in co_consts to the expected function names.
To drive this home, consider this more complex example, which highlights the differences between compiling the Python script and executing it:
from types import FunctionType
code_string = """\
if True == False:
print("Defining f1...")
def f1(p1, p2, p3):
print("a{}, b{}, c{}".format(p1, p2, p3))
else:
print("Defining alternate f1")
def f1(p1, p2, p3):
print("c{}, b{}, a{}".format(p3, p2, p1))
"""
builtins = {
'print': print,
}
code_globals = {'__builtins__': builtins}
print("Using compile() and code.co_consts")
code = compile(code_string, '<string>', 'exec')
for idx, item in enumerate(code.co_consts):
print(f" .co_consts[{idx}] = {item!r}")
f1 = FunctionType(code.co_consts[3], code_globals) # How do we know it is 3???
f1(1, 2, 3)
print()
print("Using exec():")
code_globals = {'__builtins__': builtins}
exec(code_string, code_globals)
f1 = code_globals['f1']
f1(1, 2, 3)
Now code_string will conditionally define the function f1 based on the execution of the code.
Output:
Using compile() and code.co_consts
.co_consts[0] = True
.co_consts[1] = False
.co_consts[2] = 'Defining f1...'
.co_consts[3] = <code object f1 at 0x000001A3B567FA30, file "<string>", line 3>
.co_consts[4] = 'Defining alternate f1'
.co_consts[5] = <code object f1 at 0x000001A3B56C8330, file "<string>", line 7>
.co_consts[6] = None
a1, b2, c3
Using exec():
Defining alternate f1
c3, b2, a1
Note that:
compile()- created two different
code object f1entities, - does not execute either
print()statement.
- created two different
exec()- actually evaluates the condition in the
ifstatement, - executes the
print()statement in one code path, - defines the function
"f1"from that code path only.
- actually evaluates the condition in the