Your code does indeed succeed in creating a shallow copy. This can be seen by inspecting the IDs of the two outer lists, and noting that they differ.
>>> id(l)
140505607684808
>>> id(x)
140505607684680
Or simply comparing using is:
>>> x is l
False
However, because it is a shallow copy rather than a deep copy, the corresponding elements of the list are the same object as each other:
>>> x[0] is l[0]
True
This gives you the behaviour that you observed when the sub-lists are appended to.
If in fact what you wanted was a deep copy, then you could use copy.deepcopy. In this case the sublists are also new objects, and can be appended to without affecting the originals.
>>> from copy import deepcopy
>>> l=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> xdeep = deepcopy(l)
>>> xdeep == l
True
>>> xdeep is l
False <==== A shallow copy does the same here
>>> xdeep[0] is l[0]
False <==== But THIS is different from with a shallow copy
>>> xdeep[0].append(10)
>>> print(l)
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> print(xdeep)
[[1, 2, 3, 10], [4, 5, 6], [7, 8, 9]]
If you wanted to apply this in your function, you could do:
from copy import deepcopy
def processed(matrix,r,i):
new_matrix = deepcopy(matrix)
new_matrix[r].append(i)
return new_matrix
l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
x = processed(l,0,10)
print(x)
print(l)
If in fact you know that the matrix is always exactly 2 deep, then you could do it more efficiently than using deepcopy and without need for the import:
def processed(matrix,r,i):
new_matrix = [sublist[:] for sublist in matrix]
new_matrix[r].append(i)
return new_matrix
l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
x = processed(l,0,10)
print(x)
print(l)
Answer from alani on Stack Overflowdeep copy of list in python - Stack Overflow
How can I make a deepcopy of a function in Python? - Stack Overflow
object oriented - Shouldn't deep copy be the default, not shallow copy? - Software Engineering Stack Exchange
Python copy.copy() appears to make a deep copy
I am following the python docs for copy functions to learn how to make a deep copy of a list or dictionary. (https://docs.python.org/3/library/copy.html) However, when I use copy.copy(list) and change one of the values it doesn't modify the original list like I thought.
Example
import copy
myL = ['first', 'second', 'third']
myLReg = copy.copy(myL)
myLReg[1] = 'new value'
console.log(myL)
What I expect the output to be is ['first', 'new value', 'third'] but I get ['first', 'second', 'third']
Am I understanding the docs wrong, is there a bug, or something else?
This is the behavior I expected from copy.deepcopy(myL)
Your code does indeed succeed in creating a shallow copy. This can be seen by inspecting the IDs of the two outer lists, and noting that they differ.
>>> id(l)
140505607684808
>>> id(x)
140505607684680
Or simply comparing using is:
>>> x is l
False
However, because it is a shallow copy rather than a deep copy, the corresponding elements of the list are the same object as each other:
>>> x[0] is l[0]
True
This gives you the behaviour that you observed when the sub-lists are appended to.
If in fact what you wanted was a deep copy, then you could use copy.deepcopy. In this case the sublists are also new objects, and can be appended to without affecting the originals.
>>> from copy import deepcopy
>>> l=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> xdeep = deepcopy(l)
>>> xdeep == l
True
>>> xdeep is l
False <==== A shallow copy does the same here
>>> xdeep[0] is l[0]
False <==== But THIS is different from with a shallow copy
>>> xdeep[0].append(10)
>>> print(l)
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> print(xdeep)
[[1, 2, 3, 10], [4, 5, 6], [7, 8, 9]]
If you wanted to apply this in your function, you could do:
from copy import deepcopy
def processed(matrix,r,i):
new_matrix = deepcopy(matrix)
new_matrix[r].append(i)
return new_matrix
l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
x = processed(l,0,10)
print(x)
print(l)
If in fact you know that the matrix is always exactly 2 deep, then you could do it more efficiently than using deepcopy and without need for the import:
def processed(matrix,r,i):
new_matrix = [sublist[:] for sublist in matrix]
new_matrix[r].append(i)
return new_matrix
l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
x = processed(l,0,10)
print(x)
print(l)
What you're looking for is a deeper copy than what you did. A shallow copy only replaces the top layer, which does not seem to be what you're looking for. If you wanted a different outcome, try something like this:
def processed(matrix,r,i):
matrix[r] = [*matrix[r], i]
return matrix
l=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
x=l[:]
print(processed(x,0,10))
print(l)
The difference is that this makes two shallow copies - first to copy the outer list, and then the function copies the inner list before modifying it. The downside of this approach is that every call to processed now has extra overhead. If you wanted to do the copying all at once, you can do this:
def processed(matrix,r,i):
matrix[r].append(i)
return matrix
l=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
x=[inner[:] for inner in l]
print(processed(x,0,10))
print(l)
This copies two layers deep, using list comprehensions. Your structure only has two layers, so this fully copies the list.
The FunctionType constructor is used to make a deep copy of a function.
import types
def copy_func(f, name=None):
return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
f.func_defaults, f.func_closure)
def A():
"""A"""
pass
B = copy_func(A, "B")
B.__doc__ = """B"""
My goal is to have two functions with the same implementation but with different docstrings.
Most users will do this, say the original function is in old_module.py:
def implementation(arg1, arg2):
"""this is a killer function"""
and in new_module.py
from old_module import implementation as _implementation
def implementation(arg1, arg2):
"""a different docstring"""
return _implementation(arg1, arg2)
This is the most straightforward way to reuse functionality. It is easy to read and understand the intent.
Nevertheless, perhaps you have a good reason for your main question:
How can I make a deepcopy of a function in Python?
To keep this compatible with Python 2 and 3, I recommend using the function's special __dunder__ attributes. For example:
import types
def copy_func(f, name=None):
'''
return a function with same code, globals, defaults, closure, and
name (or provide a new name)
'''
fn = types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
f.__defaults__, f.__closure__)
# in case f was given attrs (note this dict is a shallow copy):
fn.__dict__.update(f.__dict__)
return fn
And here's an example usage:
def main():
from logging import getLogger as _getLogger # pyflakes:ignore, must copy
getLogger = copy_func(_getLogger)
getLogger.__doc__ += '\n This function is from the Std Lib logging module.\n '
assert getLogger.__doc__ is not _getLogger.__doc__
assert getLogger.__doc__ != _getLogger.__doc__
A commenter says:
This canโt work for builtโin functions
Well I wouldn't do this for a built-in function. I have very little reason to do this for functions written in pure Python, and my suspicion is that if you are doing this, you're probably doing something very wrong (though I could be wrong here).
If you want a function that does what a builtin function does, and reuses the implementation, like a copy would, then you should wrap the function with another function, e.g.:
_sum = sum
def sum(iterable, start=0):
"""sum function that works like the regular sum function, but noisy"""
print('calling the sum function')
return _sum(iterable, start)
Let's say we have a Car that has four Wheels and an Owner. If we want to deep-copy the Car, we'd probably want to copy the Wheels, but it probably wouldn't make much sense to copy the Owner (and thus copy everything else hanging off the Owner, possibly including the Car itself).
If object A has a reference to object B, it may be because A "owns" B, and B should be included in a deep copy. But it may also be because A merely has some kind of association with B, and B and shouldn't be included in a deep copy. Unless the language provides a way to distinguish between these two types of references, we can't automate a deep copy, and the logic will have to be explicitly specified.
Without keeping a big table of all the objects you've copied so far, you can't safely deep copy objects at all if there may be circular references.
where every object always has a copy method
Is this true of everything? In which languages? Doesn't this mess with e.g. lock objects? Streams? Use of the RAII pattern? Handles to operating system objects? Intentional singletons? There's lots of cases where you may legitimately want to prevent objects being copied at all.
I'd also like to push back on
Shouldn't the default always be optimized for safety and not for performance? Isn't optimizing copy for speed premature optimization?
Deep copy can get arbitrarily expensive. You can't overlook that entirely.
But there are lots of different opinions about "safety". Quite a lot of functional programming and FP influence in modern languages moves towards immutability. Objects once created aren't changed. This is a more mathematical view; if you have f(x) = y then it is always true that f(x) = y, and none of f, x, or y are mutable.
If an object is immutable, there is no point in copying it. You can just pass around references to it. You only ever produce mutated copies, instead of changing values within the object. You start thinking more like value types: nobody asks how many different copies of "2" there are in their program, nor do they care about deep vs shallow copy of "2".
Then we consider the reverse operation: interning. Sometimes people ask how many copies of the string "hello" there are in their program, and would find it convenient if all references to "hello" returned the same object. https://en.wikipedia.org/wiki/String_interning ; this makes deep copy meaningless, because in that situation you only ever want one copy of the interned string.
(Incidentally, it turns out that Java does care about how many copies of "2" there are in your program if you start boxing them - all Int(2) are the same object. But not all Int(2000).)
I expect that when I later on pass the same copy to the same method, it will behave exactly the same but that isn't guaranteed, as the objects the array references may have changed, despite me having never changed them in my code
Immutability fixes this.
In Haskell, the default containers are immutable, and it's quite hard to write a mutable one. You don't add an item to a list, you create a new list containing the contents plus the new item. https://www.fpcomplete.com/haskell/library/containers/
Rust takes a different approach: every reference is annotated with ownership and mutability semantics. So you can call methods confident in the knowledge that they won't mutate particular objects.