You are creating a new list object each time by concatenating. This requires copying all elements from the old list into a new one, plus one extra. So yes, using l = l + [i] is an O(N) algorithm, not O(1).
At the very least, don't use + concatenation; use += augmented concatenation, which is the same thing as list.extend() with a re-assignment to the same reference:
def test3():
l = []
for i in range(1000):
l += [i] # or use l.extend([i])
return l
This produces:
>>> print timeit.repeat(stmt=test1, number=100, repeat=2)
[0.1333179473876953, 0.12804388999938965]
>>> print timeit.repeat(stmt=test2, number=100, repeat=2)
[0.01052403450012207, 0.007989168167114258]
>>> print timeit.repeat(stmt=test3, number=100, repeat=2)
[0.013209104537963867, 0.011193037033081055]
Answer from Martijn Pieters on Stack Overflowalgorithm - Complexity of creating list with concat operator in python - Stack Overflow
What is Python's list.append() method WORST Time Complexity? It can't be O(1), right?
python - Time complexity of appending a list to a list - Stack Overflow
Runtime of merging two lists in Python - Stack Overflow
The time complexity is not O(N)
The time complexity of the concat operation for two lists, A and B, is O(A + B). This is because you aren't adding to one list, but instead are creating a whole new list and populating it with elements from both A and B, requiring you to iterate through both.
Therefore, doing the operation l = l + [i] is O(len(l)), leaving you with N steps of doing an N operation, resulting in an overall complexity of O(N^2)
You are confusing concat with the append or extend function, which doesn't create a new list but adds to the original. If you used those functions, your time complexity would indeed be O(N)
An additional note:
The notation l = l + [i] can be confusing because intuitively it seems like [i] is simply being added to the existing l. This isn't true!
l + [i] builds a entirely new list and then has l point to that list.
On the other hand l += [i] modifies the original list and behaves like extend
Here is my thought process: I know that the concat operator is O(k) where k is the size of the list being added to the original list. Since the size of k is always 1 in this case because we are adding one character lists at a time, the concat operation takes 1 step.
This assumption is incorrect. If you write:
l + [i]
you construct a new list, this list will have m+1 elements, with m the number of elements in l, given a list is implemented like an array, we know that constructing such list will take O(m) time. We then assign the new list to l.
So that means that the total number of steps is:
n
---
\ 2
/ O(m) = O(n )
---
m=0
so the time complexity is O(n2).
You can however boost performance, by using l += [i], or even faster l.append(i), where the amortize cost is, for both l += [i] and l.append(i) O(1), so then the algorithm is O(n), the l.append(i) will however likely be a bit faster because we save on constructing a new list, etc.
I know that lists in Python are implemented using arrays that store addresses to the information. Therefore, after several appends, when an array is loaded, it needs to reserve a new space and copy the entire array of addresses to the new place.
I've read on Stackoverflow that in Python Array doubles in size when run of space. So basically it has to copy all addresses log(n) times.
The bigger the list, the more copying it will need to do. So how can append operation have a Constant Time Complexity O(1) if it has some dependence on the array size
I assume since it copies addresses, not information, it shouldn't take long, python takes only 8 bytes for address after all. Moreover, it does so very rarely. Does that mean that O(1) is the average time complexity? Is my assumption right?
Referencing my comment again, "appending" a list to a list is still O(1), you are just adding the reference to the list, "extending" a list of size m is going to cost you O(m), since it has to traverse the other list of size m to append every single element.
seq = [1, 2, 3]
new_seq = [4, 5, 6]
# Appending is O(1)
seq.append(new_seq)
print(seq) # [1, 2, 3, [4, 5, 6]]
seq = [1, 2, 3]
new_seq = [4, 5, 6]
# Extending is O(m), m = len(new_seq)
seq.extend(new_seq)
print(seq) #[1, 2, 3, 4, 5, 6]
list_ = []
list_.append([_ for _ in range(0,n)])
You take O(n) time to create the list [_ for _ in range(0,n)] then O(1) time to add the reference of this list to your list_. O(n) in total for the line list_.append([_ for _ in range(0,n)]).
list.append(item) is O(1), because lists are random-access. list.extend(other_list) is O(k), where k is the size of the other_list, presumably because memcpy is also a linear-time operation.
When you concatenate the two lists with A + B, you create a completely new list in memory. This means your guess is correct: the complexity is O(n + m) (where n and m are the lengths of the lists) since Python has to walk both lists in turn to build the new list.
You can see this happening in the list_concat function in the source code for Python lists:
static PyObject *
list_concat(PyListObject *a, PyObject *bb)
{
/* ...code snipped... */
src = a->ob_item;
dest = np->ob_item;
for (i = 0; i < Py_SIZE(a); i++) { /* walking list a */
PyObject *v = src[i];
Py_INCREF(v);
dest[i] = v;
}
src = b->ob_item;
dest = np->ob_item + Py_SIZE(a);
for (i = 0; i < Py_SIZE(b); i++) { /* walking list b */
PyObject *v = src[i];
Py_INCREF(v);
dest[i] = v;
}
/* ...code snipped... */
If you don't need a new list in memory, it's often a good idea to take advantage of the fact that lists are mutable (and this is where Python is smart). Using A.extend(B) is O(m) in complexity meaning that you avoid the overhead of copying list a.
The complexity of various list operations are listed here on the Python wiki.
My first guess is
O(n+m), not sure if Python is smarter than that.
Nothing can be smarter than that while returning a copy. Though even if A, B were immutable sequences such as strings; CPython still makes a full copy instead of aliasing the same memory (it simplifies implementation of the garbage collection for such strings).
In some specific cases, the operation could be O(1) depending on what you want to do with the result e.g., itertools.chain(A, B) allows to iterate over all items (it does not make a copy, the change in A, B affects yielded items). Or if you need a random access; you could emulate it using a Sequence subclass e.g., WeightedPopulation but in the general case the copy and therefore O(n+m) runtime is unavoidable.