You are creating a new list object each time by concatenating. This requires copying all elements from the old list into a new one, plus one extra. So yes, using l = l + [i] is an O(N) algorithm, not O(1).
At the very least, don't use + concatenation; use += augmented concatenation, which is the same thing as list.extend() with a re-assignment to the same reference:
def test3():
l = []
for i in range(1000):
l += [i] # or use l.extend([i])
return l
This produces:
>>> print timeit.repeat(stmt=test1, number=100, repeat=2)
[0.1333179473876953, 0.12804388999938965]
>>> print timeit.repeat(stmt=test2, number=100, repeat=2)
[0.01052403450012207, 0.007989168167114258]
>>> print timeit.repeat(stmt=test3, number=100, repeat=2)
[0.013209104537963867, 0.011193037033081055]
Answer from Martijn Pieters on Stack OverflowThe time complexity is not O(N)
The time complexity of the concat operation for two lists, A and B, is O(A + B). This is because you aren't adding to one list, but instead are creating a whole new list and populating it with elements from both A and B, requiring you to iterate through both.
Therefore, doing the operation l = l + [i] is O(len(l)), leaving you with N steps of doing an N operation, resulting in an overall complexity of O(N^2)
You are confusing concat with the append or extend function, which doesn't create a new list but adds to the original. If you used those functions, your time complexity would indeed be O(N)
An additional note:
The notation l = l + [i] can be confusing because intuitively it seems like [i] is simply being added to the existing l. This isn't true!
l + [i] builds a entirely new list and then has l point to that list.
On the other hand l += [i] modifies the original list and behaves like extend
Here is my thought process: I know that the concat operator is O(k) where k is the size of the list being added to the original list. Since the size of k is always 1 in this case because we are adding one character lists at a time, the concat operation takes 1 step.
This assumption is incorrect. If you write:
l + [i]
you construct a new list, this list will have m+1 elements, with m the number of elements in l, given a list is implemented like an array, we know that constructing such list will take O(m) time. We then assign the new list to l.
So that means that the total number of steps is:
n
---
\ 2
/ O(m) = O(n )
---
m=0
so the time complexity is O(n2).
You can however boost performance, by using l += [i], or even faster l.append(i), where the amortize cost is, for both l += [i] and l.append(i) O(1), so then the algorithm is O(n), the l.append(i) will however likely be a bit faster because we save on constructing a new list, etc.
Runtime of merging two lists in Python - Stack Overflow
python - Is the time-complexity of iterative string append actually O(n^2), or O(n)? - Stack Overflow
python - Time complexity of appending a list to a list - Stack Overflow
python - Improve speed for list concatenation? - Stack Overflow
When you concatenate the two lists with A + B, you create a completely new list in memory. This means your guess is correct: the complexity is O(n + m) (where n and m are the lengths of the lists) since Python has to walk both lists in turn to build the new list.
You can see this happening in the list_concat function in the source code for Python lists:
static PyObject *
list_concat(PyListObject *a, PyObject *bb)
{
/* ...code snipped... */
src = a->ob_item;
dest = np->ob_item;
for (i = 0; i < Py_SIZE(a); i++) { /* walking list a */
PyObject *v = src[i];
Py_INCREF(v);
dest[i] = v;
}
src = b->ob_item;
dest = np->ob_item + Py_SIZE(a);
for (i = 0; i < Py_SIZE(b); i++) { /* walking list b */
PyObject *v = src[i];
Py_INCREF(v);
dest[i] = v;
}
/* ...code snipped... */
If you don't need a new list in memory, it's often a good idea to take advantage of the fact that lists are mutable (and this is where Python is smart). Using A.extend(B) is O(m) in complexity meaning that you avoid the overhead of copying list a.
The complexity of various list operations are listed here on the Python wiki.
My first guess is
O(n+m), not sure if Python is smarter than that.
Nothing can be smarter than that while returning a copy. Though even if A, B were immutable sequences such as strings; CPython still makes a full copy instead of aliasing the same memory (it simplifies implementation of the garbage collection for such strings).
In some specific cases, the operation could be O(1) depending on what you want to do with the result e.g., itertools.chain(A, B) allows to iterate over all items (it does not make a copy, the change in A, B affects yielded items). Or if you need a random access; you could emulate it using a Sequence subclass e.g., WeightedPopulation but in the general case the copy and therefore O(n+m) runtime is unavoidable.
In CPython, the standard implementation of Python, there's an implementation detail that makes this usually O(n), implemented in the code the bytecode evaluation loop calls for + or += with two string operands. If Python detects that the left argument has no other references, it calls realloc to attempt to avoid a copy by resizing the string in place. This is not something you should ever rely on, because it's an implementation detail and because if realloc ends up needing to move the string frequently, performance degrades to O(n^2) anyway.
Without the weird implementation detail, the algorithm is O(n^2) due to the quadratic amount of copying involved. Code like this would only make sense in a language with mutable strings, like C++, and even in C++ you'd want to use +=.
The author relies on an optimization that happens to be here, but is not explicitly dependable. strA = strB + strC is typically O(n), making the function O(n^2). However, it is pretty easy to make sure it the whole process is O(n), use an array:
output = []
# ... loop thing
output.append('%20')
# ...
output.append(char)
# ...
return ''.join(output)
In a nutshell, the append operation is amortized O(1), (although you can make it strong O(1) by pre-allocating the array to the right size), making the loop O(n).
And then the join is also O(n), but that's okay because it is outside the loop.
Referencing my comment again, "appending" a list to a list is still O(1), you are just adding the reference to the list, "extending" a list of size m is going to cost you O(m), since it has to traverse the other list of size m to append every single element.
seq = [1, 2, 3]
new_seq = [4, 5, 6]
# Appending is O(1)
seq.append(new_seq)
print(seq) # [1, 2, 3, [4, 5, 6]]
seq = [1, 2, 3]
new_seq = [4, 5, 6]
# Extending is O(m), m = len(new_seq)
seq.extend(new_seq)
print(seq) #[1, 2, 3, 4, 5, 6]
list_ = []
list_.append([_ for _ in range(0,n)])
You take O(n) time to create the list [_ for _ in range(0,n)] then O(1) time to add the reference of this list to your list_. O(n) in total for the line list_.append([_ for _ in range(0,n)]).
list.append(item) is O(1), because lists are random-access. list.extend(other_list) is O(k), where k is the size of the other_list, presumably because memcpy is also a linear-time operation.
In a performance oriented code, it is not a good idea to add 2 strings together, it is preferable to use a "".join(_items2join_) instead. (I found some benchmarks there : http://www.skymind.com/~ocrow/python_string/)
Since accessing an element in a python list is O(1), and appending a list to another is O(1) (which is probably the time complexity of concatenating strings in python), The code you have provided is running as fast as it can as far as I can tell. :) You probably can't afford to do this, but when I need speed I go to C++ or some other compiled language when I need to process that much information. Things run much quicker. For time complexity of list operations in python, you may consult this web site: http://wiki.python.org/moin/TimeComplexity and here: What is the runtime complexity of python list functions?