Some answers claimed a "10x" speed advantage for deque vs list-used-as-FIFO when both have 1000 entries, but that's a bit of an overbid:
$ python -mtimeit -s'q=range(1000)' 'q.append(23); q.pop(0)'
1000000 loops, best of 3: 1.24 usec per loop
$ python -mtimeit -s'import collections; q=collections.deque(range(1000))' 'q.append(23); q.popleft()'
1000000 loops, best of 3: 0.573 usec per loop
python -mtimeit is your friend -- a really useful and simple micro-benchmarking approach! With it you can of course also trivially explore performance in much-smaller cases:
$ python -mtimeit -s'q=range(100)' 'q.append(23); q.pop(0)'
1000000 loops, best of 3: 0.972 usec per loop
$ python -mtimeit -s'import collections; q=collections.deque(range(100))' 'q.append(23); q.popleft()'
1000000 loops, best of 3: 0.576 usec per loop
(not very different for 12 instead of 100 items btw), and in much-larger ones:
$ python -mtimeit -s'q=range(10000)' 'q.append(23); q.pop(0)'
100000 loops, best of 3: 5.81 usec per loop
$ python -mtimeit -s'import collections; q=collections.deque(range(10000))' 'q.append(23); q.popleft()'
1000000 loops, best of 3: 0.574 usec per loop
You can see that the claim of O(1) performance for deque is well founded, while a list is over twice as slow around 1,000 items, an order of magnitude around 10,000. You can also see that even in such cases you're only wasting 5 microseconds or so per append/pop pair and decide how significant that wastage is (though if that's all you're doing with that container, deque has no downside, so you might as well switch even if 5 usec more or less won't make an important difference).
Answer from Alex Martelli on Stack OverflowNormal Queue vs Lists?
Implementing an efficient queue in Python - Stack Overflow
Is it possible to convert list to queue in python? - Stack Overflow
Initialize queue from an existing list - Ideas - Discussions on Python.org
Videos
Some answers claimed a "10x" speed advantage for deque vs list-used-as-FIFO when both have 1000 entries, but that's a bit of an overbid:
$ python -mtimeit -s'q=range(1000)' 'q.append(23); q.pop(0)'
1000000 loops, best of 3: 1.24 usec per loop
$ python -mtimeit -s'import collections; q=collections.deque(range(1000))' 'q.append(23); q.popleft()'
1000000 loops, best of 3: 0.573 usec per loop
python -mtimeit is your friend -- a really useful and simple micro-benchmarking approach! With it you can of course also trivially explore performance in much-smaller cases:
$ python -mtimeit -s'q=range(100)' 'q.append(23); q.pop(0)'
1000000 loops, best of 3: 0.972 usec per loop
$ python -mtimeit -s'import collections; q=collections.deque(range(100))' 'q.append(23); q.popleft()'
1000000 loops, best of 3: 0.576 usec per loop
(not very different for 12 instead of 100 items btw), and in much-larger ones:
$ python -mtimeit -s'q=range(10000)' 'q.append(23); q.pop(0)'
100000 loops, best of 3: 5.81 usec per loop
$ python -mtimeit -s'import collections; q=collections.deque(range(10000))' 'q.append(23); q.popleft()'
1000000 loops, best of 3: 0.574 usec per loop
You can see that the claim of O(1) performance for deque is well founded, while a list is over twice as slow around 1,000 items, an order of magnitude around 10,000. You can also see that even in such cases you're only wasting 5 microseconds or so per append/pop pair and decide how significant that wastage is (though if that's all you're doing with that container, deque has no downside, so you might as well switch even if 5 usec more or less won't make an important difference).
You won't run out of memory using the list implementation, but performance will be poor. From the docs:
Though
listobjects support similar operations, they are optimized for fast fixed-length operations and incur O(n) memory movement costs forpop(0)andinsert(0, v)operations which change both the size and position of the underlying data representation.
So using a deque will be much faster.
Reading the code on a Queue
queue.Queue
This just seems like a fancy list initialized as a dequeue which only .get()s the first item of a list.
As I understand, both list.pop(0) and dequeue.popleft() operate in O(1).
The only difference that I notice is that when you pop(0) a list, it needs to move all the memory "to the side 1 space" so to speak. Is this the only real difference between dequeues and lists?
Should I just use a list as a FIFO, or is a list better for LIFO\Stack?
As Uri Goren astutely noted above, the Python stdlib already implemented an efficient queue on your fortunate behalf: collections.deque.
What Not to Do
Avoid reinventing the wheel by hand-rolling your own:
- Linked list implementation. While doing so reduces the worst-case time complexity of your
dequeue()andenqueue()methods to O(1), thecollections.dequetype already does so. It's also thread-safe and presumably more space and time efficient, given its C-based heritage. - Python list implementation. As I note below, implementing the
enqueue()methods in terms of a Python list increases its worst-case time complexity to O(n). Since removing the last item from a C-based array and hence Python list is a constant-time operation, implementing thedequeue()method in terms of a Python list retains the same worst-case time complexity of O(1). But who cares?enqueue()remains pitifully slow.
To quote the official deque documentation:
Though
listobjects support similar operations, they are optimized for fast fixed-length operations and incur O(n) memory movement costs forpop(0)andinsert(0, v)operations which change both the size and position of the underlying data representation.
More critically, deque also provides out-of-the-box support for a maximum length via the maxlen parameter passed at initialization time, obviating the need for manual attempts to limit the queue size (which inevitably breaks thread safety due to race conditions implicit in if conditionals).
What to Do
Instead, implement your Queue class in terms of the standard collections.deque type as follows:
from collections import deque
class Queue:
'''
Thread-safe, memory-efficient, maximally-sized queue supporting queueing and
dequeueing in worst-case O(1) time.
'''
def __init__(self, max_size = 10):
'''
Initialize this queue to the empty queue.
Parameters
----------
max_size : int
Maximum number of items contained in this queue. Defaults to 10.
'''
self._queue = deque(maxlen=max_size)
def enqueue(self, item):
'''
Queues the passed item (i.e., pushes this item onto the tail of this
queue).
If this queue is already full, the item at the head of this queue
is silently removed from this queue *before* the passed item is
queued.
'''
self._queue.append(item)
def dequeue(self):
'''
Dequeues (i.e., removes) the item at the head of this queue *and*
returns this item.
Raises
----------
IndexError
If this queue is empty.
'''
return self._queue.pop()
The proof is in the hellish pudding:
>>> queue = Queue()
>>> queue.enqueue('Maiden in Black')
>>> queue.enqueue('Maneater')
>>> queue.enqueue('Maiden Astraea')
>>> queue.enqueue('Flamelurker')
>>> print(queue.dequeue())
Flamelurker
>>> print(queue.dequeue())
Maiden Astraea
>>> print(queue.dequeue())
Maneater
>>> print(queue.dequeue())
Maiden in Black
It Is Dangerous to Go Alone
Actually, don't do that either.
You're better off just using a raw deque object rather than attempting to manually encapsulate that object in a Queue wrapper. The Queue class defined above is given only as a trivial demonstration of the general-purpose utility of the deque API.
The deque class provides significantly more features, including:
...iteration, pickling,
len(d),reversed(d),copy.copy(d),copy.deepcopy(d), membership testing with the in operator, and subscript references such asd[-1].
Just use deque anywhere a single- or double-ended queue is required. That is all.
You can keep head and tail node instead of a queue list in queue class
class Node:
def __init__(self, item = None):
self.item = item
self.next = None
self.previous = None
class Queue:
def __init__(self):
self.length = 0
self.head = None
self.tail = None
def enqueue(self, value):
newNode = Node(value)
if self.head is None:
self.head = self.tail = newNode
else:
self.tail.next = newNode
newNode.previous = self.tail
self.tail = newNode
self.length += 1
def dequeue(self):
item = self.head.item
self.head = self.head.next
self.length -= 1
if self.length == 0:
self.tail = None
return item
pop from the front of a list is not very efficient as all the references in the list need to be updated.
deque will allow you do queue like operations efficiently
>>> from collections import deque
>>> deque([1,2,3,4])
deque([1, 2, 3, 4])
Since I was looking for an answer to this question while using queue.Queue, I thought I should share my findings. It is possible to convert a list into a queue using queue.queue.
import queue
l = [i for i in range(1000)]
q = queue.Queue()
[q.put(i) for i in l]
q2 = queue.Queue()
q2.queue = queue.deque(l)
After this code has been run, q and q2 are two different queues that contain the exact same entries, but with the second method being >300 times faster on my machine.
Not related to the question, but the opposite can be done by l = list(q.queue) if q is an instance of queue.Queue. Hope this saves you some trouble!
This was all tested in python 3.5.2.