Brave Search

Learning about Queue module in python (how to run it)

stackoverflow.com › questions › 14585597 › learning-about-queue-module-in-python-how-to-run-it

The for loop is launching a number of worker threads to perform the function defined by "worker". Here is working code that should run on your system in python 2.7.

import Queue
import threading

# input queue to be processed by many threads
q_in = Queue.Queue(maxsize=0)

# output queue to be processed by one thread
q_out = Queue.Queue(maxsize=0)

# number of worker threads to complete the processing
num_worker_threads = 10

# process that each worker thread will execute until the Queue is empty
def worker():
    while True:
        # get item from queue, do work on it, let queue know processing is done for one item
        item = q_in.get()
        q_out.put(do_work(item))
        q_in.task_done()

# squares a number and returns the number and its square
def do_work(item):
    return (item,item*item)

# another queued thread we will use to print output
def printer():
    while True:
        # get an item processed by worker threads and print the result. Let queue know item has been processed
        item = q_out.get()
        print "%d squared is : %d" % item
        q_out.task_done()

# launch all of our queued processes
def main():
    # Launches a number of worker threads to perform operations using the queue of inputs
    for i in range(num_worker_threads):
         t = threading.Thread(target=worker)
         t.daemon = True
         t.start()

    # launches a single "printer" thread to output the result (makes things neater)
    t = threading.Thread(target=printer)
    t.daemon = True
    t.start()

    # put items on the input queue (numbers to be squared)
    for item in range(10):
        q_in.put(item)

    # wait for two queues to be emptied (and workers to close)   
    q_in.join()       # block until all tasks are done
    q_out.join()

    print "Processing Complete"

main()

Python 3 version per @handle

import queue 
import threading

# input queue to be processed by many threads
q_in = queue.Queue(maxsize=0) 

# output queue to be processed by one thread
q_out = queue.Queue(maxsize=0) 

# number of worker threads to complete the processing
num_worker_threads = 10

# process that each worker thread will execute until the Queue is empty
def worker():
    while True:
        # get item from queue, do work on it, let queue know processing is done for one item
        item = q_in.get()
        q_out.put(do_work(item))
        q_in.task_done()

# squares a number and returns the number and its square
def do_work(item):
    return (item,item*item)

# another queued thread we will use to print output
def printer():
    while True:
        # get an item processed by worker threads and print the result. Let queue know item has been processed
        item = q_out.get()
        print("{0[0]} squared is : {0[1]}".format(item) )
        q_out.task_done()

# launch all of our queued processes
def main():
    # Launches a number of worker threads to perform operations using the queue of inputs
    for i in range(num_worker_threads):
         t = threading.Thread(target=worker)
         t.daemon = True
         t.start()

    # launches a single "printer" thread to output the result (makes things neater)
    t = threading.Thread(target=printer)
    t.daemon = True
    t.start()

    # put items on the input queue (numbers to be squared)
    for item in range(10):
        q_in.put(item)

    # wait for two queues to be emptied (and workers to close)   
    q_in.join()       # block until all tasks are done
    q_out.join()

    print( "Processing Complete" )

main()

Answer from Paul Seeb on Stack Overflow

Python

docs.python.org › 3 › library › queue.html

queue — A synchronized queue class

February 23, 2026 - The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks. Example of how to wait for enqueued tasks to be completed:

Python

docs.python.org › 2 › library › queue.html

8.10. Queue — A synchronized queue class — Python 2.7.18 documentation

The count of unfinished tasks goes ... to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks. New in version 2.5. Example of how to ...

Discussions

multithreading - Python 2.7 threading queue, reuse threads - Stack Overflow

I am looking for a way to have a background thread queue run for an unlimited amount of times. The code below is what I came up with from research, but I am being limited to the amount of threads I More on stackoverflow.com

stackoverflow.com

multithreading - Learning about Queue module in python (how to run it) - Stack Overflow

Was recently introduced to the queue design in regards to ability to defer processing as well as implementing a "FIFO" etc. Looked through the documentation in attempt to get a sample queue going... More on stackoverflow.com

stackoverflow.com

python - Queue between Python2 and Python3 - Stack Overflow

Unfortunately changing the existing solution to something new is too much work. It is surprising that compatibility mode between between Python 2 and Python 3 is not available. Should not be too difficult, because the same code using queue works exactly the same way in both pythons. More on stackoverflow.com

stackoverflow.com

January 20, 2019

Huey task queue 2.0

Big fan of Huey, thanks for all the work! More on reddit.com

r/Python

April 2, 2019

Videos

youtube.com

#13. Класс Queue модуля queue | The Python Standard Library

18:47

YouTube

Queues in Python Explained [ Step-by-Step Guide to Data Structures ...

July 17, 2025

26:12

YouTube

Working with Queues in Python - An introductory guide - YouTube

March 28, 2024

15:23

YouTube

Implement Queue using Stacks - Leetcode 232 - Python - YouTube

#15. Делаем очередь (queue) | Алгоритмы на ...

geeksforgeeks.org › python › queue-in-python

Queue in Python - GeeksforGeeks

December 11, 2025 - Explanation: popleft() efficiently removes the first element without shifting, making deque ideal for queues. Python’s queue module provides a thread-safe FIFO queue. You can specify a maxsize.

Real Python

realpython.com › ref › stdlib › queue

queue | Python Standard Library – Real Python

import logging import queue import threading import time logging.basicConfig(level=logging.INFO, format="%(threadName)s %(message)s") def worker(tasks): while True: task = tasks.get() if task is None: tasks.task_done() break logging.info(f"processing {task}") time.sleep(1) tasks.task_done() tasks = queue.Queue() for task in ["task1", "task2", "task3", "task4", "task5"]: tasks.put(task) num_workers = 2 for _ in range(num_workers): tasks.put(None) threads = [ threading.Thread(target=worker, args=(tasks,)) for _ in range(num_workers) ] for thread in threads: thread.start() tasks.join() for thread

Python Module of the Week

pymotw.com › 2 › Queue

Queue – A thread-safe FIFO implementation - Python Module of the Week

It is simplistic and unsuitable for actual use, but the skeleton implementation gives us enough code to work with to provide an example of using the Queue module. # System modules from Queue import Queue from threading import Thread import time # Local modules import feedparser # Set up some global variables num_fetch_threads = 2 enclosure_queue = Queue() # A real app wouldn't use hard-coded data...

Stack Overflow

stackoverflow.com › questions › 35442782 › python-2-7-threading-queue-reuse-threads

multithreading - Python 2.7 threading queue, reuse threads - Stack Overflow

import urllib2, time from threading import Thread from Queue import Queue # Set up global variables num_threads = 5 queue = Queue() urlList = [ "http://google.com", "http://googleeeeeee1111.com" ] def notify(i, q): print "Thread %s: started" ...

Stack Overflow

stackoverflow.com › questions › 14585597 › learning-about-queue-module-in-python-how-to-run-it

multithreading - Learning about Queue module in python (how to run it) - Stack Overflow

Top answer

1 of 2

The for loop is launching a number of worker threads to perform the function defined by "worker". Here is working code that should run on your system in python 2.7.

import Queue
import threading

# input queue to be processed by many threads
q_in = Queue.Queue(maxsize=0)

# output queue to be processed by one thread
q_out = Queue.Queue(maxsize=0)

# number of worker threads to complete the processing
num_worker_threads = 10

# process that each worker thread will execute until the Queue is empty
def worker():
    while True:
        # get item from queue, do work on it, let queue know processing is done for one item
        item = q_in.get()
        q_out.put(do_work(item))
        q_in.task_done()

# squares a number and returns the number and its square
def do_work(item):
    return (item,item*item)

# another queued thread we will use to print output
def printer():
    while True:
        # get an item processed by worker threads and print the result. Let queue know item has been processed
        item = q_out.get()
        print "%d squared is : %d" % item
        q_out.task_done()

# launch all of our queued processes
def main():
    # Launches a number of worker threads to perform operations using the queue of inputs
    for i in range(num_worker_threads):
         t = threading.Thread(target=worker)
         t.daemon = True
         t.start()

    # launches a single "printer" thread to output the result (makes things neater)
    t = threading.Thread(target=printer)
    t.daemon = True
    t.start()

    # put items on the input queue (numbers to be squared)
    for item in range(10):
        q_in.put(item)

    # wait for two queues to be emptied (and workers to close)   
    q_in.join()       # block until all tasks are done
    q_out.join()

    print "Processing Complete"

main()

Python 3 version per @handle

import queue 
import threading

# input queue to be processed by many threads
q_in = queue.Queue(maxsize=0) 

# output queue to be processed by one thread
q_out = queue.Queue(maxsize=0) 

# number of worker threads to complete the processing
num_worker_threads = 10

# process that each worker thread will execute until the Queue is empty
def worker():
    while True:
        # get item from queue, do work on it, let queue know processing is done for one item
        item = q_in.get()
        q_out.put(do_work(item))
        q_in.task_done()

# squares a number and returns the number and its square
def do_work(item):
    return (item,item*item)

# another queued thread we will use to print output
def printer():
    while True:
        # get an item processed by worker threads and print the result. Let queue know item has been processed
        item = q_out.get()
        print("{0[0]} squared is : {0[1]}".format(item) )
        q_out.task_done()

# launch all of our queued processes
def main():
    # Launches a number of worker threads to perform operations using the queue of inputs
    for i in range(num_worker_threads):
         t = threading.Thread(target=worker)
         t.daemon = True
         t.start()

    # launches a single "printer" thread to output the result (makes things neater)
    t = threading.Thread(target=printer)
    t.daemon = True
    t.start()

    # put items on the input queue (numbers to be squared)
    for item in range(10):
        q_in.put(item)

    # wait for two queues to be emptied (and workers to close)   
    q_in.join()       # block until all tasks are done
    q_out.join()

    print( "Processing Complete" )

main()

2 of 2

You can think of the number of worker threads as the number of bank tellers at a bank. So people (your items) stand in line (your queue) to be processed by a bank teller (your worker thread). Queues are actually an easy and well understood mechanism to manage complexities in threads.

I have adjusted your code a bit to show how it works.

import queue
import time
from threading import Thread

def do_work(item):
    print("processing", item)

def source():
    item = 1
    while True:
        print("starting", item)
        yield item
        time.sleep(0.2)
        item += 1

def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

q = queue.Queue(maxsize=0)
def main():
    for i in range(2):
        t = Thread(target=worker)
        t.daemon = True
        t.start()

    for item in source():
        q.put(item)

    q.join()       # block until all tasks are done

main()

Find elsewhere

Google Bing Mojeek

Python

docs.python.org › 3 › library › asyncio-queue.html

Queues — Python 3.14.4 documentation

February 22, 2026 - import asyncio import random import time async def worker(name, queue): while True: # Get a "work item" out of the queue. sleep_for = await queue.get() # Sleep for the "sleep_for" seconds. await asyncio.sleep(sleep_for) # Notify the queue that the "work item" has been processed. queue.task_done() print(f'{name} has slept for {sleep_for:.2f} seconds') async def main(): # Create a queue that we will use to store our "workload". queue = asyncio.Queue() # Generate random timings and put them into the queue. total_sleep_time = 0 for _ in range(20): sleep_for = random.uniform(0.05, 1.0) total_sleep_

Guru99

guru99.com › home › python › python queue: fifo, lifo example

Python Queue: FIFO, LIFO Example

August 12, 2024 - import queue q1 = queue.LifoQueue() for i in range(20): q1.put(i) # this will additem from 0 to 20 to the queue while not q1.empty(): print("The value is ", q1.get()) # get() will remove the item from the queue.

W3Schools

w3schools.com › python › python_dsa_queues.asp

Queues with Python

Queues are often mentioned together with Stacks, which is a similar data structure described on the previous page. For Python lists (and arrays), a Queue can look and behave like this:

Medium

basillica.medium.com › working-with-queues-in-python-a-complete-guide-aa112d310542

Working with Queues in Python — A Complete Guide | by Basillica | Medium

March 27, 2024 - Example usage: from collections import deque dq = deque() dq.append(1) dq.append(2) dq.appendleft(3) print(dq) # deque([3, 1, 2]) dq.pop() # 2 dq.popleft() # 3 · deque provides a thread-safe version called deque.deque that can be used in multi-threaded programs. Implementing a Queue in Python ·

BitDegree

bitdegree.org › learn › python-queue

The Four Types of Python Queue: Definitions and Examples

February 19, 2020 - You can also create a queue in Python that follows the LIFO principle (Last In, First Out). In such a case, the first element to remove will be the one that got added last.

Read the Docs

stackless.readthedocs.io › en › 2.7-slp › library › queue.html

8.10. Queue — A synchronized queue class — Stackless-Python 2.7.15 documentation

If maxsize is less than or equal to zero, the queue size is infinite. The lowest valued entries are retrieved first (the lowest valued entry is the one returned by sorted(list(entries))[0]). A typical pattern for entries is a tuple in the form: (priority_number, data). New in version 2.6.

Great Learning

mygreatlearning.com › blog › it/software development › python queue

Python Queue

October 14, 2024 - Queues are very useful for example, if we need a process to be executed one after other automatically in an order i.e. just like a list. It will process the tasks in a manner called “First In First Out” which means the first process or task in the queue will be executed and removed first, after that other processes will be started. A queue can be implemented in programming languages such as Python, Java, C++, etc.

Real Python

realpython.com › queue-in-python

Python Stacks, Queues, and Priority Queues in Practice – Real Python

December 1, 2023 - The .__iter__() method above is an example of a generator iterator, which yields elements lazily. Note: The implementation of .__iter__() causes your custom queue to reduce its size by dequeuing elements from itself as you iterate over it. Restart the Python interpreter and import your class again to see the updated code in action: ... >>> from queues import Queue >>> fifo = Queue("1st", "2nd", "3rd") >>> len(fifo) 3 >>> for element in fifo: ...

Troy Fawkes

troyfawkes.com › home › blog › the basics of python multithreading and queues

The Basics of Python Multithreading and Queues - Troy Fawkes

May 13, 2024 - It’s the bare-bones concepts of Queuing and Threading in Python. Before you do anything else, import Queue. from Queue import Queue · A queue is kind of like a list: my_list = [] my_list.append(1) my_list.append(2) my_list.append(3) print my_list.pop(0) # Outputs: 1 ·

Runestone Academy

runestone.academy › ns › books › published › pythonds › BasicDS › ImplementingaQueueinPython.html

4.12. Implementing a Queue in Python — Problem Solving with Algorithms and Data Structures

Q-2: Suppose you have the following series of queue operations. q = Queue() q.enqueue('hello') q.enqueue('dog') q.enqueue(3) q.dequeue() ... Remember the first thing added to the queue is the first thing removed. FIFO ... Queues, and Stacks are both data structures where you can only access ...

Stack Overflow

stackoverflow.com › questions › 54277946 › queue-between-python2-and-python3

python - Queue between Python2 and Python3 - Stack Overflow

January 20, 2019 - from multiprocessing.managers import BaseManager class QueueManager(BaseManager): pass QueueManager.register('get_queue') m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra') m.connect() queue = m.get_queue() queue.put('hello') ...

Spark By {Examples}

sparkbyexamples.com › home › python › queue in python with examples

Queue in Python with Examples - Spark By {Examples}

May 31, 2024 - # Output: Processing task: Task 2 with priority: 1 Processing task: Task 1 with priority: 2 Processing task: Task 3 with priority: 3 · There are two common ways to implement a queue: using a list or using the queue class from the built-in queue module. Let’s see them one by one. In Python, a queue can be implemented using a class with methods for enqueue, dequeue, size, and isEmpty.

Intellipaat

intellipaat.com › home › blog › queue in python – implementation explained

Queue in Python: How to Implement Queue in Python

October 14, 2025 - The function uses the append() method available for lists in Python to add the specified element to the end of the queue list. The example usage demonstrates how to create an empty queue (my_queue) and enqueue three elements: ‘apple’, ‘banana’, and ‘cherry’. After enqueuing the elements, the print(my_queue) statement outputs [‘apple’, ‘banana’, ‘cherry’], indicating that the elements have been successfully enqueued in the desired order.