python queue thread-safe

Is python Queue.queue get and put thread safe?

stackoverflow.com › questions › 14053102 › is-python-queue-queue-get-and-put-thread-safe

See 8.10 Queue — A synchronized queue class (at the top)

The Queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads.

Answer from Pavel Anossov on Stack Overflow

Python

docs.python.org › 3 › library › queue.html

queue — A synchronized queue class

February 23, 2026 - The queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads.

Stack Overflow

stackoverflow.com › questions › 14053102 › is-python-queue-queue-get-and-put-thread-safe

Is python Queue.queue get and put thread safe? - Stack Overflow

Top answer

1 of 2

68

See 8.10 Queue — A synchronized queue class (at the top)

The Queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads.

2 of 2

-1

We need a python3 version:

import threading
import queue

q = queue.Queue()

def worker():
    while True:
        item = q.get()
        print(f'Working on {item}')
        print(f'Finished {item}')
        q.task_done()

# Turn-on the worker thread.
threading.Thread(target=worker, daemon=True).start()

# Send thirty task requests to the worker.
for item in range(30):
    q.put(item)

# Block until all tasks are done.
q.join()
print('All work completed')

reference: https://docs.python.org/3/library/queue.html#queue-objects

Discussions

algorithms - How are queues made thread-safe in Python - Software Engineering Stack Exchange

I admit this was asked to me in interview a long time ago, but I never bothered to check it. The question was simple, how does Python make Queue thread-safe? My answer was, because of Interpreter... More on softwareengineering.stackexchange.com

softwareengineering.stackexchange.com

February 2, 2018

Is `std::queue` thread-safe? How can I use queues safely in a multi-threaded environment?

Is `std::queue` thread-safe? How can I use queues safely in a multi-threaded environment? More on studyplan.dev

studyplan.dev

1

May 24, 2023

Beginner Question on Queue and Multithreading

You are way off. It has nothing to do with the number of threads The first thing you need to understand is a "race condition". Imagine you have a list of tasks that you would like your threads to accomplish. Each thread will be while to_do_list: task = to_do_list[0] del to_do_list[0] # remove this task from the list do_work(task) When you have several threads working at the same time, it's possible that a 2 threads read the same task from the list at the same time. So basically after threadA reads the task but before threadA has a chance to delete it from the todo list, threadB has also read the task. In our example the first task in the list is done twice and the second task is skipped. But it could lead to much more serious data corruption. A Queue is simply a list that's protected against this condition. A thread can get and delete from the Queue without worry that another thread interferes. So we use Queues to pass data in and out of threads. But you should note we don't always need to use a Queue in python, because many of pythons basic functions have the same threadsafe protections built in (aka the GIL). An advanced user may use list.pop to do the same thing. More on reddit.com

r/learnpython

6

4

October 22, 2024

How/do I need to use a queue to make concurrent futures threadsafe?

Since you are working with accounts, logins and such, the limiting factor won't the the CPU, but waiting for IO. So Multiprocessing doesn't seem idea here, apart from having more CPU power for more threads. Threading seems to be the best way to go, so we avoid multiprocessing and it's potential problems. I'll just refer to the code example in the offical docs: import concurrent.futures import urllib.request URLS = ['http://www.foxnews.com/', 'http://www.cnn.com/', 'http://europe.wsj.com/', 'http://www.bbc.co.uk/', 'http://some-made-up-domain.com/'] # Retrieve a single page and report the URL and contents def load_url(url, timeout): with urllib.request.urlopen(url, timeout=timeout) as conn: return conn.read() # We can use a with statement to ensure threads are cleaned up promptly with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: # Start the load operations and mark each future with its URL future_to_url = {executor.submit(load_url, url, 60): url for url in URLS} for future in concurrent.futures.as_completed(future_to_url): url = future_to_url[future] try: data = future.result() except Exception as exc: print('%r generated an exception: %s' % (url, exc)) else: print('%r page is %d bytes' % (url, len(data))) More on reddit.com

r/learnpython

6

3

September 29, 2019

Videos