After reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:

import cv2
import multiprocessing as mp
import numpy as np
import psutil

img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes  # Assuming img.shape[0] is divisible by 4 in this case

output = mp.Queue()

def mp_filter(x, output):
    print(psutil.virtual_memory())  # monitor memory usage
    output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :], 
               (kernel_size, kernel_size), kernel_size/5))
    # note that you actually have to process a slightly larger block and leave out the border.

if __name__ == 'main':
    processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]

    for p in processes:
        p.start()

    result = []
    for ii in range(num_processes):
        result.append(output.get(True))

    for p in processes:
        p.join()

Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)

result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)

Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :

result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
Answer from user3667217 on Stack Overflow
🌐
PyImageSearch
pyimagesearch.com › home › blog › multiprocessing with opencv and python
Multiprocessing with OpenCV and Python - PyImageSearch
December 29, 2022 - We examined multiprocessing with OpenCV through indexing a dataset of images for building an image hashing search engine; however, you can modify/implement your own process_images function to include your own functionality.
Top answer
1 of 3
5

After reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:

import cv2
import multiprocessing as mp
import numpy as np
import psutil

img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes  # Assuming img.shape[0] is divisible by 4 in this case

output = mp.Queue()

def mp_filter(x, output):
    print(psutil.virtual_memory())  # monitor memory usage
    output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :], 
               (kernel_size, kernel_size), kernel_size/5))
    # note that you actually have to process a slightly larger block and leave out the border.

if __name__ == 'main':
    processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]

    for p in processes:
        p.start()

    result = []
    for ii in range(num_processes):
        result.append(output.get(True))

    for p in processes:
        p.join()

Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)

result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)

Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :

result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
2 of 3
4

I don't know what types of filters you need, but if it's reasonably simple, you could consider libvips. It's an image processing system for very large images (larger than the amount of memory you have). It came out of a series of EU-funded scientific art imaging projects, so the focus is on the types of operation you need for image capture and comparison: convolution, rank, morphology, arithmetic, colour analysis, resampling, histograms, and so on.

It's fast (faster than OpenCV, on some benchmarks at least), needs little memory, and there's a high-level Python binding. It works on Linux, OS X and Windows. It handles all the multiprocessing for you automatically.

Discussions

[Question] How does multiprocessing work in python-opencv?
OpenCV uses ffmpeg libraries under the hood, and their implementation does not expose controls for how that additional processing is handled. If you really need to control the number of cores used or similar video processing optimizations you'll need to work within the ffmpeg libraries themselves. I have a c++ ffmpeg playback library I wrote specifically for this purpose. There are two github projects, one for the ffmpeg playback library and one for a win10 wxWidgets Desktop Application hosting the library, demonstrating it's capabilities. The library is intended for video security and ML/DL training, so their is no audio handling and all video processing occurs as fast as possible on a single CPU core. You may find it useful: https://github.com/bsenftner/ffvideo A variant of this library is used within one of the top 5 global enterprise facial recognition systems; I wrote enterprise FR for about a decade before leaving that industry. More on reddit.com
🌐 r/opencv
6
1
November 14, 2022
Taking advantage of multiprocessing with OpenCV on raspberry Pi 2 - Raspberry Pi Stack Exchange
I used to use OpenCV on Raspberry for different projects (face recognition, sync cam, stitcher etc...). I bought a raspberry Pi 2 for the advantages given by the new hardware config. However, I co... More on raspberrypi.stackexchange.com
🌐 raspberrypi.stackexchange.com
Python OpenCV - multiprocessing doesn't work with CUDA
Hello, I am trying to run CUDA ORB key-point detection with multiple GPUs. The principle of work is to split list of video frames between available GPU devices (load them into GPU memory). However when I run it with multiple threads by threading , i observe that each GPU slows down - I suppose ... More on forums.developer.nvidia.com
🌐 forums.developer.nvidia.com
2
0
June 8, 2021
Python OpenCV multiprocessing cv2.VideoCapture mp4 - Stack Overflow
I want to run some mp4 videos inside a process but only camera feed works. The software gets stuck with no error message. I already tried both and I found this doesn't run. The print is where the c... More on stackoverflow.com
🌐 stackoverflow.com
🌐
OpenCV Q&A Forum
answers.opencv.org › question › 234787 › opencv-44-and-multiprocessing-on-windows-10
OpenCV 4.4 and multiprocessing on Windows 10 - OpenCV Q&A Forum
It has to do with the way Python 3.8 revamped how module extension dlls are loaded. So if you want to use multiprocessing for OpenCV on Windows 10, use Python 3.7 or earlier.
🌐
Xailient
xailient.com › blog › parallel-processing-using-python-for-faster-video-processing
Parallel Processing Using Python for Faster Video Processing | Xailient
cv2: OpenCV library to read from ... pipes, and obtain their return codes. multiprocessing: to parallelize the execution of a function across multiple input values, distributing the input data across processes ·...
🌐
OpenCV Q&A Forum
answers.opencv.org › question › 201262 › how-to-implement-multi-processing
How to implement multi-processing ? - OpenCV Q&A Forum
October 16, 2018 - E:\git\opencv-dnn-demo\face_detectors>python sample.py · i am detect i am detect i am detect i am detect i am detect i am detect Traceback (most recent call last): File "sample.py", line 17, in <module> pool.map(fd.detect, imgs) File "C:\Anaconda3\lib\multiprocessing\pool.py", line 260, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "C:\Anaconda3\lib\multiprocessing\pool.py", line 608, in get raise self._value File "C:\Anaconda3\lib\multiprocessing\pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "C:\Anaconda3\lib\multiprocessing\pool.py",
Author: faisalshahbaz
🌐
Reddit
reddit.com › r/opencv › [question] how does multiprocessing work in python-opencv?
r/opencv on Reddit: [Question] How does multiprocessing work in python-opencv?
November 14, 2022 -

Hi. When I run a python VideoCapture loop, there are processes launched for each CPU core that show the python script running. How does this work under the hood in regard to the OpenCV library? Does it split the incoming video frames so they can be processed in parallel? It shows 15% resource use for each core (out of 20 cores) for a 30 FPS webcam showing simple video without processing using imshow. Can it be fit into a single process if it's only showing the frames? I couldn't find documentation on it.

I'm trying to optimize video processing without using a video card and thought the first step would be to not use all 20 CPU cores when showing video without processing, so that later the code can be used on a smaller device like a Raspberry Pi.

Top answer
1 of 2
3

You should check out the multiprocessing library. If you have parts of you program that you can run in parallel, like processing multiple images and detecting a faces, you can simply define a process Pool. By default a pool will start as many processes simultaneously as you have processors. You can also define the pool size and fix it to 4.

e.g:

from multiprocessing import Pool
from glob import glob

def detect_face(file_name):
    ...

if __name__ == '__main__':
    with Pool() as p:
       p.map(detect_face, glob('*.jpg'))

You should also checkout sherlock, a collection of example on how to combine multiprocessing and opencv.

Limitations:

The algorithms in OpenCV itself are not easily executed in multiple processes. As OpenCV is written in C++, their execution depends highly on their implementation.

2 of 2
3

If you compile the OpenCV library with the OpenMP option active in the cmake file (it is off by default), many algorithms will use multiple cores automatically, without you needing to change anything in your source code. In my case I managed to achieve an up to 4.5x increase in performance. Yes, the Raspberry Pi 2 only has 4 cores, but if you do heavy number-crunching, a good division of the tasks can significantly reduce waiting times. For shorter algorithms don't expect much more than a 3x speedup.

Take care, with compiling the OpenMP libraries there is a bug in the libpng package. If the make halts with an error because libpng cannot find zlib, a quick and dirty solution can be that you just copy the source of zlib into the libpng source folder (or turn libpng off, if you don't need it).

Find elsewhere
🌐
Nrsyed
nrsyed.com › 2018 › 07 › 05 › multithreading-with-opencv-python-to-improve-video-processing-performance
Multithreading with OpenCV-Python to improve video processing performance • Najam R. Syed
July 6, 2018 - We import the threading module, which will allow us to spawn new threads. In the class __init__() method, we initialize an OpenCV VideoCapture object and read the first frame.
🌐
GitHub
github.com › mahdizynali › OpenCV-video-multiprocessing
GitHub - mahdizynali/OpenCV-video-multiprocessing: opencv python video multiprocessing besides tkinter video multiprocessing
with this script, it will be possible ... mode, by multiprocessing and seprating frames into some parts and process each part by a thread and add them into a queue, so finally stick the queue processing result together....
Author: mahdizynali
🌐
NVIDIA Developer Forums
forums.developer.nvidia.com › accelerated computing › cuda › cuda programming and performance
Python OpenCV - multiprocessing doesn't work with CUDA - CUDA Programming and Performance - NVIDIA Developer Forums
June 8, 2021 - Hello, I am trying to run CUDA ORB key-point detection with multiple GPUs. The principle of work is to split list of video frames between available GPU devices (load them into GPU memory).
🌐
Fast.ai
forums.fast.ai › part 1 (2018) › beginner (2018)
Why doesn't opencv seem to work with multiprocessing? - Beginner (2018) - fast.ai Course Forums
April 10, 2018 - I am new to using multiprocessing. This works [download] But if I were to use the open_image function that lives in dataset.py instead of the PIL Image.open a process will get stuck on reading in the first file, so it …
🌐
Stack Overflow
stackoverflow.com › questions › 72212226 › multiprocessing-opencv-in-python
multithreading - Multiprocessing OpenCV in Python - Stack Overflow
I recognize the issue is slightly different, but I do have an old example of handling continuous frames with opencv between multiple processes using multiprocessing.shared_memory as a frame buffer between the processes: stackoverflow.com/a/66522825/3220135
Top answer
1 of 3
11

The simplest approach is to use the newer cv2 module that's based on NumPy arrays. That way you don't have to mess with manual pickling. Here's the fix (I just changed 4 lines of code):

import multiprocessing
import cv2

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cap = cv2.VideoCapture(0)
    print 'querying frame'
    hello, img = cap.read()
    print 'queueing image'
    queue_from_cam.put(img)
    print 'cam_loop done'

cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting image'
from_queue = queue_from_cam.get()
print 'saving image'
cv2.imwrite('temp.png', from_queue)
print 'image saved'
2 of 3
6

It appears that the solution was to convert the opencv iplimage object to string, then pickle it before adding it to the queue:

import multiprocessing
import cv
import Image
import pickle
import time

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cam = cv.CaptureFromCAM(-1)
    print 'querying frame'
    img = cv.QueryFrame(cam)
    print 'converting image'
    pimg = img.tostring()
    print 'pickling image'
    pimg2 = pickle.dumps(pimg,-1)
    print 'queueing image'
    queue_from_cam.put([pimg2,cv.GetSize(img)])
    print 'cam_loop done'


cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting pickled image'
from_queue = queue_from_cam.get()
print 'unpickling image'
pimg = pickle.loads(from_queue[0])
print 'unconverting image'
cv_im = cv.CreateImageHeader(from_queue[1], cv.IPL_DEPTH_8U, 3)
cv.SetData(cv_im, pimg)
print 'saving image'
cv.SaveImage('temp.png',cv_im)
print 'image saved'
🌐
The Neural Base
theneuralbase.com › home › opencv › advanced course › parallel image processing
Parallel image processing | Opencv Advanced Course | The Neural Base
The key mechanical insight: you can't share a NumPy array (which backs OpenCV's Mat objects) across processes without explicit serialization. You must either: (1) pass image file paths and let each worker load and process independently, or (2) use multiprocessing.Queue or mp.Pool with proper serialization.
🌐
OpenCV Q&A Forum
answers.opencv.org › question › 227029 › how-to-use-multiprocessing-with-opencv-c
How to use multiprocessing with OpenCV-C++ ? - OpenCV Q&A Forum
February 3, 2020 - You need to compile OpenCV from the sources with some flags such as TBB for best multithreading performances. Moreover you can set many flags, I sugget you to use CMake GUI and check it.
🌐
GitHub
github.com › opencv › opencv › issues › 6034
Python Multiprocessing Problem · Issue #6034 · opencv/opencv
January 30, 2016 - class Consumer(multiprocessing.Process): def __init__(self, task_queue, result_queue): multiprocessing.Process.__init__(self) self.task_queue = task_queue self.result_queue = result_queue def run(self): #Creating window and starting video camputer from camera cv2.namedWindow("preview") vc = cv2.VideoCapture(0) vc.set(cv.CV_CAP_PROP_FRAME_WIDTH, 640) vc.set(cv.CV_CAP_PROP_FRAME_HEIGHT, 480) if vc.isOpened(): rval, frame = vc.read() else: rval = False while rval: cv2.imshow("preview", frame) rval, frame = vc.read() key = cv2.waitKey(20) if key == 27: break cv2.destroyWindow("preview") return ################################## #MAIN PROGAM ################################## #SETTING UP MULTIPROCESSING STUFF tasks = multiprocessing.JoinableQueue() results = multiprocessing.Queue() consumer = Consumer(tasks,results) consumer.start() #Rest of main program
Author: opencv
Top answer
1 of 2
1

I think multiprocessing will work when you are running the function for multiple images at the same time.

Right now you are using a for loop and passing one image at a time to a single process.

Try the following approach and compare the time this takes with your original approach.

if __name__ == '__main__':
    p1 = Process(target=Upscale,args(file_list[0:len(file_list)//2]))
    p1.start()
    p2 = Process(target=Upscale,args(file_list[len(file_list)//2:len(file_list)]))
    p2.start()
    p1.join()
    p2.join()

Inside the Upscale function, make a for loop and loop through the images, and perform the same tasks as before.

Also, note that the function's argument is mentioned separately in "args" of the function "multiprocessing.Process".

P.S This is my first time answering, let me know if something is not clear.

For the output, you can try the following approach, I picked it up online through this https://superfastpython.com/multiprocessing-return-value-from-process/

# example of returning a variable from a process using a value
from random import random
from time import sleep
from multiprocessing import Value
from multiprocessing import Process

# function to execute in a child process
def task(variable):
    # generate some data
    data = random()
    # block, to simulate computational effort
    print(f'Generated {data}', flush=True)
    sleep(data)
    # return data via value
    variable.value = data

# protect the entry point
if __name__ == '__main__':
    # create shared variable
    variable = Value('f', 0.0)
    # create a child process process
    process = Process(target=task, args=(variable,))
    # start the process
    process.start()
    # wait for the process to finish
    process.join()
    # report return value
    print(f'Returned: {variable.value}')
2 of 2
1

Multithreading / multiprocessing can be inefficient or even counterproductive when used to optimise the execution of neural networks on CPUs since a network execution may already be multithreaded.

Before any optimisation work you should first profile your code in order to identify what is slow and what can be improved. Here is some relevant questions you can ask yourself during this optimisation process:

  • Does executing a single network already max out all the CPU's?
  • Is it possible to execute the network on a faster accelerator such as the GPU?
  • In the first place, is your network optimised (FP16, vectorisation, etc.)?
  • Is OpenCV up to date and compiled to take advantage of all the available compute power?
  • What is the memory footprint of a single network and how many could you execute at the same time on the CPU without out of memory errors?
  • Is there any unnecessary redundant allocation (such as the one I pointed out in another of your question)?
  • Are there obvious operations that would greatly benefit from being run concurrently (such as imwrite and imread being memory-bound while upscale is compute-bound)?

This list is not exhaustive, try to search for methods to profile, benchmark and optimise neural networks executions.

🌐
Raspberry Pi Forums
forums.raspberrypi.com › board index › programming › python
Learning about multiprocessing with python - Raspberry Pi Forums
We have a requirement in a program I'm writing to execute to tasks that are going to run parallel to each other. I'm thinking multiprocessing is the way to do this since it will use multiple cores to run each task separately. The program uses OpenCV to fetch camera frames as well as pigpio to communicate with a controller.