After reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:

import cv2
import multiprocessing as mp
import numpy as np
import psutil

img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes  # Assuming img.shape[0] is divisible by 4 in this case

output = mp.Queue()

def mp_filter(x, output):
    print(psutil.virtual_memory())  # monitor memory usage
    output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :], 
               (kernel_size, kernel_size), kernel_size/5))
    # note that you actually have to process a slightly larger block and leave out the border.

if __name__ == 'main':
    processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]

    for p in processes:
        p.start()

    result = []
    for ii in range(num_processes):
        result.append(output.get(True))

    for p in processes:
        p.join()

Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)

result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)

Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :

result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
Answer from user3667217 on Stack Overflow
🌐
PyImageSearch
pyimagesearch.com › home › blog › multiprocessing with opencv and python
Multiprocessing with OpenCV and Python - PyImageSearch
December 29, 2022 - In this tutorial, you will learn how to use multiprocessing with OpenCV and Python to perform feature extraction. You’ll learn how to use multiprocessing with OpenCV to parallelize feature extraction across the system bus, including all processors and cores on your computer.
Top answer
1 of 3
5

After reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:

import cv2
import multiprocessing as mp
import numpy as np
import psutil

img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes  # Assuming img.shape[0] is divisible by 4 in this case

output = mp.Queue()

def mp_filter(x, output):
    print(psutil.virtual_memory())  # monitor memory usage
    output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :], 
               (kernel_size, kernel_size), kernel_size/5))
    # note that you actually have to process a slightly larger block and leave out the border.

if __name__ == 'main':
    processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]

    for p in processes:
        p.start()

    result = []
    for ii in range(num_processes):
        result.append(output.get(True))

    for p in processes:
        p.join()

Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)

result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)

Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :

result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
2 of 3
4

I don't know what types of filters you need, but if it's reasonably simple, you could consider libvips. It's an image processing system for very large images (larger than the amount of memory you have). It came out of a series of EU-funded scientific art imaging projects, so the focus is on the types of operation you need for image capture and comparison: convolution, rank, morphology, arithmetic, colour analysis, resampling, histograms, and so on.

It's fast (faster than OpenCV, on some benchmarks at least), needs little memory, and there's a high-level Python binding. It works on Linux, OS X and Windows. It handles all the multiprocessing for you automatically.

Discussions

[Question] How does multiprocessing work in python-opencv?
OpenCV uses ffmpeg libraries under the hood, and their implementation does not expose controls for how that additional processing is handled. If you really need to control the number of cores used or similar video processing optimizations you'll need to work within the ffmpeg libraries themselves. I have a c++ ffmpeg playback library I wrote specifically for this purpose. There are two github projects, one for the ffmpeg playback library and one for a win10 wxWidgets Desktop Application hosting the library, demonstrating it's capabilities. The library is intended for video security and ML/DL training, so their is no audio handling and all video processing occurs as fast as possible on a single CPU core. You may find it useful: https://github.com/bsenftner/ffvideo A variant of this library is used within one of the top 5 global enterprise facial recognition systems; I wrote enterprise FR for about a decade before leaving that industry. More on reddit.com
🌐 r/opencv
6
1
November 14, 2022
multithreading - Multiprocessing OpenCV in Python - Stack Overflow
I have a simple Algorithm, I want to run it fast in parallel. The algo is. while stream: img = read_image() pre_process_img = pre_process(img) text = ocr(pre_process_img) fine_text = More on stackoverflow.com
🌐 stackoverflow.com
Opencv multiprocessing in python - queue sync - Stack Overflow
I am a beginner with multiprocessing in Python. I am developing a multiprocessing script for OpenCV, since my computer does not succeed in real-time processing of OpenCV frames. I aim at loading and More on stackoverflow.com
🌐 stackoverflow.com
April 25, 2017
Multithreading vs. Multiprocessing with OpenCV in Python - Stack Overflow
I have the following function: def resize(path_to_image): img = cv2.imread(path_to_image) return img This function takes in an image, upscales the image, and then writes the image to a fol... More on stackoverflow.com
🌐 stackoverflow.com
🌐
Reddit
reddit.com › r/opencv › [question] how does multiprocessing work in python-opencv?
r/opencv on Reddit: [Question] How does multiprocessing work in python-opencv?
November 14, 2022 -

Hi. When I run a python VideoCapture loop, there are processes launched for each CPU core that show the python script running. How does this work under the hood in regard to the OpenCV library? Does it split the incoming video frames so they can be processed in parallel? It shows 15% resource use for each core (out of 20 cores) for a 30 FPS webcam showing simple video without processing using imshow. Can it be fit into a single process if it's only showing the frames? I couldn't find documentation on it.

I'm trying to optimize video processing without using a video card and thought the first step would be to not use all 20 CPU cores when showing video without processing, so that later the code can be used on a smaller device like a Raspberry Pi.

🌐
Xailient
xailient.com › blog › parallel-processing-using-python-for-faster-video-processing
Parallel Processing Using Python for Faster Video Processing | Xailient
Details of libraries used: cv2: ... pipes, and obtain their return codes. multiprocessing: to parallelize the execution of a function across multiple input values, distributing the input data across processes ·...
🌐
GitHub
github.com › mahdizynali › OpenCV-video-multiprocessing
GitHub - mahdizynali/OpenCV-video-multiprocessing: opencv python video multiprocessing besides tkinter video multiprocessing
In this script, we use an extra processor to make a switch case between script and gui program by creating a thread line that perform us to update ui window simultaneously when multiprocess is under working; it mean that the ui window won't intrrupt and never freeze during the procedure so we can set our orders into ui window :)) it's a sample of tkinter opencv video multiprocessing module that can be developed for other kind of gui systems.
Author: mahdizynali
Author: faisalshahbaz
🌐
Stack Overflow
stackoverflow.com › questions › 72212226 › multiprocessing-opencv-in-python
multithreading - Multiprocessing OpenCV in Python - Stack Overflow
I recognize the issue is slightly different, but I do have an old example of handling continuous frames with opencv between multiple processes using multiprocessing.shared_memory as a frame buffer between the processes: stackoverflow.com/a/66522825/3220135
Find elsewhere
Top answer
1 of 1
2

This should work (explanation of changes below):

#!/usr/bin/env python
from multiprocessing import Process, Queue
from Queue import Empty
from PIL import Image
import cv2
import cv2.cv as cv
import numpy as np

def image_display(taskqueue):
   cv2.namedWindow ('image_display', cv2.CV_WINDOW_AUTOSIZE)
   while True:

      image = taskqueue.get()              # Added
      if image is None:  break             # Added
      cv2.imshow ('image_display', image)  # Added
      cv2.waitKey(10)                      # Added
      continue                             # Added

      if taskqueue.get()==None:
         continue
      else:
         image = taskqueue.get()
         im = Image.fromstring(image['mode'], image['size'], image['pixels'])
         num_im = np.asarray(im)
         cv2.imshow ('image_display', num_im)


if __name__ == '__main__':
   taskqueue = Queue()
   vidFile = cv2.VideoCapture('doppler.wmv')
   p = Process(target=image_display, args=(taskqueue,))
   p.start()
   while True:
      flag, image=vidFile.read()

      taskqueue.put(image)  # Added
      import time           # Added
      time.sleep(0.010)     # Added
      continue              # Added

      if flag == 0:
         break
      im = Image.fromarray(image)
      im_dict = {
      'pixels': im.tostring(),
      'size': im.size,
      'mode': im.mode,
      }
      taskqueue.put(im_dict)

taskqueue.put(None)
p.join()
cv.DestroyAllWindows()

I tried to make minimal changes to your code by just adding lines (lines containing comments # Added):

1) Just put the image itself (the original NumPy array) on the queue.

2) Pause a little bit in the master process before reading another frame. You need this so as not to overrun the queue, because imshow() in the spawned process may take a bit longer since it's calling X. You might need to increase this value (in seconds) depending on your system.

3) Spawned process has to do the waitKey() after every imshow().

4) Master process puts the special None image on the queue when it's done.

Top answer
1 of 2
1

I think multiprocessing will work when you are running the function for multiple images at the same time.

Right now you are using a for loop and passing one image at a time to a single process.

Try the following approach and compare the time this takes with your original approach.

if __name__ == '__main__':
    p1 = Process(target=Upscale,args(file_list[0:len(file_list)//2]))
    p1.start()
    p2 = Process(target=Upscale,args(file_list[len(file_list)//2:len(file_list)]))
    p2.start()
    p1.join()
    p2.join()

Inside the Upscale function, make a for loop and loop through the images, and perform the same tasks as before.

Also, note that the function's argument is mentioned separately in "args" of the function "multiprocessing.Process".

P.S This is my first time answering, let me know if something is not clear.

For the output, you can try the following approach, I picked it up online through this https://superfastpython.com/multiprocessing-return-value-from-process/

# example of returning a variable from a process using a value
from random import random
from time import sleep
from multiprocessing import Value
from multiprocessing import Process

# function to execute in a child process
def task(variable):
    # generate some data
    data = random()
    # block, to simulate computational effort
    print(f'Generated {data}', flush=True)
    sleep(data)
    # return data via value
    variable.value = data

# protect the entry point
if __name__ == '__main__':
    # create shared variable
    variable = Value('f', 0.0)
    # create a child process process
    process = Process(target=task, args=(variable,))
    # start the process
    process.start()
    # wait for the process to finish
    process.join()
    # report return value
    print(f'Returned: {variable.value}')
2 of 2
1

Multithreading / multiprocessing can be inefficient or even counterproductive when used to optimise the execution of neural networks on CPUs since a network execution may already be multithreaded.

Before any optimisation work you should first profile your code in order to identify what is slow and what can be improved. Here is some relevant questions you can ask yourself during this optimisation process:

  • Does executing a single network already max out all the CPU's?
  • Is it possible to execute the network on a faster accelerator such as the GPU?
  • In the first place, is your network optimised (FP16, vectorisation, etc.)?
  • Is OpenCV up to date and compiled to take advantage of all the available compute power?
  • What is the memory footprint of a single network and how many could you execute at the same time on the CPU without out of memory errors?
  • Is there any unnecessary redundant allocation (such as the one I pointed out in another of your question)?
  • Are there obvious operations that would greatly benefit from being run concurrently (such as imwrite and imread being memory-bound while upscale is compute-bound)?

This list is not exhaustive, try to search for methods to profile, benchmark and optimise neural networks executions.

🌐
OpenCV Q&A Forum
answers.opencv.org › question › 201262 › how-to-implement-multi-processing
How to implement multi-processing ? - OpenCV Q&A Forum
October 16, 2018 - you maybe want to use the python bindings for yolo directly - https://github.com/AlexeyAB/darknet -> How to use Yolo as DLL · It also works on linux(.so). holger ( 2018-10-17 00:04:37 -0600 )edit · please DO NOT TRY to use multiprocessing with this. (neural networks are "stateful", and can't be shared between processes / threads) if you have several images to process, feed those into the nn as a batch for speedup.
🌐
NVIDIA Developer Forums
forums.developer.nvidia.com › accelerated computing › cuda › cuda programming and performance
Python OpenCV - multiprocessing doesn't work with CUDA - CUDA Programming and Performance - NVIDIA Developer Forums
June 8, 2021 - Hello, I am trying to run CUDA ORB key-point detection with multiple GPUs. The principle of work is to split list of video frames between available GPU devices (load them into GPU memory).
Top answer
1 of 3
11

The simplest approach is to use the newer cv2 module that's based on NumPy arrays. That way you don't have to mess with manual pickling. Here's the fix (I just changed 4 lines of code):

import multiprocessing
import cv2

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cap = cv2.VideoCapture(0)
    print 'querying frame'
    hello, img = cap.read()
    print 'queueing image'
    queue_from_cam.put(img)
    print 'cam_loop done'

cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting image'
from_queue = queue_from_cam.get()
print 'saving image'
cv2.imwrite('temp.png', from_queue)
print 'image saved'
2 of 3
6

It appears that the solution was to convert the opencv iplimage object to string, then pickle it before adding it to the queue:

import multiprocessing
import cv
import Image
import pickle
import time

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cam = cv.CaptureFromCAM(-1)
    print 'querying frame'
    img = cv.QueryFrame(cam)
    print 'converting image'
    pimg = img.tostring()
    print 'pickling image'
    pimg2 = pickle.dumps(pimg,-1)
    print 'queueing image'
    queue_from_cam.put([pimg2,cv.GetSize(img)])
    print 'cam_loop done'


cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting pickled image'
from_queue = queue_from_cam.get()
print 'unpickling image'
pimg = pickle.loads(from_queue[0])
print 'unconverting image'
cv_im = cv.CreateImageHeader(from_queue[1], cv.IPL_DEPTH_8U, 3)
cv.SetData(cv_im, pimg)
print 'saving image'
cv.SaveImage('temp.png',cv_im)
print 'image saved'
🌐
Fast.ai
forums.fast.ai › part 1 (2018) › beginner (2018)
Why doesn't opencv seem to work with multiprocessing? - Beginner (2018) - fast.ai Course Forums
April 10, 2018 - I am new to using multiprocessing. This works [download] But if I were to use the open_image function that lives in dataset.py instead of the PIL Image.open a process will get stuck on reading in the first file, so it …
🌐
Prodevsblog
prodevsblog.com › questions › 114165 › python-opencv-and-multiprocessing
Q&A ProDevsBlog | Python opencv and multiprocessing
import multiprocessing import cv queue_from_cam = multiprocessing.Queue() def cam_loop(queue_from_cam): print 'initializing cam' cam = cv.CaptureFromCAM(-1) print 'querying frame' img = cv.QueryFrame(cam) print 'queueing image' queue_from_cam.put(img) print 'cam_loop done' cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,)) cam_process.start() while queue_from_cam.empty(): pass print 'getting image' from_queue = queue_from_cam.get() print 'saving image' cv.SaveImage('temp.png',from_queue) print 'image saved' This code should run up to the print out of "saving image" but then hang. Any ideas how I can go about fixing this? ... I don't know much about openCV, but I bet the problem is that some internal state data is not copied between the processes.
🌐
GitHub
github.com › opencv › opencv › issues › 5150
OpenCV + Python multiprocessing breaks on OSX · Issue #5150 · opencv/opencv
August 7, 2015 - I'm trying to use OpenCV with Python's multiprocessing module, but it breaks on me even with very simple code. Here is an example: import cv2 import multiprocessing import glob import numpy def job(path): image = cv2.imread(path) image =...
Author: opencv
🌐
GitHub
gist.github.com › toori67 › 3a79a302d98e34a19defb818001d3e33
Python + opencv sift multiprocessing · GitHub
August 7, 2015 - Python + opencv sift multiprocessing. GitHub Gist: instantly share code, notes, and snippets.
🌐
Reddit
reddit.com › r/learnpython › pytesseract, opencv and multiprocessing
r/learnpython on Reddit: Pytesseract, opencv and multiprocessing
April 16, 2022 -

I'm working on a gaming bot and i'm thinking about using multiprocessing to separate pyteseract, opencv matchtemplate and normal computing stuff. I want to do this because I would like to use them in parallele because Pytesseract takes like 8 seconds to do it's stuff and matchtemplate takes approximatly 15 seconds and while they work, the game is stillgoing forward. So if i decide to go down that path, what can i expect to happen? Will Pytesseract and matchtemplate work slower, will i they be able to communicate variable with each other? Is there any hardware specification that i should be worried about? I never did it so i would like to know if the end result will be worth the time. I know it's worth learning, i just want to have my project benefit from the work, i'll be putting in learning it.

🌐
OpenCV Q&A Forum
answers.opencv.org › question › 181695 › multiprocessing-in-image-processing
Multiprocessing in image processing - OpenCV Q&A Forum
January 4, 2018 - Hello guys, i am currently working on image processing using the multiprocessing library in python, i have been thnking of 2 possible route: 1.) what I want to do is I have a video and we know the video is a stream of frames. I have core i5 what I want to do each core will process one frame of these frames example (what will happen in one second) core 1 processing frame 1 core 2 processing frame 2 core 3 processing frame 3 core 4 processing frame4.