After reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:
import cv2
import multiprocessing as mp
import numpy as np
import psutil
img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes # Assuming img.shape[0] is divisible by 4 in this case
output = mp.Queue()
def mp_filter(x, output):
print(psutil.virtual_memory()) # monitor memory usage
output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :],
(kernel_size, kernel_size), kernel_size/5))
# note that you actually have to process a slightly larger block and leave out the border.
if __name__ == 'main':
processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]
for p in processes:
p.start()
result = []
for ii in range(num_processes):
result.append(output.get(True))
for p in processes:
p.join()
Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)
result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)
Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :
result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
Answer from user3667217 on Stack OverflowAfter reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:
import cv2
import multiprocessing as mp
import numpy as np
import psutil
img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes # Assuming img.shape[0] is divisible by 4 in this case
output = mp.Queue()
def mp_filter(x, output):
print(psutil.virtual_memory()) # monitor memory usage
output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :],
(kernel_size, kernel_size), kernel_size/5))
# note that you actually have to process a slightly larger block and leave out the border.
if __name__ == 'main':
processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]
for p in processes:
p.start()
result = []
for ii in range(num_processes):
result.append(output.get(True))
for p in processes:
p.join()
Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)
result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)
Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :
result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
I don't know what types of filters you need, but if it's reasonably simple, you could consider libvips. It's an image processing system for very large images (larger than the amount of memory you have). It came out of a series of EU-funded scientific art imaging projects, so the focus is on the types of operation you need for image capture and comparison: convolution, rank, morphology, arithmetic, colour analysis, resampling, histograms, and so on.
It's fast (faster than OpenCV, on some benchmarks at least), needs little memory, and there's a high-level Python binding. It works on Linux, OS X and Windows. It handles all the multiprocessing for you automatically.
[Question] How does multiprocessing work in python-opencv?
Taking advantage of multiprocessing with OpenCV on raspberry Pi 2 - Raspberry Pi Stack Exchange
Python OpenCV - multiprocessing doesn't work with CUDA
Python OpenCV multiprocessing cv2.VideoCapture mp4 - Stack Overflow
Hi. When I run a python VideoCapture loop, there are processes launched for each CPU core that show the python script running. How does this work under the hood in regard to the OpenCV library? Does it split the incoming video frames so they can be processed in parallel? It shows 15% resource use for each core (out of 20 cores) for a 30 FPS webcam showing simple video without processing using imshow. Can it be fit into a single process if it's only showing the frames? I couldn't find documentation on it.
I'm trying to optimize video processing without using a video card and thought the first step would be to not use all 20 CPU cores when showing video without processing, so that later the code can be used on a smaller device like a Raspberry Pi.
You should check out the multiprocessing library. If you have parts of you program that you can run in parallel, like processing multiple images and detecting a faces, you can simply define a process Pool. By default a pool will start as many processes simultaneously as you have processors. You can also define the pool size and fix it to 4.
e.g:
from multiprocessing import Pool
from glob import glob
def detect_face(file_name):
...
if __name__ == '__main__':
with Pool() as p:
p.map(detect_face, glob('*.jpg'))
You should also checkout sherlock, a collection of example on how to combine multiprocessing and opencv.
Limitations:
The algorithms in OpenCV itself are not easily executed in multiple processes. As OpenCV is written in C++, their execution depends highly on their implementation.
If you compile the OpenCV library with the OpenMP option active in the cmake file (it is off by default), many algorithms will use multiple cores automatically, without you needing to change anything in your source code. In my case I managed to achieve an up to 4.5x increase in performance. Yes, the Raspberry Pi 2 only has 4 cores, but if you do heavy number-crunching, a good division of the tasks can significantly reduce waiting times. For shorter algorithms don't expect much more than a 3x speedup.
Take care, with compiling the OpenMP libraries there is a bug in the libpng package. If the make halts with an error because libpng cannot find zlib, a quick and dirty solution can be that you just copy the source of zlib into the libpng source folder (or turn libpng off, if you don't need it).
The simplest approach is to use the newer cv2 module that's based on NumPy arrays. That way you don't have to mess with manual pickling. Here's the fix (I just changed 4 lines of code):
import multiprocessing
import cv2
queue_from_cam = multiprocessing.Queue()
def cam_loop(queue_from_cam):
print 'initializing cam'
cap = cv2.VideoCapture(0)
print 'querying frame'
hello, img = cap.read()
print 'queueing image'
queue_from_cam.put(img)
print 'cam_loop done'
cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()
while queue_from_cam.empty():
pass
print 'getting image'
from_queue = queue_from_cam.get()
print 'saving image'
cv2.imwrite('temp.png', from_queue)
print 'image saved'
It appears that the solution was to convert the opencv iplimage object to string, then pickle it before adding it to the queue:
import multiprocessing
import cv
import Image
import pickle
import time
queue_from_cam = multiprocessing.Queue()
def cam_loop(queue_from_cam):
print 'initializing cam'
cam = cv.CaptureFromCAM(-1)
print 'querying frame'
img = cv.QueryFrame(cam)
print 'converting image'
pimg = img.tostring()
print 'pickling image'
pimg2 = pickle.dumps(pimg,-1)
print 'queueing image'
queue_from_cam.put([pimg2,cv.GetSize(img)])
print 'cam_loop done'
cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()
while queue_from_cam.empty():
pass
print 'getting pickled image'
from_queue = queue_from_cam.get()
print 'unpickling image'
pimg = pickle.loads(from_queue[0])
print 'unconverting image'
cv_im = cv.CreateImageHeader(from_queue[1], cv.IPL_DEPTH_8U, 3)
cv.SetData(cv_im, pimg)
print 'saving image'
cv.SaveImage('temp.png',cv_im)
print 'image saved'
I think multiprocessing will work when you are running the function for multiple images at the same time.
Right now you are using a for loop and passing one image at a time to a single process.
Try the following approach and compare the time this takes with your original approach.
if __name__ == '__main__':
p1 = Process(target=Upscale,args(file_list[0:len(file_list)//2]))
p1.start()
p2 = Process(target=Upscale,args(file_list[len(file_list)//2:len(file_list)]))
p2.start()
p1.join()
p2.join()
Inside the Upscale function, make a for loop and loop through the images, and perform the same tasks as before.
Also, note that the function's argument is mentioned separately in "args" of the function "multiprocessing.Process".
P.S This is my first time answering, let me know if something is not clear.
For the output, you can try the following approach, I picked it up online through this https://superfastpython.com/multiprocessing-return-value-from-process/
# example of returning a variable from a process using a value
from random import random
from time import sleep
from multiprocessing import Value
from multiprocessing import Process
# function to execute in a child process
def task(variable):
# generate some data
data = random()
# block, to simulate computational effort
print(f'Generated {data}', flush=True)
sleep(data)
# return data via value
variable.value = data
# protect the entry point
if __name__ == '__main__':
# create shared variable
variable = Value('f', 0.0)
# create a child process process
process = Process(target=task, args=(variable,))
# start the process
process.start()
# wait for the process to finish
process.join()
# report return value
print(f'Returned: {variable.value}')
Multithreading / multiprocessing can be inefficient or even counterproductive when used to optimise the execution of neural networks on CPUs since a network execution may already be multithreaded.
Before any optimisation work you should first profile your code in order to identify what is slow and what can be improved. Here is some relevant questions you can ask yourself during this optimisation process:
- Does executing a single network already max out all the CPU's?
- Is it possible to execute the network on a faster accelerator such as the GPU?
- In the first place, is your network optimised (FP16, vectorisation, etc.)?
- Is OpenCV up to date and compiled to take advantage of all the available compute power?
- What is the memory footprint of a single network and how many could you execute at the same time on the CPU without out of memory errors?
- Is there any unnecessary redundant allocation (such as the one I pointed out in another of your question)?
- Are there obvious operations that would greatly benefit from being run concurrently (such as
imwriteandimreadbeing memory-bound whileupscaleis compute-bound)?
This list is not exhaustive, try to search for methods to profile, benchmark and optimise neural networks executions.