After reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:
import cv2
import multiprocessing as mp
import numpy as np
import psutil
img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes # Assuming img.shape[0] is divisible by 4 in this case
output = mp.Queue()
def mp_filter(x, output):
print(psutil.virtual_memory()) # monitor memory usage
output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :],
(kernel_size, kernel_size), kernel_size/5))
# note that you actually have to process a slightly larger block and leave out the border.
if __name__ == 'main':
processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]
for p in processes:
p.start()
result = []
for ii in range(num_processes):
result.append(output.get(True))
for p in processes:
p.join()
Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)
result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)
Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :
result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
Answer from user3667217 on Stack OverflowAfter reading some SO posts, I've come up with a way to use OpenCV in Python3 with multiprocessing. I recommend doing this on linux, because according to this post, spawned processes share memory with their parent as long as the content is not changed. Here's a minimal example:
import cv2
import multiprocessing as mp
import numpy as np
import psutil
img = cv2.imread('test.tiff', cv2.IMREAD_ANYDEPTH) # here I'm using a indexed 16-bit tiff as an example.
num_processes = 4
kernel_size = 11
tile_size = img.shape[0]/num_processes # Assuming img.shape[0] is divisible by 4 in this case
output = mp.Queue()
def mp_filter(x, output):
print(psutil.virtual_memory()) # monitor memory usage
output.put(x, cv2.GaussianBlur(img[img.shape[0]/num_processes*x:img.shape[0]/num_processes*(x+1), :],
(kernel_size, kernel_size), kernel_size/5))
# note that you actually have to process a slightly larger block and leave out the border.
if __name__ == 'main':
processes = [mp.Process(target=mp_filter, args=(x, output)) for x in range(num_processes)]
for p in processes:
p.start()
result = []
for ii in range(num_processes):
result.append(output.get(True))
for p in processes:
p.join()
Instead of using Queue, another way to collect the result from the processes is to create a shared array through multiprocessing module. (Has to import ctypes)
result = mp.Array(ctypes.c_uint16, img.shape[0]*img.shape[1], lock = False)
Then each process can write to different portions of the array assuming there is no overlap. Creating a large mp.Array is surprisingly slow, however. This actually defies the purpose of speeding up the operation. So use it only when the added time is not much when compared with total computation time. This array can be turned into a numpy array by :
result_np = np.frombuffer(result, dtypye=ctypes.c_uint16)
I don't know what types of filters you need, but if it's reasonably simple, you could consider libvips. It's an image processing system for very large images (larger than the amount of memory you have). It came out of a series of EU-funded scientific art imaging projects, so the focus is on the types of operation you need for image capture and comparison: convolution, rank, morphology, arithmetic, colour analysis, resampling, histograms, and so on.
It's fast (faster than OpenCV, on some benchmarks at least), needs little memory, and there's a high-level Python binding. It works on Linux, OS X and Windows. It handles all the multiprocessing for you automatically.
[Question] How does multiprocessing work in python-opencv?
multithreading - Multiprocessing OpenCV in Python - Stack Overflow
Opencv multiprocessing in python - queue sync - Stack Overflow
Multithreading vs. Multiprocessing with OpenCV in Python - Stack Overflow
Hi. When I run a python VideoCapture loop, there are processes launched for each CPU core that show the python script running. How does this work under the hood in regard to the OpenCV library? Does it split the incoming video frames so they can be processed in parallel? It shows 15% resource use for each core (out of 20 cores) for a 30 FPS webcam showing simple video without processing using imshow. Can it be fit into a single process if it's only showing the frames? I couldn't find documentation on it.
I'm trying to optimize video processing without using a video card and thought the first step would be to not use all 20 CPU cores when showing video without processing, so that later the code can be used on a smaller device like a Raspberry Pi.
I think multiprocessing will work when you are running the function for multiple images at the same time.
Right now you are using a for loop and passing one image at a time to a single process.
Try the following approach and compare the time this takes with your original approach.
if __name__ == '__main__':
p1 = Process(target=Upscale,args(file_list[0:len(file_list)//2]))
p1.start()
p2 = Process(target=Upscale,args(file_list[len(file_list)//2:len(file_list)]))
p2.start()
p1.join()
p2.join()
Inside the Upscale function, make a for loop and loop through the images, and perform the same tasks as before.
Also, note that the function's argument is mentioned separately in "args" of the function "multiprocessing.Process".
P.S This is my first time answering, let me know if something is not clear.
For the output, you can try the following approach, I picked it up online through this https://superfastpython.com/multiprocessing-return-value-from-process/
# example of returning a variable from a process using a value
from random import random
from time import sleep
from multiprocessing import Value
from multiprocessing import Process
# function to execute in a child process
def task(variable):
# generate some data
data = random()
# block, to simulate computational effort
print(f'Generated {data}', flush=True)
sleep(data)
# return data via value
variable.value = data
# protect the entry point
if __name__ == '__main__':
# create shared variable
variable = Value('f', 0.0)
# create a child process process
process = Process(target=task, args=(variable,))
# start the process
process.start()
# wait for the process to finish
process.join()
# report return value
print(f'Returned: {variable.value}')
Multithreading / multiprocessing can be inefficient or even counterproductive when used to optimise the execution of neural networks on CPUs since a network execution may already be multithreaded.
Before any optimisation work you should first profile your code in order to identify what is slow and what can be improved. Here is some relevant questions you can ask yourself during this optimisation process:
- Does executing a single network already max out all the CPU's?
- Is it possible to execute the network on a faster accelerator such as the GPU?
- In the first place, is your network optimised (FP16, vectorisation, etc.)?
- Is OpenCV up to date and compiled to take advantage of all the available compute power?
- What is the memory footprint of a single network and how many could you execute at the same time on the CPU without out of memory errors?
- Is there any unnecessary redundant allocation (such as the one I pointed out in another of your question)?
- Are there obvious operations that would greatly benefit from being run concurrently (such as
imwriteandimreadbeing memory-bound whileupscaleis compute-bound)?
This list is not exhaustive, try to search for methods to profile, benchmark and optimise neural networks executions.
The simplest approach is to use the newer cv2 module that's based on NumPy arrays. That way you don't have to mess with manual pickling. Here's the fix (I just changed 4 lines of code):
import multiprocessing
import cv2
queue_from_cam = multiprocessing.Queue()
def cam_loop(queue_from_cam):
print 'initializing cam'
cap = cv2.VideoCapture(0)
print 'querying frame'
hello, img = cap.read()
print 'queueing image'
queue_from_cam.put(img)
print 'cam_loop done'
cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()
while queue_from_cam.empty():
pass
print 'getting image'
from_queue = queue_from_cam.get()
print 'saving image'
cv2.imwrite('temp.png', from_queue)
print 'image saved'
It appears that the solution was to convert the opencv iplimage object to string, then pickle it before adding it to the queue:
import multiprocessing
import cv
import Image
import pickle
import time
queue_from_cam = multiprocessing.Queue()
def cam_loop(queue_from_cam):
print 'initializing cam'
cam = cv.CaptureFromCAM(-1)
print 'querying frame'
img = cv.QueryFrame(cam)
print 'converting image'
pimg = img.tostring()
print 'pickling image'
pimg2 = pickle.dumps(pimg,-1)
print 'queueing image'
queue_from_cam.put([pimg2,cv.GetSize(img)])
print 'cam_loop done'
cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()
while queue_from_cam.empty():
pass
print 'getting pickled image'
from_queue = queue_from_cam.get()
print 'unpickling image'
pimg = pickle.loads(from_queue[0])
print 'unconverting image'
cv_im = cv.CreateImageHeader(from_queue[1], cv.IPL_DEPTH_8U, 3)
cv.SetData(cv_im, pimg)
print 'saving image'
cv.SaveImage('temp.png',cv_im)
print 'image saved'
I'm working on a gaming bot and i'm thinking about using multiprocessing to separate pyteseract, opencv matchtemplate and normal computing stuff. I want to do this because I would like to use them in parallele because Pytesseract takes like 8 seconds to do it's stuff and matchtemplate takes approximatly 15 seconds and while they work, the game is stillgoing forward. So if i decide to go down that path, what can i expect to happen? Will Pytesseract and matchtemplate work slower, will i they be able to communicate variable with each other? Is there any hardware specification that i should be worried about? I never did it so i would like to know if the end result will be worth the time. I know it's worth learning, i just want to have my project benefit from the work, i'll be putting in learning it.