I've had this same problem using openCV video capture with text-detection. It's not the webcam quality, but the fact that openCV can only show you frames as fast as you can process them in your gender detection. The solution that worked for me is to use multi-threading.

You can create a thread for the OpenCV video capture, then another thread for your image processing. The caveat: you can't magically make your image processing happen quicker without making changes the image process itself. It takes as long as it takes. What you can do is allow openCV to work on its own and send the frames to an exchange class, then allow the image processing to grab a frame and work at its own pace as CV2 continues as normal.

Here is a (shortened) version of my class for OCR image processing. You can see that in start() i'm creating a thread pointed at the ocr() process. This is where your gender identification process can go.

class OCR:

    # def __init__(self, exchange: VideoStream, language=None):
    def __init__(self):
        self.exchange = None
        # init stuff for OCR not relevant to my example, but note that it 
        # takes a VideoStream class called exchange which is where this class 
        # grabs frames to process

    def start(self):
        Thread(target=self.ocr, args=()).start()
        return self

    def set_exchange(self, video_stream):
        self.exchange = video_stream

    def ocr(self):

        while not self.stopped:
            if self.exchange is not None:
                frame = self.exchange.frame

                # # # OCR stuff goes here

Now here is the VideoStream class that grabs frames at its own pace, in a different thread. The image processing class (OCR) can then take these frames at it's own pace and the two don't affect each other's performance.

class VideoStream:
    """Class for CV2 video capture. The start() method will create a new 
thread to read the video stream"""
    def __init__(self, src=0):
        self.stream = cv2.VideoCapture(src)
        (self.grabbed, self.frame) = self.stream.read()
        # self._boxes = None
        self.stopped = False

    def start(self):
        Thread(target=self.get, args=()).start()
        return self

    def get(self):
        while not self.stopped:
            (self.grabbed, self.frame) = self.stream.read()

    def get_video_dimensions(self):
        width = self.stream.get(cv2.CAP_PROP_FRAME_WIDTH)
        height = self.stream.get(cv2.CAP_PROP_FRAME_HEIGHT)
        return int(width), int(height)

    def stop_process(self):
        self.stopped = True

Then you can do your CV2 imshow loop as you normally would.

exchange = VideoStream(0).start()
ocr = OCR().start()
ocr.set_exchange(exchange)

 while True:  # Begins a loop for the real-time OCR display
        pressed_key = cv2.waitKey(1) & 0xFF
        if pressed_key == ord('q'):
            stop_stream_ocr(exchange, ocr)
            break

        frame = exchange.frame 

        cv2.imshow("Video Get Frame", frame)
        cps1.increment()

Note that you can't really control what thread CV2 decides to use in its internal workings, but this approach will allow you to display your webcam at its natural fps and while image processing happens in the background.

Answer from raspberry_next on Stack Overflow
🌐
OpenCV
docs.opencv.org › 4.13.0 › dc › d71 › tutorial_py_optimization.html
OpenCV: Performance Measurement and Improvement Techniques
Vectorize the algorithm/code to the maximum extent possible, because Numpy and OpenCV are optimized for vector operations. Exploit the cache coherence. Never make copies of an array unless it is necessary. Try to use views instead. Array copying is a costly operation.
Top answer
1 of 1
4

I've had this same problem using openCV video capture with text-detection. It's not the webcam quality, but the fact that openCV can only show you frames as fast as you can process them in your gender detection. The solution that worked for me is to use multi-threading.

You can create a thread for the OpenCV video capture, then another thread for your image processing. The caveat: you can't magically make your image processing happen quicker without making changes the image process itself. It takes as long as it takes. What you can do is allow openCV to work on its own and send the frames to an exchange class, then allow the image processing to grab a frame and work at its own pace as CV2 continues as normal.

Here is a (shortened) version of my class for OCR image processing. You can see that in start() i'm creating a thread pointed at the ocr() process. This is where your gender identification process can go.

class OCR:

    # def __init__(self, exchange: VideoStream, language=None):
    def __init__(self):
        self.exchange = None
        # init stuff for OCR not relevant to my example, but note that it 
        # takes a VideoStream class called exchange which is where this class 
        # grabs frames to process

    def start(self):
        Thread(target=self.ocr, args=()).start()
        return self

    def set_exchange(self, video_stream):
        self.exchange = video_stream

    def ocr(self):

        while not self.stopped:
            if self.exchange is not None:
                frame = self.exchange.frame

                # # # OCR stuff goes here

Now here is the VideoStream class that grabs frames at its own pace, in a different thread. The image processing class (OCR) can then take these frames at it's own pace and the two don't affect each other's performance.

class VideoStream:
    """Class for CV2 video capture. The start() method will create a new 
thread to read the video stream"""
    def __init__(self, src=0):
        self.stream = cv2.VideoCapture(src)
        (self.grabbed, self.frame) = self.stream.read()
        # self._boxes = None
        self.stopped = False

    def start(self):
        Thread(target=self.get, args=()).start()
        return self

    def get(self):
        while not self.stopped:
            (self.grabbed, self.frame) = self.stream.read()

    def get_video_dimensions(self):
        width = self.stream.get(cv2.CAP_PROP_FRAME_WIDTH)
        height = self.stream.get(cv2.CAP_PROP_FRAME_HEIGHT)
        return int(width), int(height)

    def stop_process(self):
        self.stopped = True

Then you can do your CV2 imshow loop as you normally would.

exchange = VideoStream(0).start()
ocr = OCR().start()
ocr.set_exchange(exchange)

 while True:  # Begins a loop for the real-time OCR display
        pressed_key = cv2.waitKey(1) & 0xFF
        if pressed_key == ord('q'):
            stop_stream_ocr(exchange, ocr)
            break

        frame = exchange.frame 

        cv2.imshow("Video Get Frame", frame)
        cps1.increment()

Note that you can't really control what thread CV2 decides to use in its internal workings, but this approach will allow you to display your webcam at its natural fps and while image processing happens in the background.

Discussions

How to get faster/lighter video decoding - Android/Java - OpenCV
Hello and thank you in advance for your help! I’m building a simple app at Windows using Intellij and gradle that its purpose is to be able to show a couple, or maybe more, videos (mp4) simultaneously. The code is pretty simple, I am using a thread for capturing a frame and then I use the ... More on forum.opencv.org
🌐 forum.opencv.org
0
September 16, 2021
My OpenCV python script takes 22 minutes to finish. How can I make it significantly faster?
Try to reduce the amounts of template matching you need to do. For example, start by estimating the "color" of the card, and then only test against cards of that color. An even better version is to extract features from the cards (SIFT, color histograms, ...) and then find potential candidates by a database lookup. Vocabulary database document matching are search keywords you can try. More on reddit.com
🌐 r/computervision
31
30
March 19, 2022
How to make execution speed faster (python opencv) - Stack Overflow
I wrote a code which devide a image in two part and repeat this action with other images which is in same folder. However, execution speed is very slowlly. Do you have any idea that the speed make ... More on stackoverflow.com
🌐 stackoverflow.com
October 20, 2017
python 3.x - how to make cv2.videoCapture.read() faster? - Stack Overflow
My question : I was working on my computer vision project. I use opencv(4.1.2) and python to implement it. I need a faster way to pass the reading frame into image processing on my Computer(Ubun... More on stackoverflow.com
🌐 stackoverflow.com
🌐
OpenCV-Python Tutorials
opencv24-python-tutorials.readthedocs.io › en › latest › py_tutorials › py_core › py_optimization › py_optimization.html
Performance Measurement and Improvement Techniques — OpenCV-Python Tutorials beta documentation
Vectorize the algorithm/code to the maximum possible extent because Numpy and OpenCV are optimized for vector operations. Exploit the cache coherence. Never make copies of array unless it is needed. Try to use views instead. Array copying is a costly operation.
🌐
Pysource
pysource.com › home › blog › increase opencv speed by 2x with python and multithreading | tutorial
Increase OpenCV speed by 2x with Python and Multithreading | Tutorial - Pysource
October 15, 2024 - Reduced Latency: Since frames are preloaded, the program does not experience any downtime waiting for frames to be fetched from the video file, leading to faster performance. In the CapMultiThreading class, frames are stored in a buffer (e.g., 128 frames). The frame displayed comes directly from this buffer, which is constantly being filled by a separate thread. This eliminates the usual waiting time between frames, dramatically improving FPS. This tutorial uses a basic video file to demonstrate OpenCV speed-up techniques, but the same principles apply to more advanced tasks like object detection, segmentation, or even real-time keypoint recognition.
🌐
YouTube
youtube.com › watch
Boost Computer Vision Performance: Up to 40X Faster Processing with GPU VS CPU in OpenCV C++ - YouTube
Inside my school and program, I teach you my system to become an AI engineer or freelancer. Life-time access, personal help by me and I will show you exactly...
Published: June 16, 2021
🌐
YouTube
youtube.com › watch
OpenCV Build SLOW? Make it faster! - YouTube
OpenCV building slow for you? This video shows off some native builds at work on the new M1 MacBook Air (16Gb model) as compared to the Intel Core i9 16" Mac...
Published: April 16, 2021
🌐
OpenCV
forum.opencv.org › android/java
How to get faster/lighter video decoding - Android/Java - OpenCV
September 16, 2021 - Hello and thank you in advance for your help! I’m building a simple app at Windows using Intellij and gradle that its purpose is to be able to show a couple, or maybe more, videos (mp4) simultaneously. The code is prett…
Find elsewhere
🌐
Reddit
reddit.com › r/computervision › my opencv python script takes 22 minutes to finish. how can i make it significantly faster?
My OpenCV python script takes 22 minutes to finish. How can I make it significantly faster? : r/computervision
March 19, 2022 - Maybe something like pHash or diff hash (average hash) from opencv. Do that ahead of time and keep the data loaded if possible. For the large image, run the same hashing function on each card in the image. Select the closest match in the index to identify each card.
🌐
Stack Overflow
stackoverflow.com › questions › 46848869 › how-to-make-execution-speed-faster-python-opencv
How to make execution speed faster (python opencv) - Stack Overflow
October 20, 2017 - try to rotate using numpy, numpy.rot90(m, 3) -> 3 for 90*3 = 270. The one you are using says that it has some interpolation that for this kind of rotations (divisible by 90 ) you do not need... maybe this is faster ...
🌐
PyImageSearch
pyimagesearch.com › home › blog › faster video file fps with cv2.videocapture and opencv
Faster video file FPS with cv2.VideoCapture and OpenCV - PyImageSearch
April 17, 2021 - To see the results of the read_frames_fast.py script, make sure you download the source code + example video using the “Downloads” section at the bottom of this tutorial. ... Figure 3: Utilizing threading with cv2.VideoCapture and OpenCV leads to higher FPS and a larger throughput rate.
🌐
OpenCV Q&A Forum
answers.opencv.org › question › 176464 › what-compiler-flags-make-opencv-even-more-faster
What compiler flags make opencv even more faster? - OpenCV Q&A Forum
Our application is using opencv and ffmpeg; we want to build the opencv, such that it will leverage all the parallel processing and GPU hardware available under that particular os. Can you please list all compiler level flags which can be used to improve the performance of the opencv while ...
Top answer
1 of 4
5

This comes a bit late, but I was wondering this with my Logitech C920 HD Pro USB-camera on Ubuntu 20.04 and OpenCV. I tried to command the capture session to run Full HD @ 30 FPS but the FPS was fluctuating between 4-5 FPS.

The capture format for my camera defaulted as "YUYV 4:2:2". No matter how I tried to alter the video capture settings, OpenCV did not magically change the video format to match e.g. the desired FPS setting.

When I listed the video formats for my Logitech C920, it revealed:

ubuntu:~$ v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'YUYV' (YUYV 4:2:2)
<clip>
        Size: Discrete 1600x896
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 2304x1296
            Interval: Discrete 0.500s (2.000 fps)
    [1]: 'MJPG' (Motion-JPEG, compressed)
<clip>
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

The solution was to manually command the OpenCV capture device to use the compressed 'MJPG' format:

import numpy as np
import cv2

capture = cv2.VideoCapture(0)
W, H = 1920, 1080
capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'))
capture.set(cv2.CAP_PROP_FPS, 30)
2 of 4
2

In my case, I have set camera like this (in c++ code):

// cam is cv::VideoCapture object
cam.set(cv::CAP_PROP_BUFFERSIZE, 1);
cam.set(cv::CAP_PROP_FPS, 20); // set fps before set fourcc
cam.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('M', 'J', 'P', 'G'));

CAP_PROP_BUFFERSIZE doesn't matter, but CAP_PROP_FPS must be set before CAP_PROP_FOURCC.
By this setup, my camera fps increases from 5 to 20.

🌐
Reddit
reddit.com › r/computervision › speeding up frame capture in opencv
r/computervision on Reddit: Speeding up frame capture in opencv
January 15, 2020 -

I'm working on an application in python using opencv for use in athletic coaching. I'd like it to run at 60fps, but I can't seem to get it to that speed. I've determined that the part that is slowing everything down is the capture process; frames aren't getting from the camera to the cpu fast enough. All testing I've done so far indicates that the processing and the rendering are not the bottleneck; it's specifically the video input that is slowing everything down.

Now, I know it is possible for the camera to run at 60fps, because when I open up the same camera in guvcview (at the same resolution), I can get a full 60fps.

I don't want to share my entire project, but here's the part that manages frame capture: https://pastebin.com/nnx32rQK. My main app calls the get_frame() function in its own thread. I'm running this on an intel NUC6CAYH (celeron CPU, 2.3 ghz), running linux.

Any help would be welcome!

EDIT: I went into more detail below, but it looks like having the cap.read() within the Feed class is a whole lot slower than just accessing cap.read() directly..... and I have no idea why

🌐
SlideShare
slideshare.net › embeddedvision › making-opencv-code-run-fast-a-presentation-from-intel
"Making OpenCV Code Run Fast," a Presentation from Intel
JavaScript is disabled in your browser · Please enable JavaScript to proceed · A required part of this site couldn’t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser
🌐
Top Tech Boy
toptechboy.com › faster-launch-of-webcam-and-smoother-video-in-opencv-on-windows
Faster Launch of WEBcam and Smoother Video in OpenCV on Windows | Technology Tutorials
This is the code we developed in Video Lesson #6 showing how to launch a webcam quicker and with less delay in OpenCV running on windows.
Top answer
1 of 3
11

Traditional techniques for improving image analysis:

  1. Reduce the image to a monochrome sample.
  2. Reduce the range of samples, e.g. from 8-bit monochrome to 4-bit monochrome.
  3. Reduce the size of the image, e.g. 1024x1924 to 64x64.
  4. Reduce the frame rate, e.g 60fps to 5fps.
  5. Perform a higher level function to guess where the target area is with say a lower resolution, then perform the regular analysis on the cropped output, e.g. perform image recognition to locate the hand before determining the gesture.
2 of 3
1

Steve-o's answer is good for optimizing your code efficiency. I recommend adding some logic to monitor execution times to help you identify where to spend efforts optimizing.

OpenCV logic for time monitoring (python):

startTime = cv.getTickCount()
# your code execution
time = (cv.getTickCount() - startTime)/ cv.getTickFrequency()

Boost logic for time monitoring:

boost::posix_time::ptime start = boost::posix_time::microsec_clock::local_time();
// do something time-consuming
boost::posix_time::ptime end = boost::posix_time::microsec_clock::local_time();

boost::posix_time::time_duration timeTaken = end - start;
std::cout << timeTaken << std::endl;

How you configure your OpenCV build matters a lot in my experience. IPP isn't the only option to give you better performance. It really is worth kicking the tires on your build to get better hardware utilization.

The other areas to look at are CPU and memory utilization. If you watch your CPU and/or memory utilization, you'll probably find that 10% of your code is working hard and the rest of the time things are largely idle.

  • Consider restructuring your logic as a pipeline using threads so that you can process multiple images at once (if you're tracking and need the results of previous images, you need to break up your code into multiple segments such as preprocessing/analysis and use a std::queue to buffer between them, and imshow won't work from worker threads so you'll need to push result images into a queue and imshow from the main thread)
  • Consider using persistent/global objects for things like kernels/detectors that don't need to get recreated each time
  • Is your throughput slowing down the longer your program runs? You may need to look at how you are handling disposing of images/variables within the main loop's scope
  • Segmenting your code in functions makes it more readable, easier to benchmark, and descopes variables earlier (temporary Mat and results variables free up memory when descoped)
  • If you're doing low-level processing on Mat pixels where you iterate over a large portion of the image, use a single parallel for and avoid writing
  • Depending on how you are running your code, you may be able to disable debugging to get better performance
  • If you're streaming and dumping frames, prefer changing the camera settings to throttle the streaming rate instead of dumping frames
  • If you're converting from1 12 to 8 bits or only using a region of your image, prefer doing this at the camera hardware level

Here's an example of a parallel for loop:

cv::parallel_for_(cv::Range(0, img.rows * img.cols), &
{
    for (int r = range.start; r < range.end; r++)
    {
        int x = r / img.rows;
        int y = r % img.rows;
        uchar pixelVal = img.at<uchar>(y, x);
        //do work here
    }
});

If you're hardware constrained (ie fully utilizing CPU and/or memory), then you need to look at priotizing your process/OS perfomance optimizations/freeing system resources/upgrading your hardware

  • Increase the priority of the process to be more greedy with respect to other programs running on the computer (in linux you have nice(int inc) in unistd.h, in windows SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS) in windows.h)
  • Optimize your power settings for maximum performance in general
  • Disable CPU core parking
  • Optimize your acquisition hardware settings (increase rx/tx buffers, etc) to offload work from your CPU
🌐
Reddit
reddit.com › r/opencv › [question]how to improve the speed of opencv to render images real time?
r/opencv on Reddit: [Question]How to improve the speed of Opencv to render images real time?
November 20, 2022 -

Hi guys, I have this code and the execution works, but it is a bit slow with pressing a key with the condition if red goes in white. How can I make it more instant? Should I use a gpu with cuda? How do I make use of a GPU instead of CPU with below code? The program is used to automate skill checks for this simulation website: https://www.mistersyms.com/tools/gitgud/

import cv2 import numpy as np import mss from pynput.keyboard import Controller

with mss.mss() as sct: monitor = {"top": 470, "left": 890, "width": 140, "height": 140} low_white = np.array([253, 253, 253]) high_white = np.array([255, 255, 255])

low_red = np.array([160, 0, 0])
high_red = np.array([255, 30, 30])
keyboard = Controller()

cordsw = []

while True:
    img = np.array(sct.grab(monitor))
    rgb_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    maskw = cv2.inRange(rgb_image, low_white, high_white)
    maskr = cv2.inRange(rgb_image, low_red, high_red)

    cordsr = []

    yw, xw = np.where(maskw != 0)
    yr, xr = np.where(maskr != 0)

    for i in range(len(yw)):
        cordsw.append([yw[i], xw[i]])
    for i in range(len(yr)):
        cordsr.append([yr[i], xr[i]])

    for i in range(len(cordsr)):
        if cordsr[i] in cordsw:
            keyboard.press("space")
            keyboard.release("space")
            cordsw = []
            break

    if len(yw) == 0 and len(yr) == 0:
        cordsw = []
    if len(cordsr) == 0:
        cordsw = []

    if cv2.waitKey(25) & 0xFF == ord("q"):
        break
🌐
Stack Overflow
stackoverflow.com › questions › 48557550 › opencv-how-can-i-increase-performance-on-image-processing
c++ - openCV: How can I increase performance on image processing? - Stack Overflow
February 1, 2018 - This is not very easy to learn, but it's definitely worth the effort. Another way to achieve great speed is to use the GPU for such tasks. GPUs are highly parallel and even faster than multi threading on a CPU.
🌐
PyImageSearch
pyimagesearch.com › home › blog › optimizing opencv on the raspberry pi
Optimizing OpenCV on the Raspberry Pi - PyImageSearch
April 17, 2021 - If your algorithm is running too slow you should consider (1) reducing the frame size of your input images to achieve faster throughput or (2) utilizing faster hardware. ... Thanks for the reply Adrian.