This comes a bit late, but I was wondering this with my Logitech C920 HD Pro USB-camera on Ubuntu 20.04 and OpenCV. I tried to command the capture session to run Full HD @ 30 FPS but the FPS was fluctuating between 4-5 FPS.

The capture format for my camera defaulted as "YUYV 4:2:2". No matter how I tried to alter the video capture settings, OpenCV did not magically change the video format to match e.g. the desired FPS setting.

When I listed the video formats for my Logitech C920, it revealed:

ubuntu:~$ v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'YUYV' (YUYV 4:2:2)
<clip>
        Size: Discrete 1600x896
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 2304x1296
            Interval: Discrete 0.500s (2.000 fps)
    [1]: 'MJPG' (Motion-JPEG, compressed)
<clip>
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

The solution was to manually command the OpenCV capture device to use the compressed 'MJPG' format:

import numpy as np
import cv2

capture = cv2.VideoCapture(0)
W, H = 1920, 1080
capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'))
capture.set(cv2.CAP_PROP_FPS, 30)

I can reproduce the behavior you describe (i.e. cv::VideoCapture >> image locked to the frame rate of the recorded video) if opencv is compiled without ffmpeg support. If I compile opencv with ffmpeg support, I can read images from file as fast as my computer will allow. I think that in the absence of ffmpeg, opencv uses gstreamer and essentially treats the video file like its playing back a movie.

If you are using Linux, this link shows which packages you must install to get ffmpeg support for opencv.

TL;DR FPS is irrelevant to the problem, probably a performance issue

What FPS is used for? Before you can display a single frame of video, you have to read the data (from an HDD, DVD, Network, Internet or whatever) and decode it. Both of these operations take time, the amount of which differs from system to system, depending on the speed of HDD/Internet, processor speed etc. If we just display each frame as soon as it's ready, the resulting movie speed will, therefore, vary from system to system. That is usually not what we want, so along with the sequence of video frames we get the "frames per second" value (a. k. a. FPS), which tells us how soon we shall display each consecutive frame (once every 1/30-th of a second for 30 FPS, once every 1/60-th of a second for 60 FPS, etc.) If the frame is ready to be displayed but it's too early, we can wait till its time comes. If it's time to display a frame but it's not ready (on an underpowered/too busy system), there's not much we can do (maybe drop frames in some situations). To see the effect for yourself, try changing the FPS value for x2, saving the file and display it with VLC: for the same amount of data and same number of frame, you will notice that the speed of your video has doubled and the time - halved. Try writing each frame twice for your x2 FPS - you will see that the playback speed is back to normal (with double the number of frames and meaningless increase of the file size).

What FPS is not used for? When processing (not displaying) a video, we are not limited by the original FPS, and the processing goes as fast as it can. If your PC can process 1000 frames per second - good, if 1500 - even better. Needless to say, changing the FPS value in the file won't improve your CPU/HDD speed, so if you were only able to process 54 frames per second, you are still going to be able to only process 54 frames per second.

But how can VLC display faster? Assuming you didn't forget to switch from Debug to Release build before measuring time, there are still a number of possibilities: VLC is probably better optimized for the particular task of video playback (OpenCV is not really that fast at some tasks, plus it has to convert each frame to a more general Mat/UMat structure), multithreading (including "double buffering", as mentioned in the comments) is another possible reason, maybe caching as well (e. g. reading a block of data containing many frames from the HDD at once instead of reading and processing frames one by one).

Setting a frame rate doesn't always work like you expect. It depends on two things:

1. What your camera is capable of outputting.
2. Whether the current capture backend you're using supports changing frame rates.

So point (1). Your camera will have a list of formats which it is capable of delivering to a capture device (e.g. your computer). This might be 1920x1080 @ 30 fps or 1920x1080 @ 60 fps and it also specifies a pixel format. The vast majority of consumer cameras do not let you change their frame rates with any more granularity than that. And most capture libraries will refuse to change to a capture format that the camera isn't advertising.

Even machine vision cameras, which allow you much more control, typically only offer a selection of frame rates (e.g. 1, 2, 5, 10, 15, 25, 30, etc). If you want a non-supported frame rate at a hardware level, usually the only way to do it is to use hardware triggering.

And point (2). When you use cv.VideoCapture you're really calling a platform-specific library like DirectShow or V4L2. We call this a backend. You can specify exactly which backend is in use by using something like:

cv2.VideoCapture(0 + cv2.CAP_DSHOW)

There are lots of CAP_X's defined, but only some will apply to your platform (e.g CAP_V4L2 is for Linux only). On Windows, forcing the system to use DirectShow is a pretty good bet. However as above, if your camera only reports it can output 30fps and 60fps, requesting 10fps will be meaningless. Worse, a lot of settings simply report True in OpenCV when they're not actually implemented. You've seen that most of the time reading parameters will give you sensible results though, however if the parameter isn't implemented (e.g exposure is a common one that isn't) then you might get nonsense.

You're better off waiting for a period of time and then reading the last image.

Be careful with this strategy. Don't do this:

while capturing:
    res, image = cap.read()
    time.sleep(1)

you need to make sure you're continually purging the camera's frame buffer otherwise you will start to see lag in your videos. Something like the following should work:

frame_rate = 10
prev = 0

while capturing:

    time_elapsed = time.time() - prev
    res, image = cap.read()

    if time_elapsed > 1./frame_rate:
        prev = time.time()

        # Do something with your image here.
        process_image()

For an application like a hand detector, what works well is to have a thread capturing images and the detector running in another thread (which also controls the GUI). Your detector pulls the last image captured, runs and display the results (you may need to lock access to the image buffer while you're reading/writing it). That way your bottleneck is the detector, not the performance of the camera.