-------- old - look up answer on bottom --------

Thank @Micka for the many advises, I found the right thing on the way.

Using cudacodec::VideoWriter is not that easy, after compiling I was not able to use it because of this error, and even if I can make it run, the deployment PC does not have a nvidia GPU.

Since I am going to use PCs with AMD CPUs as well, I can't use the cv::CAP_INTEL_MFX for the api-reference parameter of the cv::VideoWriter. But there is also the cv::CAP_OPENCV_MJPEG, which works fine for the MJPG codec (not all video container are supported, I use .avi, sadly .mkv was not working with this configuration). If the user does not use MJPG as a codec I use cv::CAP_ANY, and opencv decides what is to use.

So,

cv::VideoWriter videoWriter(path, cv::CAP_OPENCV_MJPEG, fourcc, fps, *size);

works pretty well, even on my old system.

Unfortunately I never changed the api-reference parameter before, only from ffmpeg to gstreamer, I read in the doc of opencv only the last line "cv::CAP_FFMPEG or cv::CAP_GSTREAMER." and I did not see that there is an "e.g." before... Thank you @Micka to make me read again.

P.S. for my performance problem with cv::imshow I changed from

cv::namedWindow(WINDOW_NAME, cv::WINDOW_NORMAL);

to

cv::namedWindow(WINDOW_NAME, cv::WINDOW_OPENGL);

Which obviously uses OpenGL, and does a better job. Also changing from cv::Mat to cv::UMat can speed up the performance, see here

-------------- EDIT better solution ----------------

Since I still had problems with the OpenCV VideoWriter for some systems, I was looking for another solution. Now I write the frames with FFMPEG. For FFMPEG I can use the GPU or CPU depending on the codec I use. If FFMPEG is installed via snapd (Ubuntu 18.04) it comes with cuda enabled by default:

sudo snap install ffmpeg --devmode

(--devmode is optional, but I had problems writing files on specific location, this was the only way for me to fix it)

And here is my code:

//this string is automatically created in my program, depending on user input and the parameters of the input frames

string ffmpegCommand = "ffmpeg -y -f rawvideo -vcodec rawvideo -framerate 50 -pix_fmt bgr24 -s 1920x1080 -i - -c:v h264_nvenc -crf 14 -maxrate:v 10M -r 50 myVideoFile.mkv";

FILE *pipeout = popen(ffmpegCommand.data(), "w");

int id = metaDataWriter.insertNow(path);

//loop will be stopped from another thread
while (this->isRunning) {
    //this->frames is a stack with cv::Mat elements in the right order
    //it is filled by another thread
    while (!this->frames.empty()) {


        cv::Mat mat = frames.front();
        frames.pop();
        fwrite(mat.data, 1, s, pipeout);

    }  
}

fflush(pipeout);
pclose(pipeout);

So a file (pipeout) is used to write the mat.data to ffmpeg, ffmpeg itself is doing the encoding and file writing. To the parameters:

-y = Overwrite output files without asking

-f = format, in this case used for input rawvideo

-vcodec = codec for input which is rawvideo as well, because the used cv::Mat.data has no compression/codec

-framerate = the input framerate I receive from my grabber card/OpenCv

-pix_fmt = the format of my raw data, in this case bgr24, so 8 bit each channel, because I use a regular OpenCV bgr cv::Mat

-s = size of each frame, in my case 1920x1080

-i = input, in this case we read from the stdinput you can see it here "-", so the file (pipeout) is captured by ffmpeg

-c:v = output codec, so this is to encode the video, here h264_nvenc is used, which is a GPU codec

-r = frame output rate, also 50 in this case myVideoFile.mkv = this is just the name of the file which is produced by ffmpeg, you can change this file and path

Additional parameters for higher quality: -crf 14 -maxrate:v 10M

This works very good for me and uses my hardware acceleration of the GPU or with another codec in charge the CPU. I hope this helps other developers as well.

Check the rate at which you grab frames from the camera and make sure that this rate matches the rate at which you record the frames into the output file.

The frame rate for writing to a file is specified as fps argument to this function:

bool VideoWriter::open(const string& filename, int fourcc, 
           double fps, Size frameSize, bool isColor=true);

As for the camera fps, for some cameras you can determine its frame rate as follows

double fps = cam1.get(CV_CAP_PROP_FPS); 

Or if the camera doesn't support this method you can find its frame rate by measuring average delay between consecutive frames.

Update: If you camera doesn't support cam1.get(CV_CAP_PROP_FPS);, the frame rate can be estimated experimentally. Like this for example:

while(true) {
    int64 start = cv::getTickCount();

    //Grab a frame
    cam1 >> current;

    if(waitKey(3)>=0) {
        break;
    }

    double fps = cv::getTickFrequency() / (cv::getTickCount() - start);
    std::cout << "FPS : " << fps << std::endl;
}

Also, make sure the output video file is open for writing

if ( !writer->isOpened())
{
    cout  << "Could not open the output video for write: " << endl;
    return -1;
}

I resolved my issue after a bit of debugging; it was an issue with VideoWriter being picky on the rate at which frames were fed to it.

You're ignoring two important factors in your code:

Frame count in the while loop:

You want to write 10 seconds of video at 20 frames per second (fps). This gives you a total of 200 frames for the whole video. For this to happen, you'd need to be conscious of the waiting time inside the while loop before each frame is captured and written to file. If you ignore the waiting period, then:

  frameCount = 0
  while( int(time.time() - start_time) < capture_duration ):
      # we assume that all the operations inside the loop take 0 seconds to accomplish.

      frameCount = frameCount+1

  print('Total frames: ',frameCount)

In the above example, you'll notice that by ignoring waiting time, you'd be writing thousands of frames to video file in 10 seconds. Now 10 seconds of frames at 20 fps would give you 200 frames, to achieve this number of frames you'd need 50 milliseconds of waiting period before each frame is written to the file.

  frameCount = 0
  while( int(time.time() - start_time) < capture_duration ):
      # wait 50 milliseconds before each frame is written.
      cv2.waitKey(50)

      frameCount = frameCount+1

  print('Total frames: ',frameCount)

The total number of frames would be about 200 in the above example.

VideoCapture::read() is a blocking I/O call:

The cap.read() function performs two operations, namely VideoCapture::grab() and VideoCapture::retrieve(). This function waits for next frame to be grabbed, then decodes and returns the image. The waiting period depends on your camera fps.

So for example if your camera fps is 6, then in 10 secs you'd have captured 60 frames. You've set 20 fps as your VideoWriter property; 60 frames played as 20 fps gives you about 3 seconds of video.

To see how many frames are captured by your camera in 10 seconds:

  frameCount = 0
  while( int(time.time() - start_time) < capture_duration ):
      # wait for camera to grab next frame
      ret, frame = cap.read()
      # count number of frames
      frameCount = frameCount+1

  print('Total frames: ',frameCount)