| OpenCV 4.3.0 Open Source Computer Vision

Upscaling video

In this tutorial you will learn how to use the 'dnn_superres' interface to upscale video via pre-trained neural networks.

## Building

When building OpenCV, run the following command to build all the contrib module:

cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/

Or only build the dnn_superres module:

cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/dnn_superres

Or make sure you check the dnn_superres module in the GUI version of CMake: cmake-gui.

## Source Code of the sample

1 // This file is part of OpenCV project.
2 // It is subject to the license terms in the LICENSE file found in the top-level directory
3 // of this distribution and at http://opencv.org/license.html.
4
5 #include <iostream>
6
7 #include <opencv2/dnn_superres.hpp>
8
9 #include <opencv2/imgproc.hpp>
10 #include <opencv2/highgui.hpp>
11
12 using namespace std;
13 using namespace cv;
14 using namespace dnn_superres;
15
16 int main(int argc, char *argv[])
17 {
18 // Check for valid command line arguments, print usage
19 // if insufficient arguments were given.
20 if (argc < 4) {
21 cout << "usage: Arg 1: input video path" << endl;
22 cout << "\t Arg 2: output video path" << endl;
23 cout << "\t Arg 3: algorithm | edsr, espcn, fsrcnn or lapsrn" << endl;
24 cout << "\t Arg 4: scale | 2, 3, 4 or 8 \n";
25 cout << "\t Arg 5: path to model file \n";
26 return -1;
27 }
28
29 string input_path = string(argv[1]);
30 string output_path = string(argv[2]);
31 string algorithm = string(argv[3]);
32 int scale = atoi(argv[4]);
33 string path = string(argv[5]);
34
35 VideoCapture input_video(input_path);
36 int ex = static_cast<int>(input_video.get(CAP_PROP_FOURCC));
37 Size S = Size((int) input_video.get(CAP_PROP_FRAME_WIDTH) * scale,
38 (int) input_video.get(CAP_PROP_FRAME_HEIGHT) * scale);
39
40 VideoWriter output_video;
41 output_video.open(output_path, ex, input_video.get(CAP_PROP_FPS), S, true);
42
43 if (!input_video.isOpened())
44 {
45 std::cerr << "Could not open the video." << std::endl;
46 return -1;
47 }
48
49 DnnSuperResImpl sr;
50 sr.readModel(path);
51 sr.setModel(algorithm, scale);
52
53 for(;;)
54 {
55 Mat frame, output_frame;
56 input_video >> frame;
57
58 if ( frame.empty() )
59 break;
60
61 sr.upsample(frame, output_frame);
62 output_video << output_frame;
63
64 namedWindow("Upsampled video", WINDOW_AUTOSIZE);
65 imshow("Upsampled video", output_frame);
66
67 namedWindow("Original video", WINDOW_AUTOSIZE);
68 imshow("Original video", frame);
69
70 char c=(char)waitKey(25);
71 if(c==27)
72 break;
73 }
74
75 input_video.release();
76 output_video.release();
77
78 return 0;
79 }
cv::WINDOW_AUTOSIZE
the user cannot resize the window, the size is constrainted by the image displayed.
Definition: highgui.hpp:184
cv::VideoWriter
Video writer class.
Definition: videoio.hpp:856
std
STL namespace.
highgui.hpp
cv::imshow
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
cv
"black box" representation of the file storage associated with a file on disk.
Definition: affine.hpp:51
cv::VideoWriter::open
virtual bool open(const String &filename, int fourcc, double fps, Size frameSize, bool isColor=true)
Initializes or reinitializes video writer.
cv::VideoCapture
Class for video capturing from video files, image sequences or cameras.
Definition: videoio.hpp:620
cv::Size_
Template class for specifying the size of an image or rectangle.
Definition: types.hpp:315
cv::CAP_PROP_FOURCC
4-character code of codec. see VideoWriter::fourcc .
Definition: videoio.hpp:138
cv::CAP_PROP_FRAME_WIDTH
Width of the frames in the video stream.
Definition: videoio.hpp:135
cv::CAP_PROP_FRAME_HEIGHT
Height of the frames in the video stream.
Definition: videoio.hpp:136
cv::quality::quality_utils::scale
void scale(cv::Mat &mat, const cv::Mat &range, const T min, const T max)
Definition: quality_utils.hpp:90
cv::namedWindow
void namedWindow(const String &winname, int flags=WINDOW_AUTOSIZE)
Creates a window.
cv::Size
Size2i Size
Definition: types.hpp:347
dnn_superres.hpp
imgproc.hpp
cv::CAP_PROP_FPS
Frame rate.
Definition: videoio.hpp:137
cv::Mat
n-dimensional dense array class
Definition: mat.hpp:791
cv::waitKey
int waitKey(int delay=0)
Waits for a pressed key.

## Explanation

1. Set header and namespaces
   #include <opencv2/dnn_superres.hpp>
   using namespace std;
   using namespace cv;
   using namespace dnn_superres;
2. Create the Dnn Superres object
   DnnSuperResImpl sr;
   Instantiate a dnn super-resolution object.
3. Read the model
   path = "models/ESPCN_x2.pb"
   sr.readModel(path);
   sr.setModel("espcn", 2);
   Read the model from the given path and sets the algorithm and scaling factor.
4. Upscale a video
   for(;;)
   {
   Mat frame, output_frame;
   input_video >> frame;
   if ( frame.empty() )
   break;
   sr.upsample(frame, output_frame);
   ...
   }
   Process and upsample video frame by frame.