CN112819699B - Video processing method, device and electronic equipment - Google Patents

Abstract

The present invention relates to a video processing method, device and electronic device. The method comprises: obtaining an original image frame in a first video; determining a first area in the original image frame, wherein the original image frame comprises a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state; performing enhancement processing on the original image frame to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure state of different areas in the image to a normal exposure state, and the first area and the second area in the target image frame are both in a normal exposure state; and generating a second video according to the target image frame.

Description

Video processing method, device and electronic equipment

Technical Field

The present invention relates to the field of video processing technology, and more specifically, to a video processing method, a video processing device, an electronic device and a computer-readable storage medium.

Background Art

Due to factors such as time and technical limitations, the playback quality of old movies, old TV series, and other old videos is not ideal, such as unclear images and unsmooth playback. Therefore, it is necessary to process old videos to improve their playback quality.

The manual restoration of old films is time-consuming, labor-intensive, costly, and unstable, especially for old films with long durations, such as old TV series. If old films are automatically processed based on algorithms such as image processing, a technical solution that can improve the playback quality of old films as a whole needs to be proposed.

Summary of the invention

An object of the present invention is to provide a new technical solution for video processing.

According to a first aspect of the present invention, there is provided a video processing method, comprising:

Obtaining an original image frame in a first video;

Determining a first area in the original image frame, wherein the original image frame includes a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state;

Performing enhancement processing on the original image frame to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure status of different areas in the image to a normal exposure state, and the first area and the second area in the target image frame are both in the normal exposure state;

A second video is generated according to the target image frame.

Optionally, the performing enhancement processing on the original image frame to obtain a target image frame includes:

By performing contrast enhancement processing on the original image frame, an auxiliary image frame is obtained, wherein a first area in the auxiliary image frame is in a normal exposure state, and a second area in the auxiliary image frame is in an overexposed state;

The original image frame and the auxiliary image frame are subjected to image fusion to obtain the target image frame.

Optionally, the fusing the original image frame and the auxiliary image frame to obtain the target image frame includes:

Acquire illumination weight information of the original image, wherein the illumination weight information is used to indicate an illumination weight corresponding to each pixel in the original image, and the illumination weight is related to the brightness of the corresponding pixel;

The original image and the auxiliary image frame are fused according to the illumination weight information to obtain the target image frame.

Optionally, before obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame, the method further includes:

Performing denoising processing on the original image frame to obtain an updated original image frame, wherein the denoising processing is used to reduce image noise;

Based on the updated original image frame, the step of obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame is performed.

Optionally, before obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame, the method includes:

Performing edge enhancement processing on the original image frame to obtain an updated original image frame, wherein the edge enhancement processing is used to improve the clarity of the contour edge in the image;

Based on the updated original image frame, a step of obtaining an auxiliary image frame by performing contrast enhancement processing on the original image frame is performed.

Optionally, after fusing the original image frame and the auxiliary image frame to obtain the target image frame, the method further includes:

The target image frame is subjected to super-resolution processing, wherein the super-resolution processing is used to improve the resolution of the image.

Optionally, after generating the second video according to the target image frame, the method further includes:

The second video is subjected to super frame rate processing to obtain a third video for playback, wherein the super frame rate processing is used to increase the frame rate of the video.

According to a second aspect of the present invention, there is provided a video processing device, comprising:

An image extraction module, used to obtain an original image frame in the first video;

An image analysis module, configured to determine a first area in the original image frame, wherein the original image frame includes a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state;

an enhancement processing module, configured to perform enhancement processing on the original image frame to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure status of different areas in the image to a normal exposure status, and the first area and the second area in the target image frame are both in a normal exposure status;

The video generation module is used to generate a second video according to the target image frame.

According to a third aspect of the present invention, there is provided an electronic device, comprising:

A memory for storing executable commands;

The processor is used to execute the method described in the first aspect of the present invention under the control of the executable command.

According to a fourth aspect of the present invention, there is provided a computer-readable storage medium, characterized in that executable commands are stored therein, and when the executable commands are executed by a processor, the method described in the first aspect of the present invention is implemented.

The video processing method in this embodiment enhances the original image frame to adjust the underexposed area in the original image frame to a normally exposed state, while preventing the area in the normally exposed state in the original image frame from being overexposed, which is beneficial to improving the image quality of the video.

Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic diagram showing an electronic device that can be used to implement an embodiment of the present invention.

FIG. 2 shows a flow chart of a video processing method according to an embodiment of the present invention.

FIG. 3 shows a flowchart of a specific example of a video processing method according to an embodiment of the present invention.

DETAILED DESCRIPTION

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

<Hardware Configuration>

FIG. 1 is a schematic diagram showing a hardware configuration of an electronic device that can be used to implement an embodiment of the present invention.

The electronic device 1100 may be a portable computer, a desktop computer, a tablet computer, etc.

As shown in FIG1 , the electronic device 1100 may include a processor 1110, a memory 1120, an interface device 1130, a communication device 1140, a display device 1150, and an input device 1160. The processor 1110 may be a central processing unit CPU, a microprocessor MCU, etc. The memory 1120 may include, for example, a ROM (read-only memory), a RAM (random access memory), a non-volatile memory such as a hard disk, etc. The interface device 1130 may include, for example, a USB interface, a headphone interface, etc. The communication device 1140 may be capable of wired or wireless communication, for example. The display device 1150 may be, for example, a liquid crystal display screen, a touch display screen, etc. The input device 1160 may include, for example, a touch screen, a keyboard, etc.

In the embodiments of this specification, the memory 1120 of the electronic device 1100 is used to store instructions, which are used to control the processor 1110 to operate to support the implementation of the video processing method according to any embodiment of this specification. The technician can design instructions according to the scheme disclosed in this specification. How the instructions control the processor to operate is well known in the art, so it will not be described in detail here.

Those skilled in the art should understand that, although multiple devices of the electronic device 1100 are shown in FIG1 , the electronic device 1100 of the embodiment of the present specification may involve only some of the devices, for example, only the processor 1110, the memory 1120, the display device 1150, the input device 1160, etc.

The electronic device 1100 shown in FIG. 1 is merely illustrative and is in no way intended to limit the present description, its application, or uses.

<Method Example>

This embodiment provides a video processing method, which is implemented by, for example, the electronic device 1100 in Fig. 1. As shown in Fig. 2, the method includes the following steps S1100-S1400.

In step S1100, an original image frame in a first video is obtained.

In this embodiment, the first video is a video that needs to have its image quality enhanced, such as an old movie, an old TV series, or the like.

An image frame is the smallest unit that constitutes a video. An image frame is a still picture. Based on the visual persistence effect of the human eye, a video can be formed by playing multiple image frames in sequence quickly. The original image frame in this embodiment is an image frame used to form the first video, and the number of the original image frame is multiple.

In this embodiment, a special video processing tool can be used to obtain the original image frames in the first video. For example, the video processing software FFmpeg can be used to cut the video into multiple image frames. FFmpeg is an open source computer program that can be used to record and convert digital audio and video and convert them into streams. It provides a complete solution for recording, converting and streaming audio and video. In one example, the video can be decomposed into a sequence of pictures through the "ffmpeg -i video.mpgimage%d.jpg" command in the FFmpeg software, thereby obtaining the original image frames in the first video.

In step S1200, a first area is determined in the original image frame, wherein the original image frame includes a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state.

Generally speaking, due to the influence of shooting techniques, environment and other factors, the photos taken will be underexposed or overexposed. Underexposure refers to insufficient exposure, which is manifested as a lack of details in the darker areas of the image, and overexposure refers to overexposure, which is manifested as a lack of details in the brighter areas of the image. Old movies, old TV series and other types of videos often have underexposure problems.

In this embodiment, the original image frame may be first divided into a plurality of sub-regions, and then the exposure state of each sub-region may be determined to determine the region in the underexposed state, that is, the first region.

When dividing the sub-regions, the original image frame may be evenly divided into a plurality of grids, the grid shape being, for example, a square, a regular hexagon, etc., and each grid is regarded as a sub-region.

When dividing the sub-regions, the image contour in the original image frame may be determined first, and the area within each image contour is taken as a sub-region.

When judging the exposure state, the shooting environment and the average value of the pixels in the sub-region may be combined to make a judgment. For example, for an image shot during the day, if the average value of the pixels in the sub-region exceeds 120 and does not exceed 150, the sub-region is judged to be in a normal exposure state, if the average value of the pixels in the sub-region does not exceed 120, the sub-region is judged to be in an under-exposed state, and if the average value of the pixels in the sub-region exceeds 150, the sub-region is judged to be in an over-exposed state.

When judging the exposure state, judgment can also be made based on the brightness distribution of the sub-region. For example, if the peak value of the brightness histogram of the sub-region is concentrated on the right side, the sub-region is judged to be in an overexposed state, if the peak value of the brightness histogram of the sub-region is concentrated on the left side, the sub-region is judged to be in an underexposed state, and if the peak value of the brightness histogram of the sub-region is evenly distributed, the sub-region is judged to be in a normal exposure state.

In step S1300, the original image frame is enhanced to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure status of different areas in the image to a normal exposure state, and the first area and the second area in the target image frame are both in a normal exposure state.

In this embodiment, the original image frame is enhanced, and the enhancement process can adjust the exposure state of different areas of the image to a normal exposure state. The first area in the target image frame corresponds to the first area in the original image frame, and the second area in the target image frame corresponds to the second area in the original image frame.

In one embodiment, step S1300 further includes: performing contrast enhancement processing on the original image frame to obtain an auxiliary image frame, wherein the first area in the auxiliary image frame is in a normal exposure state and the second area in the auxiliary image frame is in an overexposed state; performing image fusion on the original image frame and the auxiliary image frame to obtain a target image frame.

Contrast enhancement is a commonly used processing method in the prior art. This method adjusts the area that was originally in an underexposed state to a normal exposed state, while often adjusting the area that was originally in a normal exposed state to an overexposed state.

In this embodiment, the auxiliary image frame is obtained by performing contrast enhancement processing on the original image frame, wherein the contrast enhancement processing may adopt specific methods such as histogram equalization and grayscale conversion.

In this embodiment, the target image frame is obtained by performing image fusion on the original image frame and the auxiliary image frame, wherein image fusion refers to the process of obtaining the target image by integrating information of two images.

In one embodiment, the image fusion process includes: obtaining illumination weight information of the original image frame, wherein the illumination weight information is used to indicate the illumination weight corresponding to each pixel in the original image frame, and the illumination weight is related to the brightness of the corresponding pixel; performing image fusion on the original image frame and the auxiliary image frame according to the illumination weight information to obtain a target image frame.

The above-mentioned illumination weight information can characterize the brightness and darkness intensity of the original image frame, and the illumination weight can be positively correlated or negatively correlated with the brightness of the corresponding pixel. Optionally, the illumination weight information can be characterized by a weight image, and the size of the weight image is consistent with the size of the original image; taking the illumination weight as an example of being positively correlated with the brightness of the corresponding pixel, the illumination weight corresponding to the darker area (first area) in the original image frame in the weight image is smaller, while the illumination weight corresponding to the brighter area (second area) in the original image frame is larger.

Optionally, the above-mentioned original image frame and weight image can be expressed in the form of an image matrix (two-dimensional matrix), the rows of the image matrix correspond to the height of the image (in pixels), the columns of the image matrix correspond to the width of the image (in pixels), the elements in the image matrix of the original image frame correspond to the pixel values of the pixels of the original image frame, and the elements in the image matrix of the weight image (lighting weights) are related to the brightness of the pixels in the original image frame. The image matrix of the original image frame and the image matrix of the weight image are the same in terms of the number of rows and columns, and the two elements at the same position of the two correspond to the same pixel in the original image.

When the original image frame and the auxiliary image frame are subjected to image fusion, different fusion weights can be assigned to the original image frame and the auxiliary image frame based on the illumination weight information, so that for the area with weak illumination (the first area), the fusion weight of the original image frame is smaller, and the fusion weight of the auxiliary image frame is larger; while for the area with strong illumination (the second area), the fusion weight of the original image frame is larger, and the fusion weight of the auxiliary image frame is smaller. In this way, it can be ensured that the brightness of part of the image in the normal exposure area or overexposed area in the original image remains unchanged, while part of the image in the darker area is enhanced, that is, the darker area (the first area) in the original image is enhanced, and the brighter area (the second area) is not overexposed after image enhancement, and the image color is not distorted, thereby effectively improving the problem that the enhancement effect of the darker area in the image is insufficient, and the brighter area is easily over-enhanced, resulting in color distortion in the prior art.

In one example, the auxiliary image frame and the original image frame may be fused pixel by pixel. For example, for pixel A1 in the original image frame (the pixel value of the pixel is recorded as a1) and pixel A2 in the auxiliary image frame (the pixel value of the pixel is recorded as a2), assuming that the illumination weight of pixel A1 is p, pixel A1 and pixel A2 are fused to obtain pixel A3, then the pixel value of pixel A3 is a3=a1*p+a2*(1-p).

In this embodiment, the illumination weight is related to the brightness of the corresponding pixel point. For example, the illumination weight corresponding to the pixel point with greater brightness is also greater. In one example, the illumination weight of the pixels in the first area of the original image frame is 0, and the illumination weight of the pixels in the second area is 1. In another example, the illumination weight of the pixels in the first area of the original image frame is 0.2, and the illumination weight of the pixels in the second area is 0.7.

It is easy to understand that the illumination weight in this embodiment makes the first area in the original image frame contribute less to the target image frame, while the second area in the original image frame contributes more to the target image frame during the fusion process. Therefore, while the target image frame increases the brightness of the first area in the original image frame, it can also effectively retain the brightness information of the second area in the original image frame to prevent overexposure.

In step S1400 , a second video is generated according to the target image frame.

In this embodiment, the target image frames are spliced into a video in sequence, that is, a processed video, ie, a second video, is obtained.

The video processing method in this embodiment enhances the original image frame to adjust the underexposed area in the original image frame to a normally exposed state, while preventing the area in the normally exposed state in the original image frame from being overexposed, which is beneficial to improving the image quality of the video.

In one embodiment, before obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame, the above-mentioned video processing method also includes: performing denoising processing on the original image frame to obtain an updated original image frame, wherein the denoising processing is used to reduce the noise of the image; based on the updated original image frame, executing the step of obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame.

Image noise refers to unnecessary or redundant interference information in image data. The causes of image noise usually involve multiple aspects. For example, during the image acquisition stage, image noise can be caused by internal reasons such as the photoelectric characteristics, mechanical movement of the equipment, equipment materials, and equipment circuits, as well as external reasons such as electromagnetic interference. After the image is acquired, new noise will be introduced in the transmission and decompression of the image data. Image noise will affect the picture quality of old movies.

In this embodiment, denoising is performed on the noise in the original image frame, and the noise in the image can be removed by using a mean filter, an adaptive Wiener filter, a median filter, a morphological noise filter, a wavelet denoising, or the like.

In one embodiment, performing denoising processing to reduce image noise on the original image frame includes: performing denoising processing to reduce image noise on the original image frame using a pre-trained denoising model.

In this embodiment, based on the neural network model, denoising is performed on the mixed noise formed by acquisition noise, compression noise, etc. in the image frame.

In one embodiment, before performing denoising, a denoising model is generated. The process of generating the denoising model includes: adding noise of random intensity to the original image to obtain a noise image; training a convolutional neural network model according to the original image and the noise image to obtain the denoising model.

In this embodiment, the original image is a picture with high picture quality. The random noise added to the original image is, for example, Gaussian noise, salt and pepper noise, etc.

In this embodiment, a denoising model is generated in the form of a convolutional neural network. Convolutional neural networks (CNN) are a type of feedforward neural network that includes convolution calculations and has a deep structure, and is one of the representative algorithms of deep learning.

In this embodiment, a training data set is formed based on the original image and the noisy image to train the convolutional neural network, wherein the noisy image is used as the input of the convolutional neural network and the original image is used as the output of the neural network. The trained convolutional neural network model is the denoising model.

In one embodiment, before obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame, the above-mentioned video processing method also includes: performing edge enhancement processing on the original image frame to obtain an updated original image frame, wherein the edge enhancement processing is used to improve the clarity of the contour edge in the image; based on the updated original image frame, executing the step of obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame.

In early videos, the edge boundaries of different areas are relatively fuzzy. In this embodiment, edge enhancement processing is performed on the image frame, which is beneficial to improving the clarity of the video.

In this embodiment, edge enhancement processing can be performed by high-pass filtering, spatial domain differentiation, etc. In one example, spatial domain differentiation method is used for edge enhancement processing, and the gradient value is calculated by the gradient modulus operator. Since the grayscale change at the edge is large, the corresponding gradient value is also large, so strengthening the grayscale value of pixels with large gradient values can highlight the details at the edge, thereby achieving the purpose of edge enhancement.

In one embodiment, after the original image frame and the auxiliary image frame are fused to obtain the target image frame, the video processing method further includes: performing super-resolution processing on the target image frame, wherein the super-resolution processing is used to improve the resolution of the image.

Due to the limitation of hardware conditions, the resolution of early videos is limited. In this embodiment, super-resolution processing is performed on the image frame to obtain an image with higher resolution, which is conducive to adapting to current display devices.

In this embodiment, super-resolution processing can be performed on the image frame based on a sparse coding method, a self-model method, a Bayesian method, a pyramid algorithm, a deep learning method, etc.

In one embodiment of the present invention, the super-resolution processing includes: performing super-resolution processing to improve the image resolution of the target image frame after the dark field enhancement processing by using a pre-trained super-resolution model.

In this embodiment, super-resolution processing is performed on the target image frame based on a neural network model.

In one embodiment, before performing super-resolution processing, a super-resolution model is generated. The process of generating the super-resolution model includes: compressing the sample image to obtain a low-resolution image; training a convolutional neural network model according to the sample image and the low-resolution image to obtain the super-resolution model.

In this embodiment, a high-resolution picture is selected as the sample image.

In this embodiment, a low-resolution image is obtained by performing quality compression on the sample image.

In this embodiment, a training data set is formed based on sample images and low-resolution images to train the convolutional neural network, wherein the low-resolution image is used as the input of the convolutional neural network and the sample image is used as the output of the neural network. The trained convolutional neural network model is the super-resolution model.

In one embodiment, after generating the second video according to the target image frame, it also includes: performing super frame rate processing on the second video to obtain a third video for playback, wherein the super frame rate processing is used to increase the frame rate of the video.

The frame rate of early movies is usually low, which will affect the smoothness of video playback. In this embodiment, super frame rate processing is performed on the video to make the video playback smoother.

In this embodiment, a simple frame rate enhancement algorithm, a frame rate enhancement algorithm including motion compensation, a frame rate enhancement algorithm based on an autoregressive model, or the like may be used to perform super frame rate processing.

In one example, a frame averaging method in a simple frame rate enhancement algorithm is used for super frame rate processing, that is, a weighted average result of two adjacent frames of the second video is used as an interpolation frame and added between the two frames, and the original frames of the second video and the interpolation frames obtained by the interpolation processing are spliced in sequence to obtain a third video.

FIG3 shows a specific example of the implementation of the video processing method in this embodiment. Referring to FIG3, the electronic device first uses the FFmpeg video processing tool to cut the video into multiple image frames, that is, execute step S101. For each image frame, the electronic device inputs the image frame into a pre-trained convolutional neural network model to obtain a denoised image frame, that is, execute step S102. For each denoised image frame, the electronic device uses a filtering or matrix-based edge sharpening method to perform edge sharpening processing on the image frame, that is, execute step S103. For each image frame after edge enhancement, the electronic device constructs an auxiliary image through an existing image enhancement method, and fuses the image frame and the auxiliary graphic according to a specific weight to improve the image quality of the weak brightness area in the image frame and keep the brightness of the strong brightness area in the image frame unchanged, that is, execute step S105. For each image frame after dark field enhancement processing, the electronic device inputs the image frame into a pre-trained convolutional neural network to obtain an image frame with higher resolution, that is, execute step S106. For multiple image frames after super-resolution processing, the electronic device inserts a new image frame into it based on the frame rate improvement algorithm of the autoregressive model, that is, execute step S107. For the multiple image frames after the interpolation process, the electronic device stitches the image frames into a video in sequence, thereby obtaining a processed video, that is, executing step S108.

<Device Example>

This embodiment provides a video processing device, including an image extraction module, an image analysis module, an enhancement processing module and a video generation module.

The image extraction module is used to obtain the original image frame in the first video.

The image analysis module is used to determine a first area in an original image frame, wherein the original image frame includes a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state.

The enhancement processing module is used to perform enhancement processing on the original image frame to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure status of different areas in the image to a normal exposure state, and the first area and the second area in the target image frame are both in a normal exposure state.

The video generation module is used to generate a second video according to the target image frame.

In one embodiment, when the enhancement processing module performs enhancement processing on the original image frame to obtain the target image frame, it is used to: obtain an auxiliary image frame by performing contrast enhancement processing on the original image frame, wherein the first area in the auxiliary image frame is in a normal exposure state and the second area in the auxiliary image frame is in an overexposed state; and perform image fusion on the original image frame and the auxiliary image frame to obtain the target image frame.

In one embodiment, when the enhancement processing module fuses the original image frame and the auxiliary image frame to obtain the target image frame, it is used to: obtain the illumination weight information of the original image, wherein the illumination weight information is used to indicate the illumination weight corresponding to each pixel in the original image, and the illumination weight is related to the brightness of the corresponding pixel; perform image fusion on the original image and the auxiliary image frame according to the illumination weight information to obtain the target image frame.

In one embodiment, the video processing device further includes a denoising processing module, which is used to: before performing contrast enhancement processing on the original image frame to obtain the auxiliary image frame, perform denoising processing on the original image frame to obtain an updated original image frame, wherein the denoising processing is used to reduce image noise. The enhancement processing module performs the step of performing contrast enhancement processing on the original image frame to obtain the auxiliary image frame based on the updated original image frame.

In one embodiment, the video processing device further includes an edge enhancement processing module, which is used to: before performing contrast enhancement processing on the original image frame to obtain the auxiliary image frame, perform edge enhancement processing on the original image frame to obtain an updated original image frame, wherein the edge enhancement processing is used to improve the clarity of the contour edge in the image. Based on the updated original image frame, the enhancement processing module performs the step of performing contrast enhancement processing on the original image frame to obtain the auxiliary image frame.

In one embodiment, the video processing device further includes a super-resolution module, which is used to: after fusing the original image frame and the auxiliary image frame to obtain the target image frame, perform super-resolution processing on the target image frame, wherein the super-resolution processing is used to improve the resolution of the image.

In one embodiment, the video processing device also includes a super frame rate module, which is used to: after generating a second video based on a target image frame, perform super frame rate processing on the second video to obtain a third video for playback, wherein the super frame rate processing is used to increase the frame rate of the video.

<Electronic Equipment Embodiment>

This embodiment provides an electronic device, which includes a memory and a processor.

The memory is used to store executable commands.

The processor is used to execute the method described in the method embodiment of the present invention under the control of the executable instructions.

<Computer Readable Storage Medium Embodiment>

This embodiment provides a computer-readable storage medium, which stores executable commands. When the executable commands are executed by a processor, the method described in the method embodiment of the present invention is executed.

The present invention may be a system, a method and/or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.

The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

Various aspects of the present invention are described herein with reference to the flow charts and/or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and/or block diagram and the combination of each box in the flow chart and/or block diagram can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and a part of a module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or the flowchart, and the combination of the boxes in the block diagram and/or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims (8)

1. A video processing method, comprising: Obtaining an original image frame in a first video; Determining a first area in the original image frame, wherein the original image frame includes a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state; Performing enhancement processing on the original image frame to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure status of different areas in the image to a normal exposure state, and the first area and the second area in the target image frame are both in the normal exposure state; Generate a second video according to the target image frame; The step of performing enhancement processing on the original image frame to obtain a target image frame includes: By performing contrast enhancement processing on the original image frame, an auxiliary image frame is obtained, wherein a first area in the auxiliary image frame is in a normal exposure state, and a second area in the auxiliary image frame is in an overexposed state; Acquire illumination weight information of the original image, wherein the illumination weight information is used to indicate the illumination weight corresponding to each pixel in the original image, the illumination weight is related to the brightness of the corresponding pixel, the illumination weight information is represented by a weight image, and the size of the weight image is consistent with the size of the original image; Performing image fusion on the original image and the auxiliary image frame according to the illumination weight information to obtain the target image frame; The performing image fusion on the original image and the auxiliary image frame according to the illumination weight information includes: The auxiliary image frame and the original image frame are fused pixel by pixel based on the following formula: A3=A1*P+A2*(1-P); Among them, a1 represents the pixel value of pixel A1 in the original image frame, a2 represents the pixel value of pixel A2 in the auxiliary image frame, a3 represents the pixel value of pixel A3 obtained by fusion of pixel A1 and pixel A2, and p represents the illumination weight of pixel A1, which is positively correlated with the brightness of the corresponding pixel point. 2. The method according to claim 1, characterized in that before obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame, the method further comprises: Performing denoising processing on the original image frame to obtain an updated original image frame, wherein the denoising processing is used to reduce image noise; Based on the updated original image frame, the step of obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame is performed. 3. The method according to claim 1, characterized in that before obtaining the auxiliary image frame by performing contrast enhancement processing on the original image frame, the method comprises: Performing edge enhancement processing on the original image frame to obtain an updated original image frame, wherein the edge enhancement processing is used to improve the clarity of the contour edge in the image; Based on the updated original image frame, a step of obtaining an auxiliary image frame by performing contrast enhancement processing on the original image frame is performed. 4. The method according to any one of claims 1 to 3, characterized in that after fusing the original image frame and the auxiliary image frame to obtain the target image frame, the method further comprises: The target image frame is subjected to super-resolution processing, wherein the super-resolution processing is used to improve the resolution of the image. 5. The method according to claim 1, characterized in that after generating the second video according to the target image frame, it also includes: The second video is subjected to super frame rate processing to obtain a third video for playback, wherein the super frame rate processing is used to increase the frame rate of the video. 6. A video processing device, comprising: An image extraction module, used to obtain an original image frame in the first video; An image analysis module, configured to determine a first area in the original image frame, wherein the original image frame includes a first area and a second area, the first area in the original image frame is in an underexposed state, and the second area in the original image frame is not in an underexposed state; an enhancement processing module, configured to perform enhancement processing on the original image frame to obtain a target image frame, wherein the enhancement processing is used to adjust the exposure status of different areas in the image to a normal exposure status, and the first area and the second area in the target image frame are both in a normal exposure status; A video generation module, used to generate a second video according to the target image frame; Wherein, the enhanced processing module is specifically used for: By performing contrast enhancement processing on the original image frame, an auxiliary image frame is obtained, wherein a first area in the auxiliary image frame is in a normal exposure state, and a second area in the auxiliary image frame is in an overexposed state; Acquire illumination weight information of the original image, wherein the illumination weight information is used to indicate the illumination weight corresponding to each pixel in the original image, the illumination weight is related to the brightness of the corresponding pixel, the illumination weight information is represented by a weight image, and the size of the weight image is consistent with the size of the original image; Performing image fusion on the original image and the auxiliary image frame according to the illumination weight information to obtain the target image frame; The enhanced processing module is specifically used for: The auxiliary image frame and the original image frame are fused pixel by pixel based on the following formula: A3=A1*P+A2*(1-P); Among them, a1 represents the pixel value of pixel A1 in the original image frame, a2 represents the pixel value of pixel A2 in the auxiliary image frame, a3 represents the pixel value of pixel A3 obtained by fusion of pixel A1 and pixel A2, and p represents the illumination weight of pixel A1, which is positively correlated with the brightness of the corresponding pixel point. 7. An electronic device, comprising: A memory, for storing executable commands; A processor, configured to execute the method according to any one of claims 1 to 5 under the control of the executable command. 8. A computer-readable storage medium, characterized in that executable commands are stored therein, and when the executable commands are executed by a processor, the method according to any one of claims 1 to 5 is implemented.