CN110796600A - Image superdivision reconstruction method, image superdivision reconstruction device and electronic equipment - Google Patents

Abstract

The present application discloses an image super-resolution reconstruction method, an image super-resolution reconstruction device, an electronic device and a computer-readable storage medium, wherein the method includes: acquiring a preview image; if there is a target to be processed in the preview image, The preview image is divided to obtain a to-be-processed image including the to-be-processed object and a scene image not including the to-be-processed target, wherein the to-be-processed target is a target that satisfies a preset condition; Perform super-resolution reconstruction on the image to obtain a processed image; and fuse the processed image with the scene image to obtain a new preview image. Through the solution of the present application, the definition of the shooting target can be improved in a targeted manner, and at the same time, the amount of data processed by the electronic device can be reduced.

Description

Image superdivision reconstruction method, image superdivision reconstruction device and electronic equipment

technical field

The present application belongs to the technical field of image processing, and in particular, relates to an image superdivision reconstruction method, an image superdivision reconstruction device, an electronic device, and a computer-readable storage medium.

Background technique

When people use electronic equipment to shoot in harsh shooting scenes, the shooting target is often not clear enough. In order to ensure the clarity of the captured image, the user can perform super-resolution reconstruction on the entire captured image through an electronic device. However, this super-resolution reconstruction method is difficult to achieve targeted processing of the shooting target.

SUMMARY OF THE INVENTION

The present application provides an image super-division reconstruction method, an image super-division reconstruction device, an electronic device and a computer-readable storage medium, which can specifically improve the clarity of a shooting target.

In a first aspect, an embodiment of the present application provides an image super-resolution reconstruction method, including:

get a preview image;

If there is an object to be processed in the preview image, segment the preview image to obtain an image to be processed including the object to be processed and a scene image not including the object to be processed, wherein the object to be processed To meet the target of the preset conditions;

Perform super-resolution reconstruction on the to-be-processed image to obtain a processed image;

The processed image and the scene image are fused to obtain a new preview image.

In a second aspect, an embodiment of the present application provides an image superdivision reconstruction device, including:

The acquisition unit is used to acquire the preview image;

a segmentation unit, configured to segment the preview image if there is an object to be processed in the preview image to obtain an image to be processed including the object to be processed and a scene image not including the object to be processed, wherein, The to-be-processed target is a target that satisfies a preset condition;

a processing unit, configured to perform super-resolution reconstruction on the to-be-processed image to obtain a processed image;

A fusion unit, configured to fuse the processed image with the scene image to obtain a new preview image.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program The method as described in the first aspect above is implemented.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to the first aspect is implemented.

In a fifth aspect, an embodiment of the present application further provides a computer program product, which implements the method described in the first aspect when the computer program product is run on an electronic device.

It can be seen from the above that in the solution of the present application, after the electronic device obtains the preview image, if there is a target to be processed in the above-mentioned preview image, the preview image will be segmented, so as to obtain the to-be-processed image including the target to be processed and the non-processed image. The scene image containing the target to be processed, and only the to-be-processed image is subjected to super-resolution reconstruction, which reduces the amount of processing data during super-resolution reconstruction, and finally fuses the processed image with the scene image to obtain a new image. The preview image can be targeted to improve the clarity of the shooting target.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present application. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a schematic diagram of an implementation flowchart of an image superdivision reconstruction method provided by an embodiment of the present application;

2-1 is a schematic diagram of a target detection frame and a to-be-processed image frame in the image super-score reconstruction method provided by the embodiment of the present application;

2-2 is another schematic diagram of a target detection frame and a to-be-processed image frame in the image super-score reconstruction method provided by the embodiment of the present application;

3-1 is a schematic diagram of an image to be processed in the image super-resolution reconstruction method provided by an embodiment of the present application;

3-2 is a schematic diagram of a scene image in the image super-resolution reconstruction method provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an overlapping area in the image superdivision reconstruction method provided by an embodiment of the present application;

5 is a schematic diagram of an image superdivision reconstruction apparatus provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

In order to illustrate the above-mentioned technical solutions of the present application, the following specific embodiments are used for description.

The image super-resolution reconstruction method in the embodiments of the present application can be applied to electronic devices such as smart phones, tablet computers, and digital cameras, which are not limited here. Taking the application of the image super-resolution reconstruction method to a smartphone as an example, an image super-resolution reconstruction method provided by an embodiment of the present application is described below, referring to FIG. 1 , including:

Step 101, obtaining a preview image;

In this embodiment of the present application, an image capture operation may be performed through a camera mounted on the electronic device to obtain a preview image, wherein the camera may be a front camera or a rear camera, which is not limited here.

Step 102, if there is a to-be-processed object in the above-mentioned preview image, the above-mentioned preview image is divided to obtain a to-be-processed image including the above-mentioned to-be-processed target and a scene image that does not include the above-mentioned to-be-processed target;

In this embodiment of the present application, the above-mentioned target to be processed is a target that satisfies a preset condition. Optionally, after obtaining the preview image, the preview image can be displayed on the screen of the above-mentioned electronic device, and if a user's click instruction on the preview image is received, the target at the input coordinate position of the above-mentioned click instruction can be displayed. It is determined as the target to be processed; alternatively, the electronic device can also intelligently detect whether there is a target to be processed in the preview image, which is not limited here. That is, the above-mentioned target to be processed may be determined by the user, or may be determined intelligently by the electronic device. When it is determined that there is an object to be processed in the preview image, the preview image can be segmented to obtain an image to be processed, and the image to be processed includes the object to be processed; at the same time, a scene image can be obtained. Contains the above pending targets.

Step 103, performing super-resolution reconstruction on the above-mentioned to-be-processed image to obtain a processed image;

In this embodiment of the present application, in order to achieve targeted processing of the target to be processed, only the above-mentioned to-be-processed image containing the target to be processed is subjected to super-score reconstruction processing. Specifically, the above-mentioned to-be-processed image can be processed through a preset super-resolution algorithm to obtain a super-resolution processed image, where the width and height of the super-resolution processed image are both N times the original to-be-processed image. , the value of N is 2 or 4; then, the bilinear interpolation method is used for the super-resolution processed image to obtain an image of the same size as the image to be processed, which is the same as the image to be processed. The processed image after super-resolution reconstruction.

Step 104 , fuse the above-mentioned processed image with the above-mentioned scene image to obtain a new preview image.

In the embodiment of the present application, after the above-mentioned processed image is obtained, the above-mentioned processed image and the above-mentioned scene image can be fused, the fused image is a new preview image, and the new preview image is displayed on the electronic device screen for the user to view. Since the size of the processed image is exactly the same as that of the to-be-processed image, and the to-be-processed image is obtained by segmenting the original preview image, the location of the to-be-processed image in the original preview image and the scene image in the original preview image can be The position in the image to realize the fusion of the processed image and the scene image. Optionally, after a new preview image is obtained, the screen of the electronic device no longer displays the original preview image, but displays the above-mentioned new preview image.

Optionally, considering that when the electronic device is shooting in the night scene, the light in the night scene is dark, which makes it difficult for the user to capture a clear image. Therefore, the above-mentioned image super-resolution reconstruction method can be optimized based on the application scenario of the night scene, then After the above-mentioned step 101, the above-mentioned image super-resolution reconstruction method includes:

A1. Detect whether the shooting scene of the above-mentioned preview image is a night scene;

Wherein, after acquiring the preview image, the electronic device may analyze the grayscale information of the preview image to determine whether the shooting scene of the preview image is a night scene. Specifically, the above step A1 includes:

B1. Calculate the grayscale average value of the above-mentioned preview image;

Wherein, after the gray value of each pixel point of the preview image is acquired, the average value calculation of the gray value of these pixel points may be continued to obtain the gray average value of the preview image.

B2, compare the grayscale average value of the above-mentioned preview image with the preset first grayscale average value threshold;

The electronic device may preset a first grayscale average threshold. Of course, the first grayscale average threshold can also be changed by the user according to actual needs, which is not limited here.

B3. If the grayscale average value of the above-mentioned preview image is less than the above-mentioned first grayscale average value threshold, then determine that the shooting scene of the above-mentioned preview image is a night scene;

B4. If the grayscale average value of the preview image is not less than the first grayscale average value threshold, it is determined that the shooting scene of the preview image is not a night scene.

Wherein, when the grayscale value is 0, it is completely black, and when the grayscale value is 255, it is completely white. Therefore, the smaller the grayscale average value of the preview image is, the darker the shooting scene of the preview image is. When the grayscale average value of the preview image is less than the first grayscale average value threshold, it can be determined that the shooting scene of the preview image is a night scene; when the grayscale average value of the preview image is not less than the first grayscale average value When the threshold value is set, it can be determined that the shooting scene of the above-mentioned preview image is not a night scene.

A2. If the shooting scene of the preview image is a night scene, detect whether there is an object to be processed in the preview image.

Wherein, when it is determined that the shooting scene of the preview image is a night scene, it can be detected whether there is a target to be processed in the preview image, that is, a target that satisfies a preset condition. Specifically, the step of detecting whether there is an object to be processed in the preview image includes:

C1, performing target detection on the above-mentioned preview image to obtain more than one target contained in the above-mentioned preview image;

Wherein, when it is determined that the shooting scene of the preview image is a night scene, target detection may be further performed on the preview image to obtain one or more targets included in the preview image. Considering that there are many types of the above targets, and the user may only care about some types of targets, therefore, after the target detection is performed on the above preview image, the obtained targets can be screened, and only the ones that the user is interested in are retained. target type. For example, in the daily shooting process, people are the most common shooting objects. Therefore, the target type that the user is interested in can be set as a human face. In this application scenario, the above step C1 can be embodied as a preview of the The image is subjected to face detection to obtain one or more faces contained in the preview image. Of course, the user can also modify the above-mentioned target type of interest by himself according to specific shooting requirements, which is not limited here.

C2. Calculate the average gray level of all targets;

Wherein, after obtaining more than one target included in the above-mentioned preview image, the gray value of the pixel points of each target can be obtained, and then the average value of the gray value of these pixel points can be calculated to obtain the average gray value of all targets. value. It should be noted that the gray average value is not calculated on a single target, but the gray average value is calculated on all targets as a whole.

C3, compare the grayscale average value of all targets with the preset second grayscale average value threshold;

The electronic device can preset a second grayscale average threshold. Of course, the second grayscale average threshold can also be changed by the user according to actual needs, which is not limited here.

C4. If the average grayscale value of all the targets is less than the above-mentioned second threshold value of the grayscale average value, all the targets are determined as the above-mentioned target to be processed.

Wherein, if the average grayscale value calculated based on all the above-mentioned targets is smaller than the above-mentioned second grayscale average value threshold, it is considered that the brightness of these targets in the above-mentioned preview image is relatively dark, and it is difficult to present a good shooting experience to the user. Based on this , these targets can be determined as pending targets. For example, assuming that in the shooting scene of a night scene, the target type of interest to the user is a human face, the electronic device will detect whether there is a human face in the preview image, and if there are multiple human faces, continue to calculate the grayscale average value of the multiple human faces , and compared with the above-mentioned second grayscale average value threshold, if the grayscale average value of these multiple faces is less than the above-mentioned second grayscale average value threshold, then these multiple human faces are all determined as the target to be processed.

Optionally, the above-mentioned steps of dividing the above-mentioned preview image to obtain the to-be-processed image including the above-mentioned target to be processed and the scene image not including the above-mentioned target to be processed specifically include:

D1. Obtain the target detection frame of the above-mentioned target to be processed;

Among them, in the process of target detection, a plurality of target detection frames will be generated. Therefore, in this step, the target detection frame of the above-mentioned target to be processed can be obtained and used as the basis for subsequent segmentation. Of course, if the above target to be processed is determined by the user, after determining the target to be processed based on the click instruction input by the user, a target detection frame can be generated to frame the target to be processed. Generally, the target detection frame is a rectangle. Of course, according to different algorithms used for target detection, the above target detection frame may also be a polygon, which is not limited here. Specifically, when the target is a face, the above-mentioned target detection frame is the face detection frame.

D2. Based on the above-mentioned target detection frame, set the to-be-processed image frame in the above-mentioned preview image;

The shape of the image frame to be processed is the same as the shape of the target detection frame, and the size of the image frame to be processed is larger than the size of the target detection frame, and each boundary of the image frame to be processed corresponds to the target detection frame. The boundaries are parallel, and each boundary of the image frame to be processed is separated from the corresponding boundary of the target detection frame by a preset distance. As shown in Figure 2-1 and Figure 2-2, Figure 2-1 is a schematic diagram of the corresponding image frame to be processed when the target detection frame is a rectangle; Figure 2-2 is when the target detection frame is a hexagon , a schematic diagram of the corresponding set to-be-processed image frame. It can be seen that the distance between the target detection frame and the correspondingly set image frame to be processed is kept as a fixed value.

D3, determining the image within the above-mentioned to-be-processed image frame as the to-be-processed image;

Wherein, based on the preview image, after the image frame to be processed is set in the preview image, the image in the image frame to be processed is determined as the image to be processed, that is, the image frame to be processed is the image to be processed the edge of. Taking the target detection frame as a rectangle as an example, as shown in Figure 3-1, the shadow part outside the frame of the image to be processed is the shadow part. After the shadow part is removed, the image to be processed is what remains.

D4. Determine the image outside the target detection frame as a scene image.

Wherein, based on the preview image, the image outside the target detection frame is determined as the scene image, that is, the target detection frame is the inner edge of the image to be processed, and the original edge of the preview image is the outer edge of the image to be processed. Still taking the target detection frame as a rectangle as an example, as shown in Figure 3-2, the above target detection frame is the shadow part. After the shadow part is removed, the scene image remains.

Optionally, in order to better fuse the scene image and the processed image, the above-mentioned image superdivision reconstruction method further includes:

E1, obtain the coordinates of each vertex of the above-mentioned to-be-processed image frame in the above-mentioned preview image;

The processed image is actually an image obtained by performing a super-resolution reconstruction operation on the to-be-processed image. Therefore, the processed image and the to-be-processed image have exactly the same shape and size. In order to realize the fusion of the processed image and the scene image, the coordinates of each vertex of the image frame to be processed in the preview image can be obtained first, and the coordinates of each vertex of the image frame to be processed in the preview image are the processed image. The coordinates of each vertex in the preview image. It should be noted that the coordinates are obtained based on the image coordinate system, that is, the coordinate system constructed in pixels with the upper left vertex of the image as the origin of the coordinate system, the abscissa u and the ordinate of the pixel. v is the number of columns and rows in the image array, respectively.

Correspondingly, the above step 104 includes:

E2, based on the coordinates of each vertex of the above-mentioned to-be-processed image frame in the above-mentioned preview image, overlapping the above-mentioned processed image and the above-mentioned scene image to obtain an overlapping area;

Wherein, since the outer edge of the scene image is the edge of the original preview image, the image coordinate systems of the preview image and the scene image are completely coincident. Based on this, according to the coordinates of each vertex of the image frame to be processed in the preview image, the coordinates of each vertex of the processed image in the scene image can be determined. As shown in FIG. 4 , the solid line part constitutes the scene image, and the dotted line part constitutes the processed image. There is an overlapping area between the scene image and the processed image, that is, the shadow part. In fact, when performing image segmentation, it can be seen that the target detection frame constitutes the inner edge of the overlapping area, and the image frame to be processed constitutes the outer edge of the overlapping area.

E3. Based on the overlapping area, fuse the edge of the scene image and the processed image to obtain a new preview image.

The parts outside the overlapping area do not need to be further processed, that is, in the scene image, the pixels outside the overlapping area remain unchanged; in the processed image, the pixels outside the overlapping area also remain unchanged. Only the pixels in the overlapping area are fused, so that the inner edge area of the scene image can be fused with the edge area of the processed image, so as to obtain a new preview image. Specifically, the above step E3 includes:

F1. For any pixel in the overlapping area, obtain the gray value of the pixel in the scene image, denoted as the first gray value, and obtain the gray value of the pixel in the processed image, is recorded as the second gray value;

Among them, since the overlapping area exists in both the scene image and the processed image, for any pixel in the overlapping area, the gray value of the pixel in the scene image will be obtained, and record is the first gray value, and at the same time, the gray value of the above-mentioned pixel point in the above-mentioned processed image is obtained, which is recorded as the second gray value, which is used as the basis for subsequent fusion.

F2. Calculate the grayscale average value of the first grayscale value and the second grayscale value;

F3. Determine the grayscale average value of the first grayscale value and the second grayscale value as the grayscale value of the pixel point after fusion.

The above steps F1 to F3 are described below with specific examples: Assume that the gray value of a pixel point P1 in the overlapping area is X1 in the scene image, and the gray value in the processed image is X2, and the gray value of X1 and X2 is obtained. The average value of the grayscale is rounded to obtain the grayscale value X3, and the grayscale value of the above-mentioned pixel point after fusion is X3. Through the above process, each pixel point in the overlapping area is obtained by fusion based on the scene image and the corresponding pixel point of the processed image. In this way, the finally obtained new preview image actually consists of three parts: one is the part of the scene image outside the image frame to be processed without any processing; The third is the part of the overlapping area of the scene image and the processed image that is fused between the image frame to be processed and the target detection frame.

As can be seen from the above, through the embodiment of the present application, after the preview image is obtained, if there is an object to be processed in the preview image, the preview image will be segmented to obtain an image to be processed that includes the object to be processed and an image that does not include the object to be processed. The scene image of the target to be processed, and only the to-be-processed image is subjected to super-resolution reconstruction, which reduces the amount of processing data during super-resolution reconstruction, and finally the above-mentioned processed image and the above-mentioned scene image are fused to obtain a new preview image. Targeted enhancement of the clarity of the shooting target.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Corresponding to the image super-division reconstruction method proposed above, an image super-division reconstruction apparatus provided by an embodiment of the present application is described below. Please refer to FIG. 5 . The above-mentioned image super-division reconstruction apparatus 5 includes:

an acquisition unit 501, used to acquire a preview image;

The segmentation unit 502 is configured to segment the preview image if there is an object to be processed in the preview image to obtain an image to be processed including the object to be processed and a scene image that does not include the object to be processed, wherein the object to be processed is obtained. The target is the target that satisfies the preset conditions;

A processing unit 503, configured to perform super-resolution reconstruction on the above-mentioned to-be-processed image to obtain a processed image;

The fusion unit 504 is configured to fuse the above-mentioned processed image and the above-mentioned scene image to obtain a new preview image.

Optionally, the above-mentioned image super-division reconstruction device 5 also includes:

a night scene detection unit, configured to detect whether the shooting scene of the above-mentioned preview image is a night scene after the above-mentioned acquisition of the preview image;

The to-be-processed target detection unit is configured to detect whether there is a to-be-processed target in the above-mentioned preview image if the shooting scene of the above-mentioned preview image is a night scene.

Optionally, the above-mentioned night scene detection unit includes:

a first calculation subunit, used for calculating the grayscale average value of the above-mentioned preview image;

a first comparison subunit, configured to compare the grayscale average value of the above-mentioned preview image with a preset first grayscale average value threshold;

a night scene judgment subunit, configured to determine that the shooting scene of the preview image is a night scene if the average grayscale value of the preview image is less than the first grayscale average value threshold;

The night scene judging subunit is further configured to determine that the shooting scene of the preview image is not a night scene if the average grayscale value of the preview image is not less than the first grayscale average value threshold.

Optionally, the above-mentioned target detection unit to be processed includes:

a target detection subunit, configured to perform target detection on the above-mentioned preview image to obtain one or more targets contained in the above-mentioned preview image;

The second calculation subunit is used to calculate the gray average value of all targets;

The second comparison subunit is used to compare the grayscale average value of all targets with a preset second grayscale average value threshold;

The to-be-processed target determination subunit is configured to determine all the targets as the above-mentioned to-be-processed targets if the grayscale average value of all the targets is smaller than the above-mentioned second grayscale average value threshold.

Optionally, the above-mentioned dividing unit 502 includes:

a target detection frame obtaining subunit, used to obtain the target detection frame of the above-mentioned target to be processed;

The image frame setting subunit to be processed is used to set the image frame to be processed in the preview image based on the target detection frame, wherein the image frame to be processed has the same shape as the target detection frame, and the image frame to be processed has the same shape as the target detection frame. Each boundary of the above-mentioned target detection frame is respectively parallel to the corresponding boundary of the above-mentioned target detection frame, and each boundary of the above-mentioned to-be-processed image frame is separated from the corresponding boundary of the above-mentioned target detection frame by a preset distance;

a to-be-processed image determination subunit, configured to determine the image within the above-mentioned to-be-processed image frame as the to-be-processed image;

The scene image determination subunit is used to determine the image outside the target detection frame as the scene image.

Optionally, the above-mentioned image super-resolution reconstruction device further includes:

a coordinate acquiring unit, used for acquiring the coordinates of each vertex of the above-mentioned to-be-processed image frame in the above-mentioned preview image;

Correspondingly, the above-mentioned fusion unit 504 includes:

an overlapping area acquisition subunit, configured to overlap the above-mentioned processed image with the above-mentioned scene image based on the coordinates of each vertex of the above-mentioned to-be-processed image frame in the above-mentioned preview image to obtain an overlapping area;

The overlapping area fusion subunit is configured to fuse the edges of the above-mentioned scene image and the above-mentioned processed image based on the above-mentioned overlapping area to obtain a new preview image.

Optionally, the above-mentioned overlapping region fusion subunit includes:

The grayscale acquisition subunit is used for obtaining the grayscale value of the pixel point in the scene image for any pixel point in the overlapping area, which is recorded as the first grayscale value, and obtains the processed image of the pixel point. The gray value in , denoted as the second gray value;

a grayscale calculation subunit, configured to calculate the grayscale average value of the first grayscale value and the second grayscale value;

The grayscale determination subunit is configured to determine the grayscale average value of the first grayscale value and the second grayscale value as the grayscale value of the pixel point after fusion.

As can be seen from the above, through the embodiment of the present application, after obtaining the preview image, the image super-segmentation reconstruction device will segment the preview image if there is a target to be processed in the above-mentioned preview image, so as to obtain a to-be-processed image including the above-mentioned target to be processed. The image and the scene image that does not contain the above-mentioned target to be processed, and only the to-be-processed image is subjected to super-division reconstruction, which reduces the amount of processing data during super-division reconstruction. Finally, the above-mentioned processed image and the above-mentioned scene image are fused to obtain a new image. The preview image can achieve targeted improvement of the clarity of the shooting target.

The embodiment of the present application also provides an electronic device, please refer to FIG. 6 , the electronic device 6 in the embodiment of the present application includes: a memory 601, one or more processors 602 (only one is shown in FIG. A computer program on memory 601 and executable on a processor. The memory 601 is used to store software programs and modules, and the processor 602 executes various functional applications and data processing by running the software programs and units stored in the memory 601 to obtain resources corresponding to the above preset events. Specifically, the above-mentioned processor 602 implements the following steps by running the above-mentioned computer program stored in the memory 601:

get a preview image;

If there is an object to be processed in the preview image, the preview image is segmented to obtain an image to be processed that includes the object to be processed and a scene image that does not include the object to be processed, wherein the object to be processed meets a preset condition The goal;

Perform super-resolution reconstruction on the above-mentioned to-be-processed image to obtain a processed image;

The above-mentioned processed image and the above-mentioned scene image are fused to obtain a new preview image.

Assuming that the above is the first possible implementation manner, in the second possible implementation manner provided on the basis of the first possible implementation manner, after the above-mentioned acquisition of the preview image, the above-mentioned processor 602 stores in the memory by running The above computer program of 601 also implements the following steps:

Detecting whether the shooting scene of the above-mentioned preview image is a night scene;

If the shooting scene of the preview image is a night scene, it is detected whether there is an object to be processed in the preview image.

In the third possible implementation manner provided on the basis of the above-mentioned second possible implementation manner, the above-mentioned detecting whether the shooting scene of the above-mentioned preview image is a night scene includes:

Calculate the grayscale average value of the above preview image;

Compare the grayscale average value of the above-mentioned preview image with the preset first grayscale average value threshold;

If the grayscale average value of the preview image is less than the first grayscale average value threshold, it is determined that the shooting scene of the preview image is a night scene;

If the grayscale average value of the preview image is not less than the first grayscale average value threshold, it is determined that the shooting scene of the preview image is not a night scene.

In the fourth possible implementation manner provided on the basis of the above-mentioned second possible implementation manner, the above-mentioned detecting whether there is an object to be processed in the above-mentioned preview image includes:

Perform target detection on the above-mentioned preview image to obtain more than one target contained in the above-mentioned preview image;

Calculate the grayscale average of all targets;

Compare the grayscale average value of all targets with the preset second grayscale average value threshold;

If the average gray level of all the targets is less than the above-mentioned second threshold value of the gray average value, then all the targets are determined as the above-mentioned objects to be processed.

Based on the above first possible implementation manner, or the above second possible implementation manner as a basis, or the above third possible implementation manner as a basis, or the above fourth possible implementation manner as a basis In a fifth possible implementation manner, the above-mentioned preview image is segmented to obtain a to-be-processed image including the above-mentioned to-be-processed target and a scene image that does not include the above-mentioned to-be-processed target, including:

Obtain the target detection frame of the above-mentioned target to be processed;

Based on the above-mentioned target detection frame, a to-be-processed image frame is set in the above-mentioned preview image, wherein the above-mentioned to-be-processed image frame and the above-mentioned target detection frame have the same shape, and each boundary of the above-mentioned to-be-processed image frame is respectively the same as the above-mentioned target detection frame. The corresponding boundaries are parallel, and each boundary of the above-mentioned image frame to be processed is separated from the corresponding boundary of the above-mentioned target detection frame by a preset distance;

Determining the image within the frame of the image to be processed as the image to be processed;

An image outside the above target detection frame is determined as a scene image.

In the sixth possible implementation manner provided on the basis of the above-mentioned fifth possible implementation manner, the above-mentioned processor 602 further implements the following steps by running the above-mentioned computer program stored in the memory 601:

Obtain the coordinates of each vertex of the above-mentioned to-be-processed image frame in the above-mentioned preview image;

Correspondingly, the above-mentioned processed image and the above-mentioned scene image are fused to obtain a new preview image, including:

Based on the coordinates of each vertex of the above-mentioned to-be-processed image frame in the above-mentioned preview image, overlapping the above-mentioned processed image and the above-mentioned scene image to obtain an overlapping area;

Based on the overlapping area, the edges of the scene image and the processed image are fused to obtain a new preview image.

In the seventh possible implementation manner provided on the basis of the sixth possible implementation manner, based on the overlapping area, the edges of the scene image and the processed image are fused to obtain a new preview image, including :

For any pixel in the overlapping area, obtain the gray value of the pixel in the scene image, denoted as the first gray value, and obtain the gray value of the pixel in the processed image, denoted as the second gray value;

Calculate the grayscale average value of the first grayscale value and the second grayscale value;

The grayscale average value of the first grayscale value and the second grayscale value is determined as the grayscale value of the pixel point after fusion.

Further, the above electronic device may further include: one or more input devices and one or more output devices. The memory 601, the processor 602, the input device and the output device are connected by a bus.

It should be understood that, in the embodiments of the present application, the processor 602 may be a central processing unit (Central Processing Unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), special-purpose processors An integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The input device may include a keyboard, a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and direction information of the fingerprint), a microphone, and the like, and the output device may include a display, a speaker, and the like.

Memory 601 may include read-only memory and random access memory, and provides instructions and data to processor 602 . Part or all of memory 601 may also include non-volatile random access memory. For example, the memory 601 may also store device type information.

As can be seen from the above, through the embodiment of the present application, after the electronic device obtains the preview image, if there is a target to be processed in the above-mentioned preview image, the preview image will be segmented to obtain the above-mentioned target to be processed. The scene image containing the above-mentioned target to be processed, and only the to-be-processed image is subjected to super-resolution reconstruction, which reduces the amount of processing data during super-resolution reconstruction, and finally the above-mentioned processed image and the above-mentioned scene image are fused to obtain a new preview image , to achieve a targeted improvement of the clarity of the shooting target.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the above device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of external device software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the system embodiments described above are only illustrative. For example, the division of the above-mentioned modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined. Either it can be integrated into another system, or some features can be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the methods of the above-mentioned embodiments, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium. The computer program When executed by a processor, the steps of each of the above method embodiments can be implemented. Wherein, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code may be in the form of source code, object code form, executable file or some intermediate form. The above-mentioned computer-readable storage medium may include: any entity or device capable of carrying the above-mentioned computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer-readable memory, a read-only memory (ROM, Read-Only Memory) ), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer-readable storage media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, computer-readable storage Excluded from the medium are electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that the above-mentioned embodiments can still be used for The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in the present application. within the scope of protection of the application.

Claims (10)

1. an image super-resolution reconstruction method, is characterized in that, comprises: get a preview image; If there is an object to be processed in the preview image, segment the preview image to obtain an image to be processed including the object to be processed and a scene image not including the object to be processed, wherein the object to be processed To meet the target of the preset conditions; Perform super-resolution reconstruction on the to-be-processed image to obtain a processed image; The processed image and the scene image are fused to obtain a new preview image. 2. The image super-resolution reconstruction method according to claim 1, characterized in that, after the acquisition of the preview image, the image super-resolution reconstruction method further comprises: Detecting whether the shooting scene of the preview image is a night scene; If the shooting scene of the preview image is a night scene, it is detected whether there is an object to be processed in the preview image. 3. The image super-resolution reconstruction method according to claim 2, wherein the detecting whether the shooting scene of the preview image is a night scene, comprising: calculating the grayscale average value of the preview image; comparing the grayscale average value of the preview image with a preset first grayscale average value threshold; If the grayscale average value of the preview image is less than the first grayscale average value threshold, determining that the shooting scene of the preview image is a night scene; If the grayscale average value of the preview image is not less than the first grayscale average value threshold, it is determined that the shooting scene of the preview image is not a night scene. 4. The image super-resolution reconstruction method according to claim 2, wherein the detecting whether there is an object to be processed in the preview image comprises: performing target detection on the preview image to obtain more than one target included in the preview image; Calculate the grayscale average of all targets; Compare the grayscale average value of all targets with the preset second grayscale average value threshold; If the average gray level of all objects is less than the second gray level average threshold, then all objects are determined as the objects to be processed. 5. The image superdivision reconstruction method according to any one of claims 1 to 4, wherein the preview image is segmented to obtain a to-be-processed image including the to-be-processed target and an image not including the to-be-processed target. Scene image of the target to be processed, including: obtaining the target detection frame of the target to be processed; Based on the target detection frame, a to-be-processed image frame is set in the preview image, wherein the to-be-processed image frame and the target detection frame have the same shape, and each boundary of the to-be-processed image frame is The corresponding boundaries of the target detection frame are parallel, and each boundary of the to-be-processed image frame is separated from the corresponding boundary of the target detection frame by a preset distance; determining the image within the to-be-processed image frame as the to-be-processed image; An image outside the target detection frame is determined as a scene image. 6. The image super-resolution reconstruction method according to claim 5, wherein the image super-resolution reconstruction method further comprises: Obtain the coordinates of each vertex of the to-be-processed image frame in the preview image; The fusion of the processed image and the scene image to obtain a new preview image includes: Based on the coordinates of each vertex of the to-be-processed image frame in the preview image, overlapping the processed image and the scene image to obtain an overlapping area; Based on the overlapping area, the edges of the scene image and the processed image are fused to obtain a new preview image. 7. The image superdivision reconstruction method according to claim 6, wherein, based on the overlapping area, the scene image and the edge of the processed image are fused to obtain a new preview image, comprising: : For any pixel in the overlapping area, obtain the gray value of the pixel in the scene image, record it as the first gray value, and obtain the gray value of the pixel in the processed image degree value, denoted as the second gray value; calculating the grayscale average value of the first grayscale value and the second grayscale value; The grayscale value of the first grayscale value and the grayscale average value of the second grayscale value is determined as the grayscale value of the pixel point after fusion. 8. An image super-division reconstruction device, characterized in that, comprising: The acquisition unit is used to acquire the preview image; a segmentation unit, configured to segment the preview image if there is an object to be processed in the preview image to obtain an image to be processed including the object to be processed and a scene image not including the object to be processed, wherein, The to-be-processed target is a target that satisfies a preset condition; a processing unit, configured to perform super-resolution reconstruction on the to-be-processed image to obtain a processed image; A fusion unit, configured to fuse the processed image with the scene image to obtain a new preview image. 9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the computer program as claimed in the claims The steps of any one of 1 to 7 of the method. 10. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented .

Images