CN107657591A - A kind of image processing method and mobile terminal - Google Patents

Abstract

The embodiments of the invention provide a kind of image processing method and mobile terminal, the image processing method includes：It whether there is Hua Ping regions in detection target two field picture；If so, then obtain the multiple image adjacent with target two field picture；Each two field picture in multiple image is subjected to similarity mode with target two field picture successively；Determine that the image that similarity is more than or equal to predetermined threshold value in multiple image is correction chart picture；Detect the correcting area for corresponding to Hua Ping regions in correction chart picture；Extract the pixel value of all pixels point in correcting area；According to the pixel value extracted, the pixel value of all pixels point in Hua Ping regions is updated.So as to ensure that the fluency of video image and definition, Consumer's Experience is effectively improved.

Description

Image processing method and mobile terminal

technical field

The embodiments of the present invention relate to the communication field, and in particular, to an image processing method and a mobile terminal.

Background technique

With the popularity of smart phones, users have higher and higher requirements for mobile phone detail optimization and experience. During the use of mobile phones, video functions are often used by users, such as video calling, or watching movies online.

However, during the video playback process, due to the influence of factors such as signal quality, blurred screens often appear in the image, so that the user cannot watch the video picture completely, resulting in a poor video experience for the user.

Contents of the invention

An embodiment of the present invention provides an image processing method, so as to solve the problem of poor user experience caused by image blurring phenomenon existing when a user uses a video function.

In a first aspect, an image processing method is provided, which is applied to a mobile terminal, and the method includes:

Detect whether there is a blurry area in the target frame image;

If so, then obtain multiple frame images adjacent to the target frame image;

Carry out similarity matching between each frame image in the multiple frame images and the target frame image in turn;

Determining an image whose similarity is greater than or equal to a preset threshold in the multi-frame images is a corrected image;

Detecting the corrected area corresponding to the blurred screen area in the corrected image;

Extract the pixel values of all pixels in the correction area;

According to the extracted pixel values, the pixel values of all pixels in the blurred screen area are updated.

On the other hand, the embodiment of the present invention also provides a mobile terminal, including:

The first detection module is used to detect whether there is a blurred screen area in the target frame image;

The acquisition module is used to obtain multiple frames of images adjacent to the target frame image if it is;

A similarity matching module is used to perform similarity matching on each frame image in the multi-frame images with the target frame image in turn;

A determining module, configured to determine that an image whose similarity is greater than or equal to a preset threshold in multiple frames of images is a corrected image;

The second detection module is used to detect the corrected area corresponding to the blurred screen area in the corrected image;

An extraction module, configured to extract pixel values of all pixels in the correction area;

The update module is configured to update the pixel values of all pixels in the blurred screen area according to the extracted pixel values.

The embodiment of the present invention also provides a mobile terminal, including: a memory, a processor, and an image processing program stored in the memory and operable on the processor. When the image processing program is executed by the processor, the process shown in the present invention The steps of any image processing method.

The embodiment of the present invention also provides a computer-readable storage medium, on which an image processing program is stored, and when the image processing program is executed by a processor, the steps of any image processing method shown in the present invention are implemented .

In this way, in the embodiment of the present invention, by detecting whether there is a blurred screen area in the target frame image; if so, then obtain multiple frame images adjacent to the target frame image; Similarity matching; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; according to the extracted Pixel value, update the pixel values of all pixels in the blurred screen area. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention , for those skilled in the art, other drawings can also be obtained according to these drawings without paying creative labor.

Fig. 1 is one of the flowcharts of an image processing method in an embodiment of the present invention;

FIG. 2 is the second flowchart of an image processing method in an embodiment of the present invention;

FIG. 3 is the third flowchart of an image processing method in an embodiment of the present invention;

Fig. 4 is one of block diagrams of the mobile terminal in the embodiment of the present invention;

Fig. 5 is the second block diagram of the mobile terminal in the embodiment of the present invention;

Fig. 6 is the third block diagram of the mobile terminal in the embodiment of the present invention;

Fig. 7 is the fourth block diagram of the mobile terminal in the embodiment of the present invention;

Fig. 8 is the fifth block diagram of the mobile terminal in the embodiment of the present invention;

Fig. 9 is the sixth block diagram of the mobile terminal in the embodiment of the present invention;

Fig. 10 is a schematic structural diagram of a mobile terminal in an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Embodiment one

Referring to FIG. 1, it shows a flowchart of an image processing method in an embodiment of the present invention, specifically including:

Step 101, detecting whether there is a blurred screen area in the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal detects whether there is a blurred screen area in the target frame image during video playback.

In an embodiment of the present invention, the target frame may be a current frame image during video playback, that is, the mobile terminal monitors each frame in real time. In another embodiment, the target frame may be any frame in the video playing process, that is, the image processing method in the embodiment of the present invention may be applied to an image with blurred screen at any moment in the video playing process.

Step 102, if yes, acquire multiple frame images adjacent to the target frame image.

Specifically, in the embodiment of the present invention, if the mobile terminal detects that there is a blurry area in the target frame image, it acquires multiple frames of images adjacent to the target frame image. In one embodiment, if the current video image is an image in a video call, N frames of images before the target frame image are acquired. In another embodiment, if the current video image is a bufferable video resource such as a movie, N frames of images before and after the target frame image are acquired. In an embodiment of the present invention, the number of adjacent multi-frame images, that is, N may be five. In other embodiments, the number of acquired adjacent images may be set according to the computing capability and actual needs of the mobile terminal, which is not limited in the present invention.

Step 103, performing similarity matching on each frame image in the multiple frame images sequentially with the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal performs similarity matching on each of the acquired multiple frame images with the target frame image one by one. The specific matching process will be described in detail in the following embodiments.

Step 104, determining an image whose similarity degree is greater than or equal to a preset threshold among the multiple frames of images as the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal determines an image whose similarity is successfully matched according to the similarity of each frame of images among the multiple frames of images corresponding to the target frame image. In the embodiment of the present invention, an image whose similarity is greater than or equal to a preset threshold can be determined as a corrected image whose similarity is successfully matched. Wherein, the preset threshold may be 90%. In other embodiments, the preset threshold can also be set according to actual needs, which is not limited in the present invention.

Step 105, detecting the corrected area corresponding to the blurred screen area in the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal locates the corrected image according to information such as the location and contour of the blurred area in the target frame image, that is, locates the corrected area corresponding to the blurred area in the corrected image.

Step 106, extracting pixel values of all pixel points in the correction area.

Specifically, in the embodiment of the present invention, the mobile terminal detects and extracts the pixel values corresponding to all the pixel points in the correction area in the correction image.

Step 107, update the pixel values of all pixels in the blurred screen area according to the extracted pixel values.

Specifically, in the embodiment of the present invention, the mobile terminal updates the pixel values of all pixels in the blurred screen area in the target frame image according to the extracted pixel values. That is, the extracted pixel values are replaced with pixel values of corresponding pixel points in the blurred screen area.

To sum up, in the embodiment of the present invention, by detecting whether there is a blurred screen area in the target frame image; if so, then obtain multiple frame images adjacent to the target frame image; Carry out similarity matching; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; according to the extracted The pixel values of all pixel points in the blurred screen area are updated. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

Embodiment two

Referring to FIG. 2, it shows a flow chart of an image processing method in an embodiment of the present invention, specifically including:

Step 201, detecting whether there is a blurry area in the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal detects whether there is a blurred screen area in the target frame image during video playback.

In an embodiment of the present invention, the target frame may be a current frame image during video playback, that is, the mobile terminal monitors each frame in real time. In another embodiment, the target frame may be any frame in the video playing process, that is, the image processing method in the embodiment of the present invention may be applied to an image with blurred screen at any moment in the video playing process.

Wherein, if there is a blurry area in the target frame image, go to step 202; if not, skip the target frame image and continue to detect the next frame image.

Specifically, step 201 specifically includes:

In sub-step 2011, edge detection processing is performed on the target frame image to obtain image edge information in the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal can detect image edge information in the target frame image by using an edge detection method. The image edge information is used to describe the graphic outline of the image. It should be noted that if there is a blurry area, there are multiple squares or quasi-squares in the outline of the image.

Sub-step 2012, according to the image edge information, determine whether there is at least one adjacent square-like region in the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal can determine whether there are multiple squares or quasi-squares stacked together in the target frame image according to the image edge information. A specific judging method may be to judge whether there is a square or a quasi-square through the 8-point detection method in the prior art embodiment. For example: According to the edge detected in the edge detection (that is, the contour in the image edge information), the side length starts from 3 pixels, judge whether the side length can form a square or a similar square, and count the formed square and similar square the number of and . Then, increase the side length by one pixel, continue traversing, and find the number of squares and quasi-squares and the maximum side length. A bunch of squares and quasi-squares formed by this side length are the blurred screen area. In other embodiments, other methods may also be used to determine whether there is a square or a quasi-square, which is not limited in the present invention.

Sub-step 2013, if yes, determine that there is a blurry area in the target frame image.

Specifically, in the embodiment of the present invention, when the mobile terminal detects stacked squares or quasi-squares, it can determine that there is a blurred area in the target frame image, and accurately locate the blurred area. In the embodiment of the present invention, by combining the edge detection method and the method of judging a square or a similar square, the blurred screen area in the target frame image can be quickly and accurately located.

Step 202, acquiring multiple frame images adjacent to the target frame image.

Specifically, in the embodiment of the present invention, if the mobile terminal detects that there is a blurry area in the target frame image, it acquires multiple frames of images adjacent to the target frame image. In one embodiment, if the current video image is an image in a video call, N frames of images before the target frame image are acquired. In another embodiment, if the current video image is a bufferable video resource such as a movie, N frames of images before and after the target frame image are acquired. In an embodiment of the present invention, the number of adjacent multi-frame images, that is, N may be five. In other embodiments, the number of acquired adjacent images may be set according to the computing capability and actual needs of the mobile terminal, which is not limited in the present invention.

Step 203, performing similarity matching on each frame image in the multiple frame images sequentially with the target frame image.

Specifically, step 203 specifically includes:

In sub-step 2031, the background area of the target frame image is obtained, and the background area is an area other than the blurred screen area.

Specifically, in the embodiment of the present invention, the mobile terminal acquires the background area of the target frame image, that is, all areas except the blurred screen area. In one embodiment, the method for obtaining the background area may be to directly remove the blurred screen area from the target frame image. In another embodiment, the method for obtaining the region may also be to set all pixel values of the blurred screen region to 0.

Sub-step 2032, acquiring the matching area corresponding to the background area in each frame of images in the multiple frames of images.

Specifically, in the embodiment of the present invention, the mobile terminal respectively obtains the background area of each frame image in the multiple frame images adjacent to the target frame image, that is, for each frame image, removes the location corresponding to the target frame image The blurred screen area is shown, and the remaining area is the background area of each frame of image, that is, the matching area in the embodiment of the present invention.

Sub-step 2033, calculating the similarity between the matching area and the background area in each frame of image.

Specifically, in the embodiment of the present invention, the mobile terminal respectively calculates the similarity between the matching area of each frame image and the background area in the target frame image. The specific calculation method can be realized by the similarity calculation method in the existing embodiments, which will not be repeated in the present invention.

In step 204, it is determined that an image whose similarity degree is greater than or equal to a preset threshold among the multiple frames of images is the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal obtains the calculated similarity between each frame image and the target frame image, and compares it with a preset threshold. In the embodiment of the present invention, the preset threshold is 90%, that is, images with a similarity greater than or equal to 90% are corrected images with successful similarity matching. Otherwise, the matching is unsuccessful. The preset threshold can also be set according to actual needs, which is not limited in the present invention. In the embodiment of the present invention, by calculating the similarity between multiple frame images and the target frame image, the corrected image with the highest similarity to the target frame image can be quickly found, which can be used for subsequent image correction on the target frame image. Compared with the prior art, the calculation amount in the embodiment of the present invention is significantly reduced, thereby reducing the system burden, and at the same time, improving the processing efficiency of image correction.

In a preferred embodiment of the present invention, if there are at least two frames of images whose similarity is greater than or equal to a preset threshold, step 204 may further include:

Sub-step 2041 , detecting the resolution corresponding to the blurred screen area in each frame of images with successful similarity matching. In the embodiment of the present invention, the mobile terminal may respectively detect the resolution corresponding to the blurred screen area in each frame of image whose similarity is greater than or equal to a preset threshold.

Sub-step 2042, select the image with the highest resolution as the corrected image.

In the embodiment of the present invention, the mobile terminal may select the image with the highest resolution corresponding to the blurred screen area, that is, the clearest image corresponding to the blurred screen area, as the corrected image.

Step 205, detecting the corrected area corresponding to the blurred screen area in the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal locates the corrected image according to information such as the position and outline of the blurred area in the target frame image, that is, locates the corrected area corresponding to the blurred area in the corrected image.

Step 206, extracting pixel values of all pixel points in the correction area.

Specifically, in the embodiment of the present invention, the mobile terminal detects and extracts pixel values corresponding to all pixel points in the correction area.

Step 207, update the pixel values of all pixels in the blurred screen area according to the extracted pixel values.

Specifically, in the embodiment of the present invention, the mobile terminal updates the pixel values of all pixels in the blurred screen area in the target frame image according to the extracted pixel values. That is, the extracted pixel values are replaced with pixel values of corresponding pixel points in the blurred screen area.

To sum up, in the embodiment of the present invention, by detecting whether there is a blurred screen area in the target frame image; if so, then obtain multiple frame images adjacent to the target frame image; Carry out similarity matching; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; according to the extracted The pixel values of all pixel points in the blurred screen area are updated. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

Embodiment three

Referring to FIG. 3 , it shows a flowchart of an image processing method in an embodiment of the present invention, specifically including:

Step 301, detecting whether there is a blurred screen area in the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal detects whether there is a blurred screen area in the target frame image during video playback.

In an embodiment of the present invention, the target frame may be a current frame image during video playback, that is, the mobile terminal monitors each frame in real time. In another embodiment, the target frame may be any frame in the video playing process, that is, the image processing method in the embodiment of the present invention may be applied to an image with blurred screen at any moment in the video playing process.

If there is a blurry area in the target frame image, go to step 302; if not, skip the target frame image and continue to detect the next frame image.

Specifically, step 301 specifically includes:

Sub-step 3011, constructing a convolutional neural network.

In sub-step 3012, parameters of the convolutional neural network are set according to multiple images in the database, wherein the multiple images include at least one image with a blurry area.

Sub-step 3013, according to the parameters of the convolutional neural network, train the convolutional neural network to obtain a blurred screen detection classifier.

Sub-step 3014, through the blurred screen detection classifier, detect whether there is a blurred screen area in the target frame image.

Specifically, in the embodiment of the present invention, the mobile terminal acquires multiple images in the database, where the multiple images in the database may be images acquired from the network side, captured images or locally stored images. In the embodiment of the present invention, the number of images may be 3000. Among the 3000 images, there are 1500 images with blurred screen and 1500 images without blurred screen.

The mobile terminal builds an m-layer convolutional neural network, where m can be 5. Then, the acquired multiple images are used as input data, and the parameters of the convolutional neural network are set. The convolutional neural network can be trained according to the settings, and finally a blurred screen detection classifier that can identify and locate the blurred screen area is obtained. For specific details, reference may be made to the convolutional neural network training method in the existing embodiments, which will not be repeated in the present invention.

Subsequently, the mobile terminal can identify whether there is a blurred area in the target frame image through the blurred screen detection classifier, and accurately locate the blurred screen area. In the embodiment of the present invention, the artifact detector can be used to quickly and accurately locate the artifact area in the target frame image.

Step 302, acquiring multiple frame images adjacent to the target frame image.

Specifically, in the embodiment of the present invention, if the mobile terminal detects that there is a blurry area in the target frame image, it acquires multiple frames of images adjacent to the target frame image. In one embodiment, if the current video image is an image in a video call, N frames of images before the target frame image are acquired. In another embodiment, if the current video image is a bufferable video resource such as a movie, N frames of images before and after the target frame image are acquired. In an embodiment of the present invention, the number of adjacent multi-frame images, that is, N may be five. In other embodiments, the number of acquired adjacent images may be set according to the computing capability and actual needs of the mobile terminal, which is not limited in the present invention.

Step 303, performing similarity matching on each frame image in the multiple frame images sequentially with the target frame image.

Specifically, step 303 specifically includes:

In sub-step 3031, the background area of the target frame image is obtained, and the background area is the area except the blurred screen area.

Specifically, in the embodiment of the present invention, the mobile terminal acquires the background area of the target frame image, that is, all areas except the blurred screen area. In one embodiment, the method for obtaining the background area may be to directly remove the blurred screen area from the target frame image. In another embodiment, the method for obtaining the region may also be to set all pixel values of the blurred screen region to 0.

Sub-step 3032, acquiring the matching area corresponding to the background area in each frame of images in the multiple frames of images.

Specifically, in the embodiment of the present invention, the mobile terminal respectively obtains the background area of each frame image in the multiple frame images adjacent to the target frame image, that is, for each frame image, removes the location corresponding to the target frame image The blurred screen area is shown, and the remaining area is the background area of each frame of image, that is, the matching area in the embodiment of the present invention.

Sub-step 3033, calculating the similarity between the matching area and the background area in each frame of image.

Specifically, in the embodiment of the present invention, the mobile terminal respectively calculates the similarity between the matching area of each frame image and the background area in the target frame image. The specific calculation method can be realized by the similarity calculation method in the existing embodiments, which will not be repeated in the present invention.

In step 304, it is determined that an image whose similarity degree is greater than or equal to a preset threshold among the multiple frames of images is the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal obtains the calculated similarity between each frame image and the target frame image, and compares it with a preset threshold. In the embodiment of the present invention, the preset threshold is 90%, that is, images with a similarity greater than or equal to 90% are corrected images with successful similarity matching. Otherwise, the matching is unsuccessful. The preset threshold can also be set according to actual needs, which is not limited in the present invention. In the embodiment of the present invention, by calculating the similarity between multiple frame images and the target frame image, the corrected image with the highest similarity to the target frame image can be quickly found, which can be used for subsequent image correction on the target frame image. Compared with the prior art, the calculation amount in the embodiment of the present invention is significantly reduced, thereby reducing the system burden, and at the same time, improving the processing efficiency of image correction.

In a preferred embodiment of the present invention, if there are at least two frames of images whose similarity is greater than or equal to a preset threshold, step 304 may further include:

Sub-step 3041 , detecting the resolution corresponding to the blurred screen area in each frame of images with successful similarity matching. In the embodiment of the present invention, the mobile terminal may respectively detect the resolution corresponding to the blurred screen area in each frame of image whose similarity is greater than or equal to a preset threshold.

Sub-step 3042, select the image with the highest resolution as the corrected image.

In the embodiment of the present invention, the mobile terminal may select the image with the highest resolution corresponding to the blurred screen area, that is, the clearest image corresponding to the blurred screen area, as the corrected image.

Step 305, detecting the corrected area corresponding to the blurred screen area in the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal locates the corrected image according to information such as the position and outline of the blurred area in the target frame image, that is, locates the corrected area corresponding to the blurred area in the corrected image.

Step 306, extracting pixel values of all pixel points in the correction area.

Specifically, in the embodiment of the present invention, the mobile terminal detects and extracts pixel values corresponding to all pixel points in the correction area.

Step 307, update the pixel values of all pixels in the blurred screen area according to the extracted pixel values.

Specifically, in the embodiment of the present invention, the mobile terminal updates the pixel values of all pixels in the blurred screen area in the target frame image according to the extracted pixel values. That is, the extracted pixel values are replaced with pixel values of corresponding pixel points in the blurred screen area.

To sum up, in the embodiment of the present invention, by detecting whether there is a blurred screen area in the target frame image; if so, then obtain multiple frame images adjacent to the target frame image; Carry out similarity matching; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; according to the extracted The pixel values of all pixel points in the blurred screen area are updated. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

Embodiment Four

Referring to FIG. 4 , it shows a block diagram of a mobile terminal 400 in an embodiment of the present invention. Specifically include:

The first detection module 401 is used to detect whether there is a blurred screen area in the target frame image.

Specifically, in the embodiment of the present invention, during the video playing process of the mobile terminal, the first acquisition module 401 detects whether there is a blurred screen area in the target frame image.

In an embodiment of the present invention, the target frame may be a current frame image during video playback, that is, the mobile terminal monitors each frame in real time. In another embodiment, the target frame may be any frame in the video playing process, that is, the image processing method in the embodiment of the present invention may be applied to an image with blurred screen at any moment in the video playing process.

The obtaining module 402 is configured to obtain multiple frame images adjacent to the target frame image if yes.

Specifically, in the embodiment of the present invention, if the mobile terminal detects that there is a blurry area in the target frame image, the first acquiring module 402 acquires multiple frame images adjacent to the target frame image. In one embodiment, if the current video image is an image in a video call, N frames of images before the target frame image are acquired. In another embodiment, if the current video image is a bufferable video resource such as a movie, N frames of images before and after the target frame image are acquired. In an embodiment of the present invention, the number of adjacent multi-frame images, that is, N may be five. In other embodiments, the number of acquired adjacent images may be set according to the computing capability and actual needs of the mobile terminal, which is not limited in the present invention.

The similarity matching module 403 is configured to sequentially perform similarity matching on each frame image in the multiple frame images and the target frame image.

Specifically, in the embodiment of the present invention, the similarity matching module 403 performs similarity matching on each frame image among the acquired multi-frame images with the target frame image one by one.

The determination module 404 is configured to determine that an image whose similarity degree is greater than or equal to a preset threshold among the multiple frames of images is a corrected image.

Specifically, in the embodiment of the present invention, the determining module 404 determines the image whose similarity is successfully matched according to the similarity of each frame of images in the multiple frames of images corresponding to the target frame image. In the embodiment of the present invention, an image whose similarity is greater than or equal to a preset threshold can be determined as a corrected image whose similarity is successfully matched. Wherein, the preset threshold may be 90%. In other embodiments, the preset threshold can also be set according to actual needs, which is not limited in the present invention.

The second detection module 405 is configured to detect the corrected area corresponding to the blurred screen area in the corrected image.

Specifically, in the embodiment of the present invention, the mobile terminal locates the corrected image according to information such as the position and outline of the blurred area in the target frame image, that is, locates the corrected area corresponding to the blurred area in the corrected image.

An extraction module 406, configured to extract pixel values of all pixel points in the correction area.

An update module 407, configured to update the pixel values of all pixels in the blurred screen area according to the extracted pixel values.

Specifically, in the embodiment of the present invention, the update module 406 updates the pixel values of all pixels in the blurred screen area in the target frame image according to the extracted pixel values. That is, the extracted pixel values are replaced with pixel values of corresponding pixel points in the blurred screen area.

To sum up, the mobile terminal in the embodiment of the present invention detects whether there is a blurred screen area in the target frame image; if so, obtains multiple frames of images adjacent to the target frame image; Perform similarity matching on the target frame image; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; According to the extracted pixel values, the pixel values of all pixels in the blurred screen area are updated. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

Referring to FIG. 5, in a preferred embodiment of the present invention, on the basis of FIG. 4, the first detection module 401 further includes:

The edge detection sub-module 4011 is used to perform edge detection processing on the target frame image to obtain image edge information in the target frame image;

The judging sub-module 4012 is used to judge whether there is at least one adjacent square-like region in the target frame image according to the image edge information;

The determination sub-module 4013 is configured to determine that if yes, there is a blurry area in the target frame image.

Referring to FIG. 6, in a preferred embodiment of the present invention, on the basis of FIG. 4, the detection module 401 further includes:

Build a submodule 4014 for building a convolutional neural network;

The setting sub-module 4015 is used to set the parameters of the convolutional neural network according to the acquired multiple images, wherein the multiple images include at least one image with a blurry area;

The training sub-module 4016 is used to train the convolutional neural network according to the parameters to obtain a blurred screen detection classifier;

The first detection sub-module 4017 is used to detect whether there is a blurred area in the target frame image through the blurred screen detection classifier.

Referring to FIG. 7, in a preferred embodiment of the present invention, on the basis of FIG. 4, the similarity matching module 403 further includes:

The first acquisition sub-module 4031 is used to acquire the background area of the target frame image, where the background area is an area other than the blurred screen area;

The second acquisition sub-module 4032 is used to acquire the matching area corresponding to the background area in each frame of images in the multi-frame images;

The calculation sub-module 4033 is used to calculate the similarity between the matching area and the background area in each frame of image.

Referring to FIG. 8, in a preferred embodiment of the present invention, on the basis of FIG. 4, the determination module 404 further includes:

The second detection sub-module 4041 is configured to detect the resolution corresponding to the blurred screen area in each frame of the at least two frames of images if there are at least two frames of images whose similarity is greater than or equal to the preset threshold;

The selection sub-module 4042 is used to select the image with the highest resolution as the corrected image.

Embodiment five

FIG. 9 is a block diagram of a mobile terminal according to another embodiment of the present invention. The mobile terminal 500 shown in FIG. 9 includes: at least one processor 501 , a memory 502 , at least one network interface 504 and other user interfaces 503 . Various components in the mobile terminal 500 are coupled together through the bus system 505 . It can be understood that the bus system 505 is used to realize connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for clarity of illustration, the various buses are labeled as bus system 505 in FIG. 5 .

Wherein, the user interface 503 may include a display, a keyboard, or a pointing device (for example, a mouse, a trackball (trackball), a touch panel, or a touch screen, and the like.

It can be understood that the memory 502 in the embodiment of the present invention may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the non-volatile memory can be read-only memory (Read-OnlyMemory, ROM), programmable read-only memory (ProgrammableROM, PROM), erasable programmable read-only memory (ErasablePROM, EPROM), electrically erasable Programming read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be random access memory (Random Access Memory, RAM), which acts as an external cache. By way of illustration and not limitation, many forms of RAM are available such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (Synchronous DRAM, SDRAM), Double data rate synchronous dynamic random access memory (DoubleDataRateSDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (SynchlinkDRAM, SLDRAM) and direct memory bus random access Memory (Direct Rambus RAM, DRRAM). The memory 502 of the systems and methods described in embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

In some implementations, the memory 502 stores the following elements, executable modules or data structures, or their subsets, or their extended sets: an operating system 5021 and an application program 5022 .

Among them, the operating system 5021 includes various system programs, such as framework layer, core library layer, driver layer, etc., for realizing various basic services and processing tasks based on hardware. The application program 5022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., and is used to implement various application services. The program for realizing the method of the embodiment of the present invention may be included in the application program 5022 .

In the embodiment of the present invention, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application program 5022, the processor 501 is used to detect whether there is a blurred screen area in the target frame image; if so, then Acquiring multiple frames of images adjacent to the target frame image; sequentially performing similarity matching on each frame of the multiple frames of images with the target frame image; determining an image with a similarity greater than or equal to a preset threshold among the multiple frames of images as a corrected image; Detecting the corrected area corresponding to the blurred screen area in the corrected image; extracting the pixel values of all pixels in the corrected area; updating the pixel values of all the pixel points in the blurred screen area according to the extracted pixel values.

The methods disclosed in the foregoing embodiments of the present invention may be applied to the processor 501 or implemented by the processor 501 . The processor 501 may be an integrated circuit chip and has signal processing capability. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 501 or instructions in the form of software. The above-mentioned processor 501 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the methods disclosed in the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and completes the steps of the above method in combination with its hardware.

It can be understood that the embodiments described in the embodiments of the present invention may be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing device (DSPDevice, DSPD), programmable logic device (ProgrammableLogicDevice, PLD ), Field-Programmable Gate Array (Field-Programmable GateArray, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing the functions described in this application, or a combination thereof.

For software implementation, the techniques described in the embodiments of the present invention may be implemented through modules (such as procedures, functions, etc.) that execute the functions described in the embodiments of the present invention. Software codes can be stored in memory and executed by a processor. Memory can be implemented within the processor or external to the processor.

Optionally, the processor 501 is further configured to: perform edge detection processing on the target frame image to obtain image edge information in the target frame image; judge whether there is at least one adjacent square in the target frame image according to the image edge information area; if yes, determine that there is a blurred screen area in the target frame image.

Optionally, the processor 501 is further configured to: construct a convolutional neural network; set parameters of the convolutional neural network according to multiple images in the database, wherein the multiple images include at least one image with a blurry area; The parameters of the convolutional neural network are trained to obtain a blurred screen detection classifier; through the blurred screen detection classifier, it is detected whether there is a blurred screen area in the target frame image.

Optionally, as another embodiment, the processor 501 is further configured to: acquire the background area of the target frame image, where the background area is an area other than the blurred screen area; Matching area; Calculate the similarity between the matching area and the background area in each frame of the image.

Optionally, the processor 501 is further configured to: if there are at least two frames of images whose similarity is greater than or equal to a preset threshold in the multiple frames of images, detect the resolution corresponding to the blurred screen area in each of the at least two frames of images; The image with the highest resolution was selected as the corrected image.

The mobile terminal 500 can implement various processes implemented by the mobile terminal in the foregoing embodiments, and to avoid repetition, details are not repeated here.

To sum up, the mobile terminal in the embodiment of the present invention detects whether there is a blurred screen area in the target frame image; if so, obtains multiple frames of images adjacent to the target frame image; Perform similarity matching on the target frame image; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; According to the extracted pixel values, the pixel values of all pixels in the blurred screen area are updated. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

Embodiment six

Fig. 10 is a schematic structural diagram of a mobile terminal according to another embodiment of the present invention. Specifically, the mobile terminal 600 in FIG. 10 may be a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), or a vehicle-mounted computer.

Mobile terminal 600 in FIG.

Wherein, the input unit 630 can be used to receive number or character information input by the user, and generate signal input related to the user setting and function control of the mobile terminal 600 . Specifically, in the embodiment of the present invention, the input unit 630 may include a touch panel 631 . The touch panel 631, also referred to as a touch screen, can collect user's touch operations on or near it (such as the user's operation on the touch panel 631 using any suitable object or accessory such as a finger, a stylus), and The specified program drives the corresponding connected device. Optionally, the touch panel 631 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and sends it to the to the processor 660, and can receive and execute commands sent by the processor 660. In addition, the touch panel 631 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 631, the input unit 630 may also include other input devices 632, which may include but not limited to physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, etc. one or more of.

Wherein, the display unit 640 can be used to display information input by the user or information provided to the user and various menu interfaces of the mobile terminal 600 . The display unit 640 may include a display panel 641. Optionally, the display panel 641 may be configured in the form of an LCD or an organic light-emitting diode (Organic Light-Emitting Diode, OLED).

It should be noted that the touch panel 631 can cover the display panel 641 to form a touch display screen. When the touch display screen detects a touch operation on or near it, it is sent to the processor 660 to determine the type of the touch event, and then the processor 660 provides corresponding visual output on the touch display screen according to the type of the touch event.

The touch display screen includes an application program interface display area and a common control display area. The arrangement of the display area of the application program interface and the display area of the commonly used controls is not limited, and may be an arrangement in which the two display areas can be distinguished, such as vertical arrangement, left-right arrangement, and the like. The application program interface display area can be used to display the interface of the application program. Each interface may include at least one interface element such as an icon of an application program and/or a widget desktop control. The application program interface display area can also be an empty interface without any content. The commonly used control display area is used to display controls with a high usage rate, for example, application icons such as setting buttons, interface numbers, scroll bars, and phonebook icons.

Wherein the processor 660 is the control center of the mobile terminal 600, utilizes various interfaces and lines to connect the various parts of the whole mobile phone, by running or executing the software programs and/or modules stored in the first memory 621, and calling the software programs and/or modules stored in the second memory 621. The data in the memory 622 executes various functions of the mobile terminal 600 and processes data, so as to monitor the mobile terminal 600 as a whole. Optionally, the processor 660 may include one or more processing units.

In the embodiment of the present invention, the processor 660 is used to detect whether there is a blurry area in the target frame image by calling the software program and/or module stored in the first memory 621 and/or the data in the second memory 622; If so, then obtain multiple frames of images adjacent to the target frame image; each frame image in the multiple frame images is sequentially matched with the target frame image for similarity; determine that the image in the multiple frame images whose similarity is greater than or equal to the preset threshold is Correcting the image; detecting the corrected area corresponding to the blurred screen area in the corrected image; extracting the pixel values of all pixels in the corrected area; updating the pixel values of all the pixel points in the blurred screen area according to the extracted pixel values.

Optionally, the processor 660 is also configured to: perform edge detection processing on the target frame image to obtain image edge information in the target frame image; judge whether there is at least one adjacent square in the target frame image according to the image edge information area; if yes, determine that there is a blurred screen area in the target frame image.

Optionally, the processor 660 is further configured to: construct a convolutional neural network; set parameters of the convolutional neural network according to multiple images in the database, wherein the multiple images include at least one image with a blurry area; The parameters of the convolutional neural network are trained to obtain a blurred screen detection classifier; through the blurred screen detection classifier, it is detected whether there is a blurred screen area in the target frame image.

Optionally, as another embodiment, the processor 660 is further configured to: acquire the background area of the target frame image, where the background area is an area other than the blurred screen area; Matching the region; calculating the similarity between the matching region and the background region in each frame of image; determining that the image whose similarity is greater than or equal to a predetermined threshold is an image whose similarity is successfully matched.

Optionally, the processor 660 is further configured to: if there are at least two frames of images whose similarity is greater than or equal to a preset threshold in the multiple frames of images, then detect the resolution corresponding to the blurred screen area in each frame of the at least two images; select The highest resolution image is the rectified image.

It can be seen that the mobile terminal in the embodiment of the present invention detects whether there is a blurred screen area in the target frame image; if so, obtains multiple frames of images adjacent to the target frame image; Perform similarity matching on the frame images; determine the image whose similarity is greater than or equal to the preset threshold in the multi-frame image is the corrected image; detect the corrected area corresponding to the blurred screen area in the corrected image; extract the pixel values of all pixels in the corrected area; The extracted pixel values update the pixel values of all pixels in the blurred screen area. In this way, the fluency and clarity of the video image are ensured, and the user experience is effectively improved.

The embodiment of the present invention also provides a mobile terminal, including: a memory, a processor, and an image processing program stored in the memory and operable on the processor. When the image processing program is executed by the processor, the process shown in the present invention The steps of any image processing method.

The embodiment of the present invention also provides a computer-readable storage medium, on which an image processing program is stored, and when the image processing program is executed by a processor, the steps of any image processing method shown in the present invention are implemented .

As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.

The image processing solutions presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct a system having the solution of the present invention is apparent from the above description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the image processing solution according to the embodiments of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

Claims (12)

1. A kind of image processing method, is applied to mobile terminal, is characterized in that, described method comprises: Detect whether there is a blurry area in the target frame image; If so, then acquire multiple frame images adjacent to the target frame image; performing similarity matching on each frame image in the multiple frame images sequentially with the target frame image; Determining that an image whose similarity is greater than or equal to a preset threshold among the multiple frames of images is a corrected image; Detecting a corrected area corresponding to the blurred screen area in the corrected image; Extracting pixel values of all pixels in the correction area; According to the extracted pixel values, the pixel values of all pixels in the blurred screen area are updated. 2. The method according to claim 1, wherein the step of detecting whether there is a blurry area in the target frame image specifically comprises: performing edge detection processing on the target frame image to obtain image edge information in the target frame image; According to the image edge information, determine whether there is at least one adjacent square-like region in the target frame image; If yes, it is determined that there is a blurry area in the target frame image. 3. The method according to claim 1, wherein the step of detecting whether there is a blurry area in the target frame image specifically comprises: Build a convolutional neural network; Setting the parameters of the convolutional neural network according to multiple images in the database, wherein the multiple images include at least one image with a blurry area; According to the parameters of the convolutional neural network, the convolutional neural network is trained to obtain a blurred screen detection classifier; Whether there is a blurred area in the target frame image is detected by the blurred screen detection classifier. 4. The method according to claim 1, wherein the step of performing similarity matching between each frame image in the multi-frame images and the target frame image in turn, specifically comprises: Acquire the background area of the target frame image, where the background area is an area other than the blurred screen area; Acquiring a matching area corresponding to the background area in each frame of images in the multiple frames of images; Calculate the similarity between the matching area in each frame of image and the background area. 5. The method according to claim 1, wherein the step of determining that the images whose similarity is greater than or equal to a preset threshold in the multi-frame images is a corrected image includes: If there are at least two frames of images whose similarity is greater than or equal to the preset threshold in the multiple frames of images, detecting the resolution corresponding to the blurred screen area in each frame of the at least two frames of images; The image with the highest resolution is selected as the corrected image. 6. A mobile terminal, characterized in that, comprising: The first detection module is used to detect whether there is a blurred screen area in the target frame image; An acquisition module, configured to acquire multiple frames of images adjacent to the target frame image if so; A similarity matching module, configured to sequentially perform similarity matching on each frame of the multi-frame images with the target frame image; A determining module, configured to determine that an image whose similarity is greater than or equal to a preset threshold among the multiple frames of images is a corrected image; A second detection module, configured to detect a corrected area corresponding to the blurred screen area in the corrected image; An extraction module, configured to extract pixel values of all pixels in the correction area; An update module, configured to update pixel values of all pixels in the blurred screen area according to the extracted pixel values. 7. The mobile terminal according to claim 6, wherein the first detection module further comprises: An edge detection submodule, configured to perform edge detection processing on the target frame image to obtain image edge information in the target frame image; A judging submodule, configured to judge whether there is at least one adjacent square-like region in the target frame image according to the image edge information; The determination sub-module is used for determining that if yes, there is a blurred screen area in the target frame image. 8. The mobile terminal according to claim 6, wherein the first detection module further comprises: Build submodules for building convolutional neural networks; The setting submodule is used to set the parameters of the convolutional neural network according to multiple images in the database, wherein the multiple images include at least one image with a blurry area; The training submodule is used to train the convolutional neural network according to the parameters of the convolutional neural network to obtain a blurred screen detection classifier; The first detection submodule is configured to use the blurred screen detection classifier to detect whether there is a blurred screen area in the target frame image. 9. The mobile terminal according to claim 6, wherein the similarity matching module further comprises: The first acquisition submodule is used to acquire the background area of the target frame image, where the background area is an area other than the blurred screen area; The second acquisition sub-module is used to acquire the matching area corresponding to the background area in each frame of the multi-frame images; The calculation submodule is used to calculate the similarity between the matching area in each frame of image and the background area. 10. The mobile terminal according to claim 6, wherein the determining module further comprises: The second detection submodule is used to detect the image corresponding to the blurred screen area in each frame of the at least two frames of images if there are at least two frames of images whose similarity is greater than or equal to the preset threshold in the multiple frames of images resolution; The selection submodule is used to select the image with the highest resolution as the corrected image. 11. A mobile terminal, characterized in that it comprises: a memory, a processor, and an image processing program stored on the memory and operable on the processor, when the image processing program is executed by the processor The steps for realizing the image processing method as described in any one of claims 1 to 5. 12. A computer-readable storage medium, characterized in that, an image processing program is stored on the computer-readable storage medium, and when the image processing program is executed by a processor, it realizes the process described in any one of claims 1 to 5. The steps of the image processing method described above.