CN117649336A - Image processing method, device and computer-readable storage medium - Google Patents
Image processing method, device and computer-readable storage medium Download PDFInfo
- Publication number
- CN117649336A CN117649336A CN202311685631.7A CN202311685631A CN117649336A CN 117649336 A CN117649336 A CN 117649336A CN 202311685631 A CN202311685631 A CN 202311685631A CN 117649336 A CN117649336 A CN 117649336A
- Authority
- CN
- China
- Prior art keywords
- image
- interference
- target
- target object
- weakening
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
本申请实施例公开了一种图像处理方法、装置和计算机可读存储介质；通过对待处理图像进行对象识别，确定目标对象和干扰对象；基于目标对象获取目标对象遮罩图；基于干扰对象对待处理图像进行干扰弱化处理，得到干扰弱化图像；对干扰弱化图像进行模糊处理，得到背景弱化图像；基于目标对象遮罩图将待处理图像中的目标对象和背景弱化图像中除目标对象之外的图像内容进行融合，得到目标对象图像；将预设的雨天特效图与目标对象图像进行混合，得到目标特效图像。以此，可以基于待处理图像得到雨天场景中具有突出目标对象主体、弱化了干扰对象以及图像背景模糊效果的目标特效图像，丰富了雨天特效效果，进而提升了图像处理效果。
Embodiments of the present application disclose an image processing method, device and computer-readable storage medium; through object recognition of the image to be processed, the target object and the interference object are determined; the target object mask map is obtained based on the target object; and the target object is to be processed based on the interference object. The image is subjected to interference weakening processing to obtain an interference weakened image; the interference weakened image is blurred to obtain a background weakened image; based on the target object mask map, the target object in the image to be processed and the image other than the target object in the background weakened image are The content is fused to obtain the target object image; the preset rainy weather special effects image is mixed with the target object image to obtain the target special effects image. In this way, a target special effects image with prominent target objects in the rainy scene, weakened interference objects and image background blur effects can be obtained based on the image to be processed, enriching the rainy day special effects and thereby improving the image processing effect.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to an image processing method, an image processing device, and a computer readable storage medium.
Background
With the rapid development of internet technology, there is an increasing demand for image processing, for example, by performing image processing on an image input by a user, so as to obtain an image with a specific effect, for example, since a photograph in rain has a better atmosphere, many content creators want to take out an image with a specific effect in the rainy day atmosphere. In the existing image processing method, an image with a specific rainy atmosphere effect is often generated by overlapping an image input by a user with the specific rainy effect image.
In the research and practice process of the prior art, the conventional image processing method for generating the specific image with the rainy atmosphere by superposing the user image and the specific image with the rainy effect has the advantages that the generated rainy effect is monotonous, so that the image processing effect is poor.
Disclosure of Invention
The embodiment of the application provides an image processing method, an image processing device and a computer readable storage medium, which can obtain a target special effect image with a prominent target object body, an interference object weakened and an image background blurring effect in a rainy scene based on an image to be processed, enrich the rainy special effect and further improve the image processing effect.
The embodiment of the application provides an image processing method, which comprises the following steps:
object identification is carried out on the image to be processed, and a target object and an interference object are determined;
acquiring a target object mask map based on the target object;
performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image;
performing fuzzy processing on the interference weakening image to obtain a background weakening image;
fusing a target object in the image to be processed and image contents except the target object in the background weakening image based on the target object mask image to obtain a target object image;
and mixing a preset rainy day special effect image with the target object image to obtain a target special effect image.
Accordingly, an embodiment of the present application provides an image processing apparatus, including:
the object recognition unit is used for recognizing the object of the image to be processed and determining a target object and an interference object;
a mask acquisition unit configured to acquire a target object mask map based on the target object;
the interference weakening unit is used for carrying out interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image;
The background blurring unit is used for blurring the interference weakening image to obtain a background weakening image;
the image fusion unit is used for fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image;
and the image mixing unit is used for mixing the preset rainy day special effect image with the target object image to obtain a target special effect image.
In an embodiment, the interference weakening unit comprises:
an interference mask acquisition subunit, configured to acquire an interference object mask map based on the interference object;
the interference object removing subunit is used for removing the content in the object image area where the interference object is located from the image to be processed according to the interference object mask diagram to obtain an initial object image which does not contain the interference object;
and the image inpainting subunit is used for carrying out image inpainting processing on the object image area in the initial object image based on the neighborhood information of the object image area in the initial object image to obtain an interference weakening image.
In an embodiment, the image inpainting subunit is configured to:
acquiring the region size information of the object image region where the interference object is located;
determining a target region in a neighboring region of the object image region based on the region size information;
and filling the image content in the target area into the object image area in the initial object image to obtain an interference weakening image.
In an embodiment, the object recognition unit is configured to:
identifying at least one candidate object and a candidate object area where the candidate object is located in the image to be processed;
determining region attribute information of the candidate object region;
and determining a target object and an interference object from the candidate objects based on the region attribute information.
In an embodiment, the background blurring unit includes:
a depth value obtaining subunit, configured to obtain a depth value of each pixel in the interference weakened image;
a blur radius determining subunit, configured to determine a blur radius corresponding to the pixel based on the depth value;
and the background blurring subunit is used for blurring the interference weakening image according to the blurring radius to obtain a background weakening image.
In an embodiment, the blur radius determination subunit is configured to:
acquiring a focal depth range;
determining a focus pixel of the interference attenuation image according to the depth value and the focus depth range;
calculating depth differences between pixels in the interference attenuation image and the focus pixels;
and determining the blur radius corresponding to the pixel based on the depth difference.
In an embodiment, the background blurring subunit is configured to:
determining a pixel to be blurred of the pixel in the interference-weakened image based on the blur radius;
sampling the color value of the pixel to be blurred to obtain a first color value of the pixel to be blurred;
performing weighted average processing on the first color value to obtain a second color value;
performing fusion processing on the first color value and the second color value to obtain a target color value corresponding to the pixel to be blurred;
a background-attenuated image is generated based on the target color value.
In addition, the embodiment of the application further provides a computer readable storage medium, where a plurality of instructions are stored, where the instructions are adapted to be loaded by a processor to perform the steps in any of the image processing methods provided in the embodiments of the application.
In addition, the embodiment of the application also provides a computer device, which comprises a processor and a memory, wherein the memory stores an application program, and the processor is used for running the application program in the memory to realize the image processing method provided by the embodiment of the application.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the image processing method provided in the embodiment of the present application.
According to the embodiment of the application, the target object and the interference object are determined by carrying out object identification on the image to be processed; acquiring a target object mask map based on the target object; performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image; performing fuzzy processing on the interference weakening image to obtain a background weakening image; fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image; and mixing the preset rainy day special effect image with the target object image to obtain the target special effect image. In this way, the target object and the interference object in the image to be processed are identified, so that interference attenuation processing is performed on the image to be processed according to the interference object, blurring processing is performed on the interference attenuation image to obtain a background attenuation image, image contents except the target object in the target object and the background attenuation image are fused to obtain a target object image which highlights the target object, and then the target object image is mixed with a preset rainy day special effect image, so that the target special effect image which highlights the target object main body, weakens the interference object and the image background blurring effect in a rainy day scene can be obtained, the rainy day special effect is enriched, and the image processing effect is further improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of an implementation scenario of an image processing method according to an embodiment of the present application;
fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;
fig. 3 is an image schematic diagram of an image processing method according to an embodiment of the present application;
fig. 4 is a schematic diagram of a rainy day special effect image of an image processing method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an image processing apparatus provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
Embodiments of the present application provide an image processing method, apparatus, and computer readable storage medium. The image processing apparatus may be integrated into a computer device, which may be a server or a terminal.
The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), basic cloud computing services such as big data and an artificial intelligent platform. Terminals may include, but are not limited to, cell phones, computers, intelligent voice interaction devices, intelligent appliances, vehicle terminals, aircraft, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.
Referring to fig. 1, taking an example that an image processing apparatus is integrated in a computer device, fig. 1 is a schematic view of an implementation scenario of an image processing method provided in an embodiment of the present application, where the computer device may be a server or a terminal, and the computer device may perform object recognition on an image to be processed to determine a target object and an interference object; acquiring a target object mask map based on the target object; performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image; performing fuzzy processing on the interference weakening image to obtain a background weakening image; fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image; and mixing the preset rainy day special effect image with the target object image to obtain the target special effect image.
It should be noted that, the schematic view of the implementation environment of the image processing method shown in fig. 1 is merely an example, and the implementation environment of the image processing method described in the embodiment of the present application is for more clearly describing the technical solution of the embodiment of the present application, and does not constitute a limitation to the technical solution provided in the embodiment of the present application. As one of ordinary skill in the art can know, with the evolution of data processing and the appearance of new business scenarios, the technical scheme provided in the application is equally applicable to similar technical problems.
The scheme provided by the embodiment of the application is specifically illustrated by the following embodiment. The following description of the embodiments is not intended to limit the preferred embodiments.
The present embodiment will be described from the viewpoint of an image processing apparatus, which may be integrated in a computer device, which may be a server or a terminal, to which the present application is not limited.
Referring to fig. 2, fig. 2 is a flowchart of an image processing method according to an embodiment of the present application. The image processing method comprises the following steps:
in step 101, object recognition is performed on the image to be processed, and a target object and an interference object are determined.
The image to be processed may be an image to be subjected to image processing, for example, an image input by a user for generating a rainy day special effect. The target object may be an object in the image to be processed, which may be a person, an animal, a toy, a doll, etc., and the interfering object may be an object that recognizes interference to the main body of the image, for example, may be an object of a passerby, a anthropomorphic sculpture, a doll, etc. in the image to be processed.
The method for identifying the object of the image to be processed and determining the target object and the interference object may be various, for example, at least one candidate object and a candidate object area where the candidate object is located may be identified in the image to be processed, area attribute information of the candidate object area may be determined, and the target object and the interference object may be determined in the candidate object based on the area attribute information.
The candidate object may be an object identified in the image to be processed, for example, may be a person identified in the image to be processed, and the candidate object area may be an area where the candidate object is located in the image to be processed, for example, may be a detection box (Bbox, i.e., a target detection box) corresponding to the candidate object. The region attribute information may be information describing a region attribute of the candidate region, and the region attribute may include information such as a size of the candidate region and a position of the candidate region in the image to be processed.
The method for identifying at least one candidate object and the candidate object region where the candidate object is located in the image to be processed may be various, for example, a human body target detection algorithm may be sampled to identify at least one candidate object and the candidate object region where the candidate object is located in the image to be processed. Optionally, an image segmentation algorithm may be further used to identify and segment the image to be processed, so as to obtain at least one candidate object and a mask corresponding to each candidate object, so that contour information of contours corresponding to candidate objects in all mask images may be extracted by using a contour retrieval function (cv::: findContours), the contour information corresponding to the mask corresponding to each candidate object is stored in a contour array (contours array), and then a bounding box of contours of candidate objects in each mask image may be calculated by using a rectangular drawing function (cv:: boundingRect), so as to obtain relevant information such as a size and a position of the bounding box, that is, a candidate object area corresponding to the candidate object may be determined.
The method for determining the target object and the interference object in the candidate objects based on the region attribute information may be various, for example, the region size of the candidate object region corresponding to the candidate object may be compared according to the region attribute information, so that the candidate object corresponding to the largest candidate object region may be determined as the target object, then, the candidate object whose region size is smaller than one half of the largest candidate object region may be determined as the interference object, for the candidate object whose region size is larger than one half of the largest candidate object region, whether the candidate object is the target object or the interference object may be determined according to the position of the candidate object in the image to be processed, for example, when the region size of the candidate object region corresponding to the candidate object is larger than one half of the largest candidate object region and the candidate object region is in a middle position in the image to be processed, the candidate object may be determined as the target object when the region size of the candidate object region corresponding to the candidate object region is larger than one half of the largest candidate object region and the candidate object region is in a middle position in the image to be processed, and the candidate object may be determined as the actual object according to the actual situation.
In an embodiment, a user may mark the interference object and the target object in the image to be processed, so that the object identification of the image to be processed may be performed with auxiliary identification based on the mark information in the image to be processed, thereby improving the identification accuracy of the target object and the interference object. Specifically, when the object identification is performed on the image to be processed, the marking information and the object type information corresponding to the marking information can be identified in the image to be processed, so that the object identification can be performed on the area marked by the marking information, and the object type of the candidate object identified in the marked area can be determined according to the object type information corresponding to the marking information. The object type information may be information indicating an object type of an object, and the object type may include a target object, an interference object, and the like. For example, it is assumed that a user draws two rectangular areas (rectangular area 1 and rectangular area 2) in an image to be processed, wherein rectangular area 1 represents an object within the rectangular area as a target object, and rectangular area 2 represents an object within the rectangular area as an interference object, so that object recognition can be performed in rectangular area 1, the recognized candidate object is determined as a target object, object recognition is performed in rectangular area 2, and the recognized candidate object is determined as an interference object. Therefore, the target object and the interference object in the image to be processed can be accurately identified based on the mark information added by the user in the image to be processed, so that the dependence on the identification precision of the object identification method can be effectively reduced, the difficulty of object identification is greatly reduced, and the accuracy of object identification is further effectively improved.
In step 102, a target object mask map is acquired based on the target object.
The target object mask map may be a mask map (mask) of a target object, and the mask map may be a binary image composed of 0 and 1, for example, in the target object mask map, a value of a pixel belonging to the target object is 1, and a value of a pixel not belonging to the target object is 0.
The method for obtaining the target object mask map based on the target object may be various, for example, an object segmentation algorithm may be used to segment the target object in the image to be processed, so as to obtain the target object mask map corresponding to the target object.
In step 103, interference attenuation processing is performed on the image to be processed based on the interference object, and an interference attenuation image is obtained.
The interference attenuation image may be an image to be processed in which interference is attenuated, for example, an image to be processed in which an interference object is blurred or eliminated.
The method includes performing interference attenuation processing on an image to be processed based on an interference object, and obtaining an interference attenuation image may include obtaining an interference object mask map based on the interference object, removing content in an object image area where the interference object is located from the image to be processed according to the interference object mask map, obtaining an initial object image that does not include the interference object, performing image inpainting processing on an object image area in the initial object image based on neighborhood information of the object image area in the initial object image, and obtaining the interference attenuation image.
The interference object mask map may be a mask map of an interference object, the object image area may be a position of the interference object in the image to be processed, the initial object image may be the image to be processed from which the interference object is removed, and the neighborhood information may be information describing that the object image area is in a neighboring area of the initial object image.
The image inpainting process is performed on the object image area in the initial object image based on the neighborhood information of the object image area in the initial object image, and there are various ways to obtain the interference attenuation image, for example, the area size information of the object image area where the interference object is located may be obtained, the target area is determined in the adjacent area of the object image area based on the area size information, and the image content in the target area is filled into the object image area in the initial object image to obtain the interference attenuation image.
The region size information may be information describing the size of the object image region, for example, coordinates of the object image region in the image to be processed, length and width of the object image region, or size of a diagonal line indicating the object image region. The adjacent region may be a region adjacent to a region where the interference object is located in the initial object image. The target region may be a region for filling an object image region in the initial object image.
The target area may be determined in a neighboring area of the object image area based on the area size information in various ways, for example, an area having the same size as the object image area may be searched in the neighboring area of the object image area, and the searched area may be determined as the target area.
After the target area is determined in the vicinity of the object image area based on the area size information, the image content in the target area may be filled into the object image area in the initial object image, resulting in an interference-weakened image. The image content in the target area may be filled into the object image area in the initial object image in various manners, for example, the pixel value in the target area may be sampled, so that the pixel value of the pixel in the target area may be sampled into the object image area in the initial object image, so as to fill the object image area in the initial object image, and obtain the interference attenuation image.
Optionally, after the image content in the target area is filled into the object image area in the initial object image, a gaussian filter may be further used to smooth the filled initial object image, thereby providing an effect of interference attenuation. Therefore, the simple elimination of the interference object is realized based on the target area, and the image processing efficiency is improved under the condition that the requirement of special effect is met.
For example, referring to fig. 3, fig. 3 is an image schematic diagram of an image processing method provided in the embodiment of the present application, where a target object and an interference object are included in an image to be processed, mask0 of the image to be processed and mask1 of the interference object may be transmitted to a fragment shader to obtain mask0-mask1 with the interference object filtered, so that the interference object in the image to be processed may be filtered according to the mask0-mask1 to obtain an initial object image, then a target area with the same size as the target area is searched in an adjacent area of the target image area, and image content of the target area is filled into the initial object image with the interference object removed, thereby weakening the interference object,
in step 104, blurring processing is performed on the interference weakened image, so as to obtain a background weakened image.
The background weakening image may be an image obtained by blurring a background in the interference weakening image.
The method for obtaining the background weakening image by blurring the interference weakening image may be various, for example, a depth value of each pixel in the interference weakening image may be obtained, a blurring radius corresponding to the pixel is determined based on the depth value, and blurring processing is performed on the interference weakening image according to the blurring radius, so as to obtain the background weakening image.
The depth value may be a number of bits used to store each pixel, and the depth value of each pixel in the interference-reduced image may be obtained by obtaining a depth image of the interference-reduced image. The depth image may be obtained by a depth sensor, a depth camera or a rendering engine, and may be a gray scale image, wherein the gray scale value of each pixel represents the depth value of the pixel. The blur radius may be information indicating the degree of blur, the larger the blur radius, the larger the blurred atmosphere.
The method for determining the blur radius corresponding to the pixel based on the depth value may be various, for example, a focus depth range may be obtained, a focus pixel of the interference attenuation image is determined according to the depth value and the focus depth range, a depth difference between the pixel in the interference attenuation image and the focus pixel is calculated, and the blur radius corresponding to the pixel is determined based on the depth difference.
The depth range of the focus may be a depth value range used for determining a focus pixel, and may be set in a customized manner according to requirements, where the focus pixel may be a pixel that remains clear when blurring an interference-weakened image, and the depth difference may be a difference between depth values of the pixel and the focus pixel.
There are various ways to determine the blur radius corresponding to the pixel based on the depth difference, for example, the depth difference may be multiplied by a preset maximum blur radius, so as to obtain the blur radius corresponding to the pixel. The maximum blur radius may be a preset value, and the magnitude of the value may be determined according to an empirical value and the degree of blur required.
After the blur radius corresponding to the pixel is determined based on the depth value, the interference weakening image can be subjected to blur processing according to the blur radius, and a background weakening image is obtained. The method includes the steps of performing blurring processing on an interference weakening image according to a blurring radius to obtain a background weakening image, for example, determining a pixel to be blurred of a pixel in the interference weakening image based on the blurring radius, sampling a color value of the pixel to be blurred to obtain a first color value of the pixel to be blurred, performing weighted average processing on the first color value to obtain a second color value, performing fusion processing on the first color value and the second color value to obtain a target color value corresponding to the pixel to be blurred, and generating the background weakening image based on the target color value.
The pixel to be blurred may be a pixel to be blurred corresponding to the pixel determined based on a blur radius corresponding to the pixel, the first color value may be a color value of the pixel to be blurred, the second color value may be a color value obtained by weighted average of first color values of all pixels to be blurred corresponding to the current pixel, and the target color value may be a color value obtained by fusing the first color value and the second color value.
The method for determining the pixel to be blurred in the interference weakened image based on the blur radius may be various, for example, an area with a distance from the current pixel in the interference weakened image being within the blur radius may be determined, and the pixel in the area is determined as the pixel to be blurred corresponding to the current pixel.
The weighted average processing may be performed on the first color value to obtain a second color value, for example, the first color values of all pixels to be blurred corresponding to the current pixel may be accumulated to obtain an accumulated color value, so that the accumulated color value may be divided by the number of samples of the first color value corresponding to the current pixel to obtain the second color value.
The method includes that a first color value and a second color value are fused to obtain a target color value corresponding to a pixel to be blurred, various manners of generating a background weakening image based on the target color value can be adopted, for example, the first color value and the second color value can be weighted and summed according to a preset weight to obtain the target color value, and the specific weight can be set according to an actual situation and a blurring degree.
In step 105, the target object in the image to be processed and the image content except for the target object in the background weakening image are fused based on the target object mask map, so as to obtain a target object image.
The target object image may be an image in which the target object and the image content other than the target object in the background-weakened image are fused.
The method includes fusing the image content except the target object in the target object mask image and the background weakening image in the image to be processed based on the target object mask image, and obtaining the target object image may have various modes, for example, the target object mask image and the image to be processed may be mixed to obtain a first image only including the target object, an anti-mask image corresponding to the target object mask image is obtained, and the anti-mask image and the background weakening image are mixed to obtain a second image not including the area where the target object is located, so that the first image and the second image may be subjected to image superposition to obtain the target object image including the clear target object, eliminating the interference object and blurring the background.
The anti-mask may be a mask obtained by binary inversion of the gray value in the target mask, for example, if the gray value of the pixel a is 1 and the gray value of the pixel b is 0 in the target mask, the gray value of the pixel a is 0 and the gray value of the pixel b is 1 in the anti-mask.
In step 106, the preset rainy day special effect image and the target object image are mixed to obtain the target special effect image.
The rainy special effect image may be an image including a rainy special effect, and the target special effect image may be an image representing that a rainy scene has a prominent target object body, weakens an interference object, and has an image background blurring effect, for example, please refer to fig. 4, fig. 4 is a schematic diagram of a rainy special effect image of an image processing method provided in the embodiment of the present application, and the rainy special effect image simulates a rainy effect, so that a finally generated target special effect image has a rainy atmosphere effect.
Alternatively, a rainy day special effect map of the rainy effect may be generated by designing various functions, for example, a random value t may be constructed for simulating the water flow of random raindrops, so that an attenuation coefficient for transforming the color of the raindrops may be calculated according to the value t, and further, the color value of the raindrops may be multiplied by a linear interpolation result, and the multiplication result is used as a result of the color transformation of the raindrops, so as to implement fine color shift of the color of the raindrops. The parameters of blurring of the surface of the glass, flowing track of raindrops on the glass, adjustment of rainfall and the like in the rain special effect diagram can be designed, and the abundant, natural and attractive raining effect is realized.
Optionally, the raindrop effect graph of the multiple layers can be generated, so that the raindrop effect graphs on the multiple layers can be overlapped, a raindrop effect graph with richer raining effect can be obtained, the display effect of the raindrop effect image is improved, and further the creation enthusiasm of a user is improved.
From the above, in the embodiment of the present application, the object recognition is performed on the image to be processed, so as to determine the target object and the interference object; acquiring a target object mask map based on the target object; performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image; performing fuzzy processing on the interference weakening image to obtain a background weakening image; fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image; and mixing the preset rainy day special effect image with the target object image to obtain the target special effect image. In this way, the target object and the interference object in the image to be processed are identified, so that interference attenuation processing is performed on the image to be processed according to the interference object, blurring processing is performed on the interference attenuation image to obtain a background attenuation image, image contents except the target object in the target object and the background attenuation image are fused to obtain a target object image which highlights the target object, and then the target object image is mixed with a preset rainy day special effect image, so that the target special effect image which highlights the target object main body, weakens the interference object and the image background blurring effect in a rainy day scene can be obtained, the rainy day special effect is enriched, and the image processing effect is further improved.
In order to better implement the above method, the embodiment of the present invention further provides an image processing apparatus, which may be integrated in a computer device, and the computer device may be a server or a terminal.
For example, as shown in fig. 5, a schematic structural diagram of an image processing apparatus provided in an embodiment of the present application may include an object recognition unit 201, a mask acquisition unit 202, an interference attenuation unit 203, a background blurring unit 204, an image fusion unit 205, and an image mixing unit 206, as follows:
an object recognition unit 201, configured to perform object recognition on an image to be processed, and determine a target object and an interference object;
a mask acquisition unit 202 for acquiring a target object mask map based on the target object;
an interference attenuation unit 203, configured to perform interference attenuation processing on an image to be processed based on an interference object, so as to obtain an interference attenuation image;
the background blurring unit 204 is configured to perform blurring processing on the interference weakened image to obtain a background weakened image;
an image fusion unit 205, configured to fuse, based on the target object mask map, a target object in the image to be processed and image contents except the target object in the background attenuation image, so as to obtain a target object image;
The image mixing unit 206 is configured to mix the preset rainy day special effect image with the target object image to obtain the target special effect image.
In an embodiment, the interference weakening unit 203 comprises:
an interference mask acquisition subunit, configured to acquire an interference object mask map based on the interference object;
the interference object removing subunit is used for removing the content in the object image area where the interference object is located from the image to be processed according to the interference object mask graph to obtain an initial object image which does not contain the interference object;
and the image inpainting subunit is used for carrying out image inpainting processing on the object image area in the initial object image based on the neighborhood information of the object image area in the initial object image to obtain an interference weakening image.
In an embodiment, the image inpainting subunit is configured to:
acquiring area size information of an object image area where an interference object is located;
determining a target region in a neighboring region of the object image region based on the region size information;
and filling the image content in the target area into the object image area in the initial object image to obtain the interference weakening image.
In an embodiment, the object recognition unit 201 is configured to:
Identifying at least one candidate object and a candidate object area where the candidate object is located in the image to be processed;
determining region attribute information of a candidate object region;
and determining a target object and an interference object from the candidate objects based on the region attribute information.
In one embodiment, the background blurring unit 204 includes:
a depth value obtaining subunit, configured to obtain a depth value of each pixel in the interference weakened image;
a blur radius determination subunit, configured to determine a blur radius corresponding to the pixel based on the depth value;
and the background blurring subunit is used for blurring the interference weakening image according to the blurring radius to obtain a background weakening image.
In an embodiment, the blur radius determination subunit is configured to:
acquiring a focal depth range;
determining focus pixels of the interference weakened image according to the depth value and the focus depth range;
calculating depth differences between pixels in the interference attenuation image and the focus pixels;
and determining the blur radius corresponding to the pixel based on the depth difference.
In an embodiment, the background blurring subunit is configured to:
determining a pixel to be blurred of the pixel in the interference weakened image based on the blur radius;
sampling the color value of the pixel to be blurred to obtain a first color value of the pixel to be blurred;
Carrying out weighted average processing on the first color value to obtain a second color value;
performing fusion processing on the first color value and the second color value to obtain a target color value corresponding to the pixel to be blurred;
a background-attenuated image is generated based on the target color values.
In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.
As can be seen from the above, in the embodiment of the present application, the object recognition unit 201 performs object recognition on the image to be processed, so as to determine the target object and the interference object; the mask acquisition unit 202 acquires a target object mask map based on the target object; the interference attenuation unit 203 performs interference attenuation processing on the image to be processed based on the interference object to obtain an interference attenuation image; the background blurring unit 204 performs blurring processing on the interference weakened image to obtain a background weakened image; the image fusion unit 205 fuses the target object in the image to be processed and the image content except the target object in the background attenuation image based on the target object mask map to obtain a target object image; the image mixing unit 206 mixes the preset rainy day special effect image with the target object image to obtain the target special effect image. In this way, the target object and the interference object in the image to be processed are identified, so that interference attenuation processing is performed on the image to be processed according to the interference object, blurring processing is performed on the interference attenuation image to obtain a background attenuation image, image contents except the target object in the target object and the background attenuation image are fused to obtain a target object image which highlights the target object, and then the target object image is mixed with a preset rainy day special effect image, so that the target special effect image which highlights the target object main body, weakens the interference object and the image background blurring effect in a rainy day scene can be obtained, the rainy day special effect is enriched, and the image processing effect is further improved.
The embodiment of the application further provides a computer device, as shown in fig. 6, which shows a schematic structural diagram of the computer device according to the embodiment of the application, where the computer device may be a server or a terminal, specifically:
the computer device may include one or more processing cores 'processors 301, one or more computer-readable storage media's memory 302, power supply 303, and input unit 304, among other components. Those skilled in the art will appreciate that the computer device structure shown in FIG. 6 is not limiting of the computer device and may include more or fewer components than shown, or may be combined with certain components, or a different arrangement of components. Wherein:
processor 301 is the control center of the computer device and uses various interfaces and lines to connect the various parts of the overall computer device, perform various functions of the computer device and process data by running or executing software programs and/or modules stored in memory 302, and invoking data stored in memory 302. Optionally, processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor primarily handles operating systems, user interfaces, applications, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 301.
The memory 302 may be used to store software programs and modules, and the processor 301 executes various functional applications and data processing by executing the software programs and modules stored in the memory 302. The memory 302 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the computer device, etc. In addition, memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.
The computer device further includes a power supply 303 for powering the various components, preferably, the power supply 303 is logically connected to the processor 301 by a power management system, such that functions such as managing charging, discharging, and power consumption are performed by the power management system. The power supply 303 may also include one or more of any components, such as a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The computer device may also include an input unit 304, which input unit 304 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the computer device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 301 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 executes the application programs stored in the memory 302, so as to implement various functions as follows:
object identification is carried out on the image to be processed, and a target object and an interference object are determined; acquiring a target object mask map based on the target object; performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image; performing fuzzy processing on the interference weakening image to obtain a background weakening image; fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image; and mixing the preset rainy day special effect image with the target object image to obtain the target special effect image.
The specific implementation of each operation may be referred to the previous embodiments, and will not be described herein. It should be noted that, the computer device provided in the embodiment of the present application and the image processing method applicable to the above embodiment belong to the same concept, and detailed implementation processes of the computer device are shown in the above method embodiment, which is not repeated herein.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, embodiments of the present application provide a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform steps in any of the image processing methods provided by the embodiments of the present application. For example, the instructions may perform the steps of:
object identification is carried out on the image to be processed, and a target object and an interference object are determined; acquiring a target object mask map based on the target object; performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image; performing fuzzy processing on the interference weakening image to obtain a background weakening image; fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image; and mixing the preset rainy day special effect image with the target object image to obtain the target special effect image.
Wherein the computer-readable storage medium may comprise: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
Because the instructions stored in the computer readable storage medium may execute the steps in any of the image processing methods provided in the embodiments of the present application, the beneficial effects that any of the image processing methods provided in the embodiments of the present application can be achieved are detailed in the previous embodiments, and are not described herein.
Among other things, according to one aspect of the present application, a computer program product or computer program is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in the various alternative implementations provided in the above embodiments.
The foregoing has described in detail the methods, apparatuses and computer readable storage medium for image processing provided by the embodiments of the present application, and specific examples have been applied herein to illustrate the principles and implementations of the present application, and the description of the foregoing examples is only for the purpose of aiding in the understanding of the methods and core ideas of the present application; meanwhile, those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, and the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. An image processing method, comprising:
object identification is carried out on the image to be processed, and a target object and an interference object are determined;
acquiring a target object mask map based on the target object;
performing interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image;
performing fuzzy processing on the interference weakening image to obtain a background weakening image;
fusing a target object in the image to be processed and image contents except the target object in the background weakening image based on the target object mask image to obtain a target object image;
and mixing a preset rainy day special effect image with the target object image to obtain a target special effect image.
2. The image processing method according to claim 1, wherein the performing interference attenuation processing on the image to be processed based on the interference object to obtain an interference attenuation image includes:
acquiring an interference object mask map based on the interference object;
removing the content in the object image area where the interference object is located from the image to be processed according to the interference object mask diagram to obtain an initial object image which does not contain the interference object;
And performing image patching processing on the object image area in the initial object image based on the neighborhood information of the object image area in the initial object image to obtain an interference weakening image.
3. The image processing method according to claim 2, wherein the performing image inpainting processing on the object image area in the initial object image based on neighborhood information of the object image area in the initial object image to obtain the interference-weakened image includes:
acquiring the region size information of the object image region where the interference object is located;
determining a target region in a neighboring region of the object image region based on the region size information;
and filling the image content in the target area into the object image area in the initial object image to obtain an interference weakening image.
4. The image processing method according to claim 1, wherein the performing object recognition on the image to be processed to determine the target object and the interference object includes:
identifying at least one candidate object and a candidate object area where the candidate object is located in the image to be processed;
Determining region attribute information of the candidate object region;
and determining a target object and an interference object from the candidate objects based on the region attribute information.
5. The image processing method according to any one of claims 1 to 4, wherein the blurring processing is performed on the interference-weakened image to obtain a background-weakened image, and the method includes:
acquiring a depth value of each pixel in the interference weakening image;
determining a blur radius corresponding to the pixel based on the depth value;
and carrying out fuzzy processing on the interference weakening image according to the fuzzy radius to obtain a background weakening image.
6. The image processing method of claim 5, wherein the determining the blur radius corresponding to the pixel based on the depth value comprises:
acquiring a focal depth range;
determining a focus pixel of the interference attenuation image according to the depth value and the focus depth range;
calculating depth differences between pixels in the interference attenuation image and the focus pixels;
and determining the blur radius corresponding to the pixel based on the depth difference.
7. The image processing method according to claim 5, wherein the blurring processing the interference-weakened image according to the blurring radius to obtain a background-weakened image includes:
Determining a pixel to be blurred of the pixel in the interference-weakened image based on the blur radius;
sampling the color value of the pixel to be blurred to obtain a first color value of the pixel to be blurred;
performing weighted average processing on the first color value to obtain a second color value;
performing fusion processing on the first color value and the second color value to obtain a target color value corresponding to the pixel to be blurred;
a background-attenuated image is generated based on the target color value.
8. An image processing apparatus, comprising:
the object recognition unit is used for recognizing the object of the image to be processed and determining a target object and an interference object;
a mask acquisition unit configured to acquire a target object mask map based on the target object;
the interference weakening unit is used for carrying out interference weakening processing on the image to be processed based on the interference object to obtain an interference weakening image;
the background blurring unit is used for blurring the interference weakening image to obtain a background weakening image;
the image fusion unit is used for fusing the target object in the image to be processed and the image content except the target object in the background weakening image based on the target object mask image to obtain a target object image;
And the image mixing unit is used for mixing the preset rainy day special effect image with the target object image to obtain a target special effect image.
9. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the image processing method of any of claims 1 to 7.
10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the image processing method of any of claims 1 to 7 when the computer program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311685631.7A CN117649336A (en) | 2023-12-08 | 2023-12-08 | Image processing method, device and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311685631.7A CN117649336A (en) | 2023-12-08 | 2023-12-08 | Image processing method, device and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117649336A true CN117649336A (en) | 2024-03-05 |
Family
ID=90043163
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311685631.7A Pending CN117649336A (en) | 2023-12-08 | 2023-12-08 | Image processing method, device and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117649336A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4607452A1 (en) * | 2024-02-26 | 2025-08-27 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and computer-readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752537A (en) * | 2011-11-29 | 2012-10-24 | 新奥特（北京）视频技术有限公司 | Method for outputting various stunt effects simultaneously |
| US20190318457A1 (en) * | 2017-12-20 | 2019-10-17 | Shenzhen Sensetime Technology Co., Ltd | Image processing methods and apparatuses, computer readable storage media and electronic devices |
| CN111556278A (en) * | 2020-05-21 | 2020-08-18 | 腾讯科技（深圳）有限公司 | Video processing method, video display device and storage medium |
| US20200334793A1 (en) * | 2018-01-05 | 2020-10-22 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and device for blurring image background, storage medium and electronic apparatus |
| CN114187172A (en) * | 2021-12-15 | 2022-03-15 | Tcl通讯科技（成都）有限公司 | Image fusion method and device, computer equipment and computer readable storage medium |
| CN114677386A (en) * | 2022-03-25 | 2022-06-28 | 北京字跳网络技术有限公司 | Special effect image processing method and device, electronic equipment and storage medium |
-
2023
- 2023-12-08 CN CN202311685631.7A patent/CN117649336A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752537A (en) * | 2011-11-29 | 2012-10-24 | 新奥特（北京）视频技术有限公司 | Method for outputting various stunt effects simultaneously |
| US20190318457A1 (en) * | 2017-12-20 | 2019-10-17 | Shenzhen Sensetime Technology Co., Ltd | Image processing methods and apparatuses, computer readable storage media and electronic devices |
| US20200334793A1 (en) * | 2018-01-05 | 2020-10-22 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and device for blurring image background, storage medium and electronic apparatus |
| CN111556278A (en) * | 2020-05-21 | 2020-08-18 | 腾讯科技（深圳）有限公司 | Video processing method, video display device and storage medium |
| CN114187172A (en) * | 2021-12-15 | 2022-03-15 | Tcl通讯科技（成都）有限公司 | Image fusion method and device, computer equipment and computer readable storage medium |
| CN114677386A (en) * | 2022-03-25 | 2022-06-28 | 北京字跳网络技术有限公司 | Special effect image processing method and device, electronic equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4607452A1 (en) * | 2024-02-26 | 2025-08-27 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and computer-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113505854B (en) | Face image quality evaluation model construction method, device, equipment and medium | |
| WO2022022154A1 (en) | Facial image processing method and apparatus, and device and storage medium | |
| CN114372931B (en) | A method, device, storage medium and electronic device for blurring a target object | |
| CN114332467B (en) | Image processing method, device, computer and readable storage medium | |
| CN113537187A (en) | Text recognition method, device, electronic device and readable storage medium | |
| CN114529490A (en) | Data processing method, device, equipment and readable storage medium | |
| CN113569608A (en) | Text recognition method, device and equipment based on deep learning and storage medium | |
| CN112749606A (en) | Text positioning method and device | |
| CN108648189A (en) | Image fuzzy detection method, apparatus, computing device and readable storage medium storing program for executing | |
| CN115115691A (en) | Monocular three-dimensional plane recovery method, equipment and storage medium | |
| CN117649336A (en) | Image processing method, device and computer-readable storage medium | |
| CN110969641A (en) | Image processing method and device | |
| CN113537186A (en) | Text image recognition method and device, electronic equipment and storage medium | |
| CN115115399A (en) | Object recommendation method, device, equipment, medium and computer program product | |
| CN113840169B (en) | A video processing method, device, computing device and storage medium | |
| CN117315406B (en) | Sample image processing method, device and equipment | |
| CN117541546A (en) | Method and device for determining image cropping effect, storage medium and electronic equipment | |
| CN113449559A (en) | Table identification method and device, computer equipment and storage medium | |
| CN118469850A (en) | Image processing method and device | |
| CN116798041A (en) | Image recognition method and device and electronic equipment | |
| CN117011158A (en) | Image processing method, device and computer-readable storage medium | |
| CN116993968A (en) | Training data acquisition method and device, storage medium, and program product | |
| CN113822959A (en) | A method, device, computer equipment and storage medium for generating Chinese painting | |
| CN114332267B (en) | Method, device, computer equipment and medium for generating ink painting images | |
| CN116798052B (en) | Training method and device of text recognition model, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20240305 |