[go: up one dir, main page]

WO2018137623A1 - Procédé et appareil de traitement d'image, et dispositif électronique - Google Patents

Procédé et appareil de traitement d'image, et dispositif électronique Download PDF

Info

Publication number
WO2018137623A1
WO2018137623A1 PCT/CN2018/073882 CN2018073882W WO2018137623A1 WO 2018137623 A1 WO2018137623 A1 WO 2018137623A1 CN 2018073882 W CN2018073882 W CN 2018073882W WO 2018137623 A1 WO2018137623 A1 WO 2018137623A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
information
target object
contour
sample set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/073882
Other languages
English (en)
Chinese (zh)
Inventor
刘建博
严琼
鲍旭
王子彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Sensetime Technology Co Ltd
Original Assignee
Shenzhen Sensetime Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Sensetime Technology Co Ltd filed Critical Shenzhen Sensetime Technology Co Ltd
Publication of WO2018137623A1 publication Critical patent/WO2018137623A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/752—Contour matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20004—Adaptive image processing
    • G06T2207/20012—Locally adaptive

Definitions

  • the embodiments of the present application relate to image processing technologies, and in particular, to an image processing method, apparatus, and electronic device.
  • the embodiment of the present application provides an image processing technical solution.
  • an image processing method includes: determining target object information from an image to be processed; determining a foreground region and the foreground region in the image according to the target object information and a predetermined object contour template. a background area; performing blurring processing on the foreground area and/or the background area.
  • the determining the foreground area and the background area in the image according to the target object information and the predetermined object contour template comprises: matching at least a partial area in the object outline template with the target object information; Determining difference information between an object contour in the object contour template and a contour of the target object in the image according to the matching result; adjusting an object contour in the object contour template according to the difference information; adjusting the object A contour is mapped into the image to obtain a foreground region of the image including the target object and a background region including at least a portion of the foreground region.
  • the difference information includes: scaling information, offset information, and/or angle information between an object contour in the object contour template and a contour of the target object in the image.
  • the image comprises: a still image or a video frame image.
  • the image is a video frame image
  • the determining target object information from the image to be processed includes: according to target object information determined from a video frame image before the video frame image to be processed, Determining the target image information by the video frame image to be processed; or determining the target object information in each video frame image in the video stream by performing video-by-video frame image detection on the video stream to be processed.
  • the image processing method further includes: determining a transition region between the foreground region and the background region; and performing a blurring process on the transition region.
  • the performing the blurring process on the transition region comprises: performing progressive blurring processing or spot processing on the transition region.
  • the determining target object information from the image to be processed includes: acquiring object selection information; and determining the target object information from the to-be-processed image according to the object selection information.
  • the determining target object information from the image to be processed includes: detecting a target object from the image to be processed, and obtaining the target object information.
  • detecting the target object from the image to be processed obtaining the target object information, including: detecting a target object from the image to be processed through a pre-trained depth neural network, and obtaining the target object information .
  • the target object information includes any one or more of the following: face information, license plate information, house number information, address information, identity ID information, and trademark information.
  • the face information includes any one or more of the following: information of a face key point, face position information, face size information, and face angle information.
  • the object contour template includes any one or more of the following: a face contour template, a human body contour template, a license plate contour template, a house card contour template, and a predetermined frame contour template.
  • the object contour template includes: a plurality of human body contour templates respectively corresponding to different human face angles; and determining the foreground area and the background area in the image according to the target object information and the predetermined object contour template And further comprising: determining, from the object contour template, a human body contour template corresponding to the face angle information in the face information.
  • the depth neural network is configured to detect face key point information and pre-training by: acquiring a first sample set, the first sample set including a plurality of unlabeled sample images; a neural network, performing key point position labeling on each of the unlabeled sample images in the first sample set to obtain a second sample set, wherein the deep neural network is used to perform key point positioning on the image; The partial sample image and the third sample set in the second sample set are adjusted to adjust parameters of the deep neural network, wherein the third sample set includes a plurality of labeled sample images.
  • the depth point neural network is used to perform key point position labeling on each of the unlabeled sample images in the first sample set to obtain a second sample set, including: collecting the first sample set Each of the unlabeled sample images is subjected to image transformation processing to obtain a fourth sample set; wherein the image transformation process includes any one or more of the following: rotation, translation, scaling, noise addition, and occlusion; a deep neural network, performing key point position labeling on the fourth sample set and each sample image in the first sample set to obtain the second sample set.
  • the parameter of the deep neural network is adjusted according to at least part of the sample image and the third sample set in the second sample set, including: for each unlabeled sample image in the first sample set, Determining, according to the key point position information of the unlabeled sample image, the key point position information of the unlabeled sample image is an optional sample; wherein the key point position information of the unlabeled sample image is And key point position information after performing image transformation processing are all included in the second sample set; adjusting the deep neural network according to each of the selectable samples and the third sample set in the second sample set Parameters.
  • the face key point includes any one or more of the following: an eye key point, a nose key point, a mouth key point, an eyebrow key point, and a face contour key point.
  • an image processing apparatus including: an object information determining module, configured to determine target object information from an image to be processed; a front background determining module, configured to use the object information according to the object information Determining the target object information determined by the module and the predetermined object contour template to determine a foreground area and a background area in the image; a blurring processing module, configured to determine a foreground area and/or the background area of the front background determining module Perform blurring.
  • the front background determining module includes: a template matching unit, configured to match at least a local area in the object contour template with the target object information; and a difference determining unit, configured to perform, according to the template matching unit
  • the matching result determines difference information between the object contour in the object contour template and the contour of the target object in the image
  • the contour adjusting unit is configured to adjust the object contour template according to the difference information determined by the difference determining unit An object contour
  • a front background determining unit configured to map an object contour adjusted by the contour adjusting unit into the image, obtain a foreground region including the target object in the image, and include at least a portion of the foreground region Background area.
  • the difference information includes: scaling information, offset information, and/or angle information between an object contour in the object contour template and a contour of the target object in the image.
  • the image comprises: a still image or a video frame image.
  • the image is a video frame image
  • the object information determining module includes: a first object information determining unit, configured to determine, according to the target object information determined from the video frame image before the video frame image to be processed, Determining the target object information from the to-be-processed video frame image; or, the second object information determining unit is configured to perform video-by-video frame image detection by the video stream to be processed, and determine each video frame image in the video stream.
  • Target object information in .
  • the image processing apparatus further includes: a transition area determining module, configured to determine a transition area between the foreground area and the background area; and a transition blur processing module, configured to determine a transition of the transition area determining module The area is blurred.
  • the transition blur processing module is configured to perform progressive blurring processing or spot processing on the transition region.
  • the object information determining module includes: a selection information acquiring unit, configured to acquire object selection information; and a third object information determining unit, configured to: according to the object selection information acquired by the selection information acquiring unit, from the to-be-processed The target object information is determined in the image.
  • the object information determining module includes: a fourth object information determining unit, configured to detect a target object from the image to be processed, and obtain the target object information.
  • the fourth object information determining unit is configured to detect the target object from the image to be processed by using a pre-trained depth neural network to obtain the target object information.
  • the target object information includes any one or more of the following: face information, license plate information, house number information, address information, identity ID information, and trademark information.
  • the face information includes any one or more of the following: information of a face key point, face position information, face size information, and face angle information.
  • the object contour template includes any one or more of the following: a face contour template, a human body contour template, a license plate contour template, a house card contour template, and a predetermined frame contour template.
  • the object contour template includes: a plurality of human body contour templates respectively corresponding to different human face angles; the front background determining module is further configured to determine the image according to the target object information and a predetermined object contour template. Before the foreground area and the background area, a human body contour template corresponding to the face angle information in the face information is determined from the object outline templates.
  • the device further includes: a sample set obtaining module, configured to acquire a first sample set, the first sample set includes a plurality of unlabeled sample images; and a key point position labeling module is configured to be based on the deep nerve a network, performing key point position labeling on each of the unlabeled sample images in the first sample set to obtain a second sample set, wherein the deep neural network is used for key point positioning of the image; the network parameter adjustment module And a parameter for adjusting the depth neural network according to at least a partial sample image and a third sample set in the second sample set, wherein the third sample set includes a plurality of labeled sample images.
  • a sample set obtaining module configured to acquire a first sample set, the first sample set includes a plurality of unlabeled sample images
  • a key point position labeling module is configured to be based on the deep nerve a network, performing key point position labeling on each of the unlabeled sample images in the first sample set to obtain a second sample set, wherein the deep neural
  • the key point location labeling module includes: an image transform processing unit, configured to perform image transform processing on each of the unlabeled sample images in the first sample set to obtain a fourth sample set;
  • the image transformation process includes any one or more of the following: rotation, translation, scaling, noise addition, and occlusion; a key point location unit for using the fourth sample set and the location based on the depth neural network
  • Each of the unlabeled sample images in the first sample set performs key point position labeling to obtain the second sample set.
  • the network parameter adjustment module includes: an optional sample determining unit, configured to perform, for each unlabeled sample image in the first sample set, a key point after image transformation processing based on the unlabeled sample image Position information, determining whether the key point position information of the unlabeled sample image is an optional sample; wherein the key point position information of the unlabeled sample image and the key point position information after performing image transformation processing are included in And the network parameter adjustment unit is configured to adjust parameters of the deep neural network according to each of the selectable samples and the third sample set in the second sample set.
  • the face key point includes any one or more of the following: an eye key point, a nose key point, a mouth key point, an eyebrow key point, and a face contour key point.
  • an electronic device including:
  • another electronic device including: a processor and a memory;
  • the memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to any of the image processing methods described above.
  • a computer program comprising computer readable code, the processor in the device executing the above-described implementation of the present application when the computer readable code is run on a device
  • a computer readable storage medium for storing computer readable instructions, and when the instructions are executed, implementing the image processing method according to any one of the foregoing embodiments of the present application. The operation of each step in the process.
  • the image to be processed is detected to determine target object information, and the foreground area and the background area in the image to be processed are acquired according to the determined target object information and the object contour template, and then The background area and/or the foreground area are blurred, so that the foreground area or the background area in which the blurring process needs to be performed can be automatically determined by the target object information detected from the image without manually marking the user to perform the blurring process
  • the area or manual execution of the blur (blur) operation improves the convenience and accuracy of the operation.
  • FIG. 1 is a flow chart of an image processing method according to an embodiment of the present application.
  • FIG. 2 is a flowchart of an image processing method according to another embodiment of the present application.
  • FIG. 3 is a flowchart of an image processing method according to still another embodiment of the present application.
  • FIG. 4 is a schematic diagram of an exemplary character outline template including a human body and a face contour template including a human face in the embodiment of the present application;
  • FIG. 5 is a flow chart of an exemplary method of training a keypoint location model in an embodiment of the present application
  • FIG. 6 is a logic block diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG. 7 is a logic block diagram showing an image processing apparatus according to another embodiment of the present application.
  • FIG. 8 is a logic block diagram showing an image processing apparatus according to still another embodiment of the present application.
  • FIG. 9 is a schematic structural view showing an electronic device according to an embodiment of the present application.
  • Embodiments of the present application can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with numerous other general purpose or special purpose computing system environments or configurations.
  • Examples of well-known terminal devices, computing systems, environments, and/or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, and the like include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients Machines, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, small computer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above, and the like.
  • Electronic devices such as terminal devices, computer systems, servers, etc., can be described in the general context of computer system executable instructions (such as program modules) being executed by a computer system.
  • program modules may include routines, programs, target programs, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types.
  • the computer system/server can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communication network.
  • program modules may be located on a local or remote computing system storage medium including storage devices.
  • FIG. 1 is a flow chart of an image processing method according to an embodiment of the present application.
  • the image processing method can be implemented in any terminal device, personal computer or server. Referring to FIG. 1, the image processing method of this embodiment includes:
  • target object information is determined from the image to be processed.
  • the image to be processed has a certain resolution, and may be an image taken by using a shooting device (such as a mobile phone, a digital camera, a camera, etc.), or may be a pre-stored image (such as an image in a mobile phone album). It can also be an image in a video sequence.
  • the image may be an image of a subject, an animal, a vehicle, an object (such as a business card, an ID card, a license plate, etc.). If the image is a person image, the image may also be a portrait (close-up), a bust, or a full-body photo.
  • the target object information can be determined/detected from the image to be processed by any suitable image analysis technique.
  • the detected target object information can be used to locate the area occupied by the target object in the image.
  • the target object information may include, but is not limited to, any one or more of the following: location, size of the target object, information of the key part (such as the position of the nose, face position and size, etc.), key points of the target object, targets The attribute information of the object (such as the skin color of the person).
  • the step S110 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by the object information determining module 610 being executed by the processor.
  • step S120 a foreground area and a background area in the image are determined based on the target object information and a predetermined object outline template.
  • the target object information determined in step S110 can be used to locate an area occupied by the target object in the image, and thus can be distinguished according to the determined target object information and the object outline template representing the shape and proportional relationship of the target object.
  • An area occupied by the target object in the image to be processed, and an area occupied by the target object in the image to be processed is determined as a foreground area of the image, and at least a part of the image area outside the foreground area is determined as a background region.
  • the human face has a relatively determined position and proportional relationship in the whole human body, and can match the detected target object information with the character outline template that characterizes the shape and proportion of the human body, thereby delineating that the character occupies the image to be processed.
  • the area is used as the foreground area, and all or part of the area other than the foreground area in the image to be processed is determined as the background area.
  • the step S120 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a front background determination module 620 that is executed by the processor.
  • step S130 the determined foreground area and/or background area is subjected to blurring processing.
  • the background area and/or the foreground area may be blurred according to the needs of the application scenario.
  • the determined background area may be blurred to highlight the captured target object in the image screen to improve the shooting effect; or the foreground area (such as the character area or the license plate) may be blurred to blur the display target.
  • the object person, ID number, or license plate number, etc. protects the privacy information; or, the determined background area and foreground area may be blurred at the same time.
  • the foreground area and/or the background area may be blurred by using any suitable image blurring technique.
  • the blurring filter can be used to blur the background area and/or the foreground area, that is, to change the adjacent pixel values by Gaussian filtering to achieve a blurred visual effect.
  • the above is only an exemplary implementation, and the foreground area and/or the background area may be blurred by any other blurring method.
  • the step S130 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a blurring processing module 630 executed by the processor.
  • the image to be processed is detected to determine target object information, and the foreground area and the background area in the image to be processed are acquired according to the determined target object information and the object contour template, and then The background area and/or the foreground area are blurred, so that the foreground area or the background area in which the blurring process needs to be performed can be automatically determined by the target object information detected from the image without manually marking the user to perform the blurring process
  • the area or manual execution of the blur (blur) operation improves the convenience and accuracy of the operation.
  • FIG. 2 is a flow chart of an image processing method according to another embodiment of the present application.
  • the image processing method of this embodiment includes:
  • step S210 target object information is determined from the image to be processed.
  • the target object may be a character, an animal, or any object (such as a license plate, a vehicle, an ID card).
  • the determined target object information may include any one or more of the following: face information, license plate information, house number information, address information, identification (ID) information, trademark information, but is not limited to the above information.
  • the target object information each characterizes at least a portion of the features of the target object in the image in which the target object is captured.
  • the step S210 may include step S212 and step S213.
  • object selection information is acquired, and the object selection information may be, for example, information of an image area specified by (user), identification (ID) information of an object, information of an object type, and the like.
  • ID identification
  • target object information is determined from the image to be processed based on the object selection information. For example, the target object information is determined in the specified image area based on the information of the image area specified by the user.
  • the steps S21 and S213 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a selection information acquiring unit 6103 and a third object information determining unit 6104, respectively, which are executed by the processor.
  • the image can be detected based on the object selection information provided separately, and the target object information can be acquired.
  • step S210 may include: S214: detecting a target object from the image to be processed, and obtaining the detected target object information. That is to say, the target object is first detected from the image, and the target object information is determined according to the detected target object.
  • the target object may be detected from the image to be processed through a pre-trained deep neural network to obtain the detected target object information.
  • a deep neural network for detecting a target object may be pre-trained by a sample image labeled with object object information for detecting a deep neural network of a target object such as a vehicle, a face, a pedestrian, an animal, or the like.
  • the image to be processed is input to the deep neural network, and the target object information is acquired by the detection process of the deep neural network.
  • the image to be processed may be a still image captured, a video frame image in the recorded video content, or a video frame image in the online video stream.
  • step S210 may include: S215, determining the target object information from the video frame image to be processed according to the target object information determined from the previous video frame image.
  • the position and size of the same target object between successive video frames are relatively close. Therefore, the video to be processed can be detected from the current video frame image to be detected according to the target object information determined from the previous or previous video frame images.
  • the target object information of the frame image thereby improving the detection efficiency.
  • the step S215 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by the first object information determining unit 6101 being executed by the processor.
  • step S210 may include: S216, performing video-by-video frame image detection by the video stream to be processed, and determining target object information in each video frame image in the video stream.
  • step S216 may be performed by the processor invoking a corresponding instruction stored in the memory, or may be performed by the second object information determining unit 6102 being executed by the processor.
  • video frame in the video stream to be processed mentioned above may represent a real frame in the video stream, and may also be represented as a sample frame in the video stream that needs to be processed. .
  • the target object information is detected from the image to be processed by the processing of any of the foregoing embodiments.
  • the step S210 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by the object information determining module 610 executed by the processor.
  • step S220 a foreground area and a background area in the image are determined according to the target object information and a predetermined object outline template.
  • step S220 in this embodiment includes the following steps S221, S223, S225, S227, and S229.
  • step S221 at least a partial area in the object contour template is matched with the determined target object information.
  • the object contour template can be preset to outline the outline of the target object that may appear in the image, or that is of interest or to be detected.
  • a character outline template, a car outline template, a dog's outline template, and the like may be set in advance for matching with the target object information.
  • the object contour template may include, but is not limited to, any one or more of the following: a face contour template, a human body contour template, a license plate contour template, a house card contour template, a predetermined frame. Outline templates, etc.
  • the face contour template is used to match the silhouette of the person in the recent photo of the person
  • the human body contour template is used to match the silhouette of the person in the whole body or the half body photo
  • the license plate contour template is used to match the license plate contour on the vehicle in the image, the predetermined frame
  • the contour template is used to match the contour of an object having a predetermined shape, such as an identity card or the like.
  • At least a local area in the object contour template may be matched with the determined target object information.
  • the determined target object information is the license plate information of the vehicle
  • the contour template of the front side of the vehicle can be matched with respect to the position of the license plate.
  • the target object since the target object may not be photographed at the time of photographing, when the object contour template is matched with the target object information, the local region of the object contour template may be matched with the determined target object information to determine that the target object is in the image. The area occupied by.
  • the step S221 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a template matching unit 6201 executed by the processor.
  • step S223 difference information between the object contour in the object contour template and the contour of the target object in the image is determined according to the matching result.
  • the object contour template that characterizes the common features of the object may not be the same size as the object size in the image to be processed, and the position, posture angle, and the like of the object may deviate from the position, posture angle, and the like in the object contour template.
  • the object contour template may be first scaled, translated, and/or rotated, and then matched with the determined object's position, size, or key point to obtain the object contour and the image to be processed in the object contour template. Information about the difference between the contours of the object.
  • the difference information may include, but is not limited to, scaling information and/or offset information between the object contour in the object contour template and the contour of the target object in the image, and the like, and may also include, for example, an object contour template. Angle information between the contour of the object and the contour of the target object in the image, and the like.
  • the step S223 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a difference determining unit 6202 executed by the processor.
  • step S225 the object contour in the object contour template is adjusted according to the difference information.
  • the object contour in the object contour template may be scaled, translated, rotated, etc. according to the difference information including the aforementioned scaling information, offset information, etc., to match the area in which the target object is located in the image.
  • the step S225 may be performed by the processor invoking a corresponding instruction stored in the memory, or may be performed by the contour adjustment unit 6203 executed by the processor.
  • step S227 the adjusted object contour is mapped into the image to be processed, and the foreground region including the target object and the background region including at least part of the non-foreground region are obtained in the image.
  • the portion of the image to be processed that falls within the adjusted character contour can be determined to include the foreground region of the target object, which is the region occupied by the target object. Further, an image area including the foreground area or an image area including a part of the non-foreground area is determined as the background area of the image.
  • step S227 may be performed by the processor invoking a corresponding instruction stored in the memory, or may be performed by the front background determining unit 6204 being executed by the processor.
  • a transition area between the foreground area and the background area is determined.
  • an image area in the background area that is smaller than a predetermined extended distance from the outer edge of the area where the target object is located may be determined as the transition area. That is to say, the outer edge of the contour of the target object is extended outward by a certain distance, and the extended area is used as the transition area.
  • step S229 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a transition region determination module 640 that is executed by the processor.
  • step S220 or S221-229 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a front background determination module 620 that is executed by the processor.
  • step S230 the determined foreground area and/or the background area are subjected to blurring processing, and progressive blurring processing or spot processing is performed on the determined transition area.
  • the blurring process performed on the determined foreground area and/or the background area is similar to the processing of step S130, and will not be described herein.
  • Progressive blurring or spot processing can be performed on the transition area to make the effect of the blurring process more natural.
  • the step S230 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a transition blur processing module 650 executed by the processor.
  • the still image or the video frame image to be processed is detected in various manners, and the target object information in the still image or the video frame image is determined, according to the determined target object information and the object contour template.
  • the face key point is used as the face information. It should be noted that the face key point is only a feasible implementation manner of the embodiment of the present application, and the face information may further include face location information, face size information, and face angle information. Any one or more of the others.
  • the image processing method of this embodiment includes:
  • step S310 face information is detected from the image to be processed.
  • the face key point is detected from the image to be processed by the pre-trained key point positioning model, and the detected face key point is used as the face information.
  • An exemplary method of training a keypoint location model will be described later.
  • body type between individuals as individuals, they have commonality from the outline of the overall figure, for example, the head is elliptical and the torso is roughly triangular.
  • 4 is a schematic diagram of an exemplary character outline template including a human body and a face contour template including a human face in the embodiment of the present application.
  • the person being photographed may be in a plurality of different angles and distances during the shooting of the person, it is also possible to preset a plurality of character contour templates such as a face, a half body, a portrait, a side body, etc., for matching from different shooting distances. Or the image to be processed captured by the shooting angle. Therefore, at this step S310, face angle information can also be detected from the image to be processed.
  • the step S310 may be performed by the processor invoking a corresponding instruction stored in the memory, or may be performed by the fourth object information determining unit 6105 being executed by the processor.
  • step S320 a human body contour template corresponding to the face angle information is determined from among predetermined body contour templates.
  • the step S320 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a front background determination module 620 that is executed by the processor.
  • step S330 a foreground area and a background area in the image are determined according to the face information and a predetermined human body contour template.
  • step S330 is similar to the processing of the foregoing step S120 or S221 to S229, and details are not described herein.
  • the step S330 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a front background determination module 620 that is executed by the processor.
  • step S340 the foreground area and/or the background area are subjected to blurring processing.
  • This step is similar to the processing of step S130 and will not be described here.
  • the step S340 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a blurring processing module 630 executed by the processor.
  • the face information is obtained by detecting the image to be processed, and the foreground area and the background area in the image to be processed are acquired according to the detected face information and the character outline template, and then The background area and/or the foreground area are blurred, so that when the image related to the person is processed, the foreground area and the background area that need to be processed can be automatically and accurately determined by the face information detected from the image.
  • an exemplary method of the training keypoint location model includes:
  • a first sample set is acquired, the first sample set including a plurality of unlabeled sample images.
  • an image that has been input into the model and has been marked with key position information is generally referred to as an annotated sample image.
  • the key position information refers to the coordinate information of the key point in the image coordinate system.
  • the sample image may be marked in advance by manual labeling or the like.
  • the key points of the face are mainly distributed in the facial organs and facial contours, such as key points of the eyes, key points of the nose, key points of the mouth, key points of the facial contour, and the like.
  • the face key position information is the coordinate information of the face key point in the face image coordinate system. For example, the upper left corner of a sample image containing a human face is recorded as the coordinate origin, and the horizontal direction is the positive direction of the X-axis, and the vertical direction is the positive direction of the Y-axis.
  • the face image coordinate system is established, and the i-th person is The sitting key of the face key point in the face image coordinate system is (x i , y i ), and the sample image obtained by the above method is the labeled sample image.
  • the sample image can be understood as an unlabeled sample image.
  • the first sample set in this step is an image set containing a plurality of the above unlabeled sample images.
  • the step S510 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a sample set acquisition module 660 executed by the processor.
  • step S520 based on the depth neural network, key point position annotation is performed on each of the unlabeled sample images in the first sample set to obtain a second sample set.
  • the deep neural network is used for key point positioning of an image.
  • the deep neural network may be a convolutional neural network, but is not limited thereto. Since the deep neural network is used to locate the key points of the image, the unlabeled sample images in the first sample set are input into the deep neural network, and the key point position annotation can be realized for each unlabeled sample image. . It should be noted that the key point position labeling is to mark the key point position information (ie, coordinate information) in the unlabeled sample image.
  • the key points may include, but are not limited to, any one or more of the following: a face key point, a limb key point, a palm print key point, and a marker key point.
  • the face key point may include, but is not limited to, any one or more of the following: an eye key point, a nose key point, a mouth key point, an eyebrow key point, and a face contour key point.
  • an unlabeled sample image containing a human face is input into a deep neural network, and the output is an unlabeled sample image itself, and key position information of the unlabeled sample image, such as an eye key.
  • key position information of the unlabeled sample image such as an eye key.
  • the coordinate information of the point, the coordinate information of the nose key point, and the like constitute the second sample set in this step.
  • the step S520 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a keypoint location labeling module 670 that is executed by the processor.
  • step S530 parameters of the deep neural network are adjusted according to at least part of the sample image and the third sample set in the second sample set.
  • the third sample set includes a plurality of labeled sample images.
  • a partial sample image or a full sample image in the second set of samples may be used, and the third set of samples together adjust the parameters of the deep neural network.
  • the labeled sample image can be referred to the description and explanation in step S510 of the embodiment, and details are not described herein again.
  • step S530 may be performed by a processor invoking a corresponding instruction stored in the memory or by a network parameter adjustment module 680 executed by the processor.
  • the method for training a key point localization model uses two sample sets to adjust parameters of the deep neural network, one of which is a second sample set, the second sample set is derived from a deep neural network, including A first sample set of a plurality of unlabeled sample images is obtained by key point position annotation; and the other is a third sample set including a plurality of labeled sample images.
  • the embodiment of the present application can improve the training accuracy of the key point positioning model under the premise that the images input to the model are not all labeled images, in other words, the sample resource waste can be avoided, and the efficiency of the model training can be improved.
  • step S520 may include a process of performing image transformation processing on each unlabeled sample image in the first sample set to obtain a fourth sample set, where the image transformation
  • the processing may include, for example but not limited to, any one or more of the following: rotation, translation, scaling, noise addition, and occlusion, but is not limited thereto; based on the depth neural network, the fourth sample set and the Each sample image in the first sample set is subjected to key point position annotation to obtain the second sample set.
  • the set angle is usually in the range of (-20°, 20°), that is, a rotation transformation of a small amplitude.
  • the translation processing is also a small displacement. Pan.
  • the first sample set includes 10,000 unlabeled sample images, and each of the unlabeled sample images is subjected to image transformation processing (such as scaling, translation, etc.) to obtain 10 image-converted unlabeled sample images. At this time, 10,000 unlabeled sample images become 100,000 unlabeled sample images, and the 100,000 unlabeled sample images constitute the fourth sample set.
  • the unlabeled sample image is input to the deep neural network based on the same principle as described in the foregoing embodiment, and the fourth sample set and the first sample set are output.
  • Each sample image itself, as well as key point location information for each sample image.
  • step S530 may include: performing key transformation after image transformation processing based on the unlabeled sample image for each unlabeled sample image in the first sample set Position information, determining whether the key point position information of the unlabeled sample image is an optional sample; wherein the key point position information of the unlabeled sample image and the key point position information after performing image transformation processing are included in The second sample set; adjusting parameters of the deep neural network according to each of the selectable samples and the third sample set in the second sample set.
  • the key point position information of the unlabeled sample image and the key point position information after the image transform processing are included in the second sample set.
  • the key point position information after the image conversion processing is performed on the unlabeled sample image, and image correction processing is performed.
  • the image correction processing is the inverse transformation processing of the image transformation processing described above. For example, if an unlabeled sample image is shifted to the right by 5 mm, the key point position information after the image conversion processing is performed on the unlabeled sample image, It is necessary to translate 5 mm to the left to implement image correction processing.
  • the covariance matrix Cov1 is obtained by image-corrected key position information (ie, coordinate values of a series of points), and the covariance matrix Cov1 is expanded into a vector form by column or row by row, and normalized into a unit vector Cov1_v. .
  • the covariance matrix Cov2 is obtained for the key point position information of the unlabeled sample image, and Cov2 is expanded into a vector form column by column or row by row, and normalized into a unit vector Cov2_v.
  • D is compared with the set inner product threshold. If D is less than the inner product threshold, the key position information of the unlabeled sample image is an optional sample. Conversely, if D is greater than or equal to the inner product threshold, the key position information of the unlabeled sample image is not an optional sample.
  • Another way to select an optional sample is that the difference from the above-mentioned judging process is only in the last step. If D is less than the set threshold, image conversion processing is performed on the unlabeled sample image. After the key point position information, image correction processing is performed to obtain image corrected key point position information. Then, the result of the data distribution of the key position information corrected by the image (for example, the mean value of the coordinate values of a series of points) is used, and the key point position is marked on the unlabeled sample image, and the key position information of the label is used as Samples are selected, including in the second sample set.
  • the parameters of the deep neural network may be adjusted according to the commonly used training methods of the deep neural network, and the parameters of the deep neural network are adjusted according to the selected samples and the third sample set in the second sample set, and details are not described herein.
  • any of the methods provided by the foregoing embodiments of the present application may be performed by any suitable device having data processing capabilities, including but not limited to: a terminal device, a server, and the like.
  • any of the methods provided by the foregoing embodiments of the present application may be executed by a processor, such as the processor, by executing a corresponding instruction stored in a memory to perform any of the methods mentioned in the foregoing embodiments of the present application. This will not be repeated below.
  • FIG. 6 is a logic block diagram of an image processing apparatus according to an embodiment of the present application.
  • the image processing apparatus of Embodiment 5 includes an object information determining module 610, a front background determining module 620, and a blurring processing module 630. among them:
  • the object information determining module 610 is configured to determine target object information from the image to be processed.
  • the front background determining module 620 is configured to determine a foreground area and a background area in the image according to the target object information determined by the object information determining module 610 and the predetermined object contour template.
  • the blurring processing module 630 is configured to perform a blurring process on the foreground area and/or the background area determined by the front background determining module 620.
  • the image processing apparatus of the present embodiment can be used to implement the corresponding image processing method in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.
  • FIG. 7 is a logic block diagram showing an image processing apparatus according to another embodiment of the present application.
  • the image processing apparatus of Embodiment 6 includes an object information determining module 610, a front background determining module 620, and a blurring processing module 630.
  • the front background determining module 620 includes a template matching unit 6201, a difference determining unit 6202, a contour adjusting unit 6203, and a front background determining unit 6204. among them:
  • the template matching unit 6201 is configured to match at least a partial area in the foregoing object contour template with the determined target object information.
  • the difference determining unit 6202 is configured to determine difference information between the object contour in the object contour template and the contour of the target object in the image according to the matching result of the template matching unit 6201.
  • the contour adjustment unit 6203 is configured to adjust an object contour in the object contour template according to the difference information determined by the difference determining unit 6202.
  • the front background determining unit 6204 is configured to map an object contour adjusted by the contour adjusting unit 6203 into the image, obtain a foreground region including the target object in the image, and a background region including at least a portion not the foreground region.
  • the difference information includes: scaling information, offset information, and/or angle information between an object contour in the object contour template and a contour of the target object in the image.
  • the image may include, but is not limited to, a still image or a video frame image.
  • the image is a video frame image.
  • the object information determining module 610 includes: a first object information determining unit 6101, configured to determine the target object information from the to-be-processed video frame image according to target object information determined from a video frame image before the video frame image to be processed;
  • the second object information determining unit 6102 is configured to perform target-by-video frame image detection by the video stream to be processed, and determine target object information in each of the video frame images.
  • the image processing apparatus of this embodiment further includes: a transition area determining module 640, configured to determine a transition area between the foreground area and the background area; and a transition blur processing module 650, configured to use the transition area The transition area determined by the determination module 640 is blushed.
  • the transition blur processing module 650 is optionally configured to perform progressive blurring processing or spot processing on the transition region.
  • the object information determining module 610 includes: a selection information acquiring unit 6103 for acquiring object selection information; and a third object information determining unit 6104 for acquiring according to the selection information acquiring unit 6103.
  • the object selection information determines the target object information from the image to be processed.
  • the object information determining module 610 includes: a fourth object information determining unit 6105, configured to detect a target object from the image to be processed, and obtain the detected target object information.
  • the fourth object information determining unit 6105 is configured to detect the target object from the image to be processed through a pre-trained depth neural network, and obtain the detected target object information.
  • the target object information may include, but is not limited to, any one or more of the following: face information, license plate information, house number information, address information, identity ID information, and trademark information.
  • the face information may include, but is not limited to, any one or more of the following: information of a face key point, face position information, face size information, and face angle information.
  • the object contour template may include, but is not limited to, any one or more of the following: a face contour template, a human body contour template, a license plate contour template, a house card contour template, and a predetermined frame contour template.
  • the predetermined object contour template may include: a plurality of human body contour templates respectively corresponding to different human face angles; correspondingly, the front background determining module 620 is further configured to use the target object information according to the target object information and the predetermined object contour Before the template determines the foreground area and the background area in the image, a human body contour template corresponding to the face angle information in the face information is determined from among the predetermined object outline templates.
  • the image processing apparatus of the present embodiment is used to implement the corresponding image processing method in the foregoing method embodiments of the present application, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.
  • FIG. 8 is a logic block diagram showing an image processing apparatus according to still another embodiment of the present application.
  • the image processing apparatus of this embodiment includes an object information determining module 610, a front background determining module 620, and a blurring processing module 630.
  • the image processing apparatus further includes a transition region determining module 640 and a transition blur processing module 650.
  • the image processing apparatus further includes a sample set acquisition module 660, a key point location labeling module 670, and a network parameter adjustment module 680. among them:
  • the sample set obtaining module 660 is configured to acquire a first sample set, where the first sample set includes a plurality of unlabeled sample images.
  • a key point location labeling module 670 configured to perform key point position labeling on each of the unlabeled sample images in the first sample set based on a depth neural network, to obtain a second sample set, where the depth neural network is used For key point positioning of the image.
  • the network parameter adjustment module 680 is configured to adjust parameters of the deep neural network according to at least a partial sample image and a third sample set in the second sample set, where the third sample set includes a plurality of labeled sample images .
  • the key point location labeling module 670 may include: an image transformation processing unit 6701, configured to perform image transformation processing on each unlabeled sample image in the first sample set to obtain a fourth sample set, where
  • the image transformation processing may include, but is not limited to, any one or more of the following: rotation, translation, scaling, noise addition, and occlusion; a key point location labeling unit 6702, for the fourth based on the depth neural network
  • the sample set and each unlabeled sample image in the first sample set perform key point position labeling to obtain the second sample set.
  • the network parameter adjustment module 680 includes: an optional sample determining unit 6801, configured to perform image transformation processing on the basis of the unlabeled sample image for each unlabeled sample image in the first sample set. Point position information, determining whether the key point position information of the unlabeled sample image is an optional sample, wherein the key point position information of the unlabeled sample image and the key point position information after the image transformation processing are included
  • the network parameter adjustment unit 6802 is configured to adjust parameters of the deep neural network according to each of the selectable samples and the third sample set in the second sample set.
  • the face key point may include, but is not limited to, any one or more of the following: an eye key point, a nose key point, a mouth key point, an eyebrow key point, and a face contour key point.
  • the image processing apparatus of the present embodiment is used to implement the corresponding image processing method in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, and details are not described herein again.
  • the embodiment of the present application further provides an electronic device, including: a processor and an image processing apparatus according to any of the foregoing embodiments of the present application.
  • a processor runs the image processing apparatus, the image of any of the foregoing embodiments of the present application
  • the units in the processing unit are operated.
  • the embodiment of the present application further provides another electronic device, including: a processor and a memory, where the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform image processing in any of the foregoing embodiments of the present application.
  • a processor and a memory where the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform image processing in any of the foregoing embodiments of the present application. The corresponding operation of the method.
  • FIG. 9 is a schematic structural view showing an electronic device according to an embodiment of the present application.
  • a block diagram of an electronic device 900 suitable for use in implementing a terminal device or server of an embodiment of the present application is shown. As shown in FIG.
  • electronic device 900 includes one or more processors, communication elements, etc., one or more processors such as one or more central processing units (CPUs) 901, and/or one or more image processing
  • processors such as one or more central processing units (CPUs) 901, and/or one or more image processing
  • the GPU 913 or the like, the processor may perform various appropriate operations according to executable instructions stored in the read only memory (ROM) 902 or executable instructions loaded from the storage portion 908 into the random access memory (RAM) 903.
  • the communication component includes a communication component 912 and a communication interface 909.
  • the communication component 912 can include, but is not limited to, a network card, which can include, but is not limited to, an IB (Infiniband) network card, the communication interface 909 includes a communication interface of a network interface card such as a LAN card, a modem, etc., and the communication interface 909 is via a network such as the Internet. Perform communication processing.
  • a network card which can include, but is not limited to, an IB (Infiniband) network card
  • the communication interface 909 includes a communication interface of a network interface card such as a LAN card, a modem, etc.
  • the communication interface 909 is via a network such as the Internet. Perform communication processing.
  • the processor can communicate with the read only memory 902 and/or the random access memory 930 to execute executable instructions, connect to the communication component 912 via the bus 904, and communicate with other target devices via the communication component 912, thereby completing the embodiments of the present application.
  • Corresponding operations of any one of the methods for example, determining target object information from the image to be processed; determining foreground and background regions in the image according to the target object information and the predetermined object contour template; / or the background area is blurred.
  • RAM 903 various programs and data required for the operation of the device can be stored.
  • the CPU 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904.
  • ROM 902 is an optional module.
  • the RAM 903 stores executable instructions or writes executable instructions to the ROM 902 at runtime, the executable instructions causing the processor 901 to perform operations corresponding to the above-described communication methods.
  • An input/output (I/O) interface 905 is also coupled to bus 904.
  • the communication component 912 can be integrated or can be configured to have multiple sub-modules (eg, multiple IB network cards) and be on a bus link.
  • the following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a storage portion 908 including a hard disk or the like. And a communication interface 909 including a network interface card such as a LAN card, a modem, or the like.
  • Driver 910 is also connected to I/O interface 905 as needed.
  • a removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like is mounted on the drive 910 as needed so that a computer program read therefrom is installed into the storage portion 908 as needed.
  • FIG. 9 is only an optional implementation manner.
  • the number and types of components in FIG. 9 may be selected, deleted, added, or replaced according to actual needs.
  • implementations such as separate settings or integrated settings may also be adopted.
  • the GPU and the CPU may be detachably set or the GPU may be integrated on the CPU
  • the communication component 912 may be separately configured, or may be integrated in the CPU or On the GPU, and so on.
  • embodiments of the present application include a computer program product comprising a computer program tangibly embodied on a machine readable medium, the computer program comprising program code for executing the method illustrated in the flowchart, the program code comprising the corresponding execution
  • the instruction corresponding to the method step provided by the embodiment of the present application determines, for example, the target object information from the image to be processed; and determines the foreground area and the background area in the image according to the target object information and the predetermined object contour template; The foreground area and/or the background area are subjected to blurring processing.
  • the computer program can be downloaded and installed from the network via a communication component, and/or installed from the removable media 911.
  • the computer program is executed by the central processing unit (CPU) 901, the above-described functions defined in the method of the embodiment of the present application are executed.
  • the electronic device 900 of the eighth embodiment detects the image to be processed to determine target object information, and acquires a foreground area and a background area in the image to be processed according to the determined target object information and the object contour template, and then the background area or the foreground area.
  • the blurring process is performed so that the foreground area or the background area in which the blurring process needs to be performed can be automatically determined by the target object information detected from the image without the user manually marking the area where the blurring processing is to be performed or manually performing the virtual (fuzzy) operation to improve the convenience and accuracy of operation.
  • the embodiment of the present application further provides a computer program, including computer readable code, when the computer readable code is run on the device, the processor in the device is configured to implement any of the foregoing embodiments of the present application. Instructions for each step in the image processing method.
  • the embodiment of the present application further provides a computer readable storage medium for storing computer readable instructions, which are executed to implement the operations of the steps in the image processing method of any of the foregoing embodiments of the present application.
  • the methods, apparatus, and apparatus of the present application may be implemented in a number of ways.
  • the method, apparatus, and apparatus of the embodiments of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware.
  • the above-described sequence of steps for the method is for illustrative purposes only, and the steps of the method of the embodiments of the present application are not limited to the order of the above optional description unless otherwise specified.
  • the present application may also be embodied as a program recorded in a recording medium, the programs including machine readable instructions for implementing a method in accordance with embodiments of the present application.
  • the present application also covers a recording medium storing a program for executing the method according to the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Biomedical Technology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geometry (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé et un appareil de traitement d'image, ainsi qu'un dispositif électronique. Le procédé de traitement d'image comprend les étapes consistant à : déterminer des informations d'objet cible à partir d'une image à traiter ; déterminer une zone de premier plan et une zone d'arrière-plan dans l'image selon les informations d'objet cible et un modèle de profil d'objet prédéfini ; et effectuer un traitement de flou sur la zone de premier plan et/ou sur la zone d'arrière-plan. La solution technique fournie dans les modes de réalisation de la présente invention améliore la commodité et la précision d'un traitement de flou d'image.
PCT/CN2018/073882 2017-01-24 2018-01-23 Procédé et appareil de traitement d'image, et dispositif électronique Ceased WO2018137623A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710060426.X 2017-01-24
CN201710060426.XA CN108230252B (zh) 2017-01-24 2017-01-24 图像处理方法、装置以及电子设备

Publications (1)

Publication Number Publication Date
WO2018137623A1 true WO2018137623A1 (fr) 2018-08-02

Family

ID=62657248

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/073882 Ceased WO2018137623A1 (fr) 2017-01-24 2018-01-23 Procédé et appareil de traitement d'image, et dispositif électronique

Country Status (2)

Country Link
CN (1) CN108230252B (fr)
WO (1) WO2018137623A1 (fr)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109242763A (zh) * 2018-08-27 2019-01-18 Oppo广东移动通信有限公司 图片处理方法、图片处理装置、终端设备
CN110147708A (zh) * 2018-10-30 2019-08-20 腾讯科技（深圳）有限公司 一种图像数据处理方法和相关装置
CN110827371A (zh) * 2019-11-05 2020-02-21 厦门美图之家科技有限公司 证件照生成方法、装置、电子设备及存储介质
CN110889314A (zh) * 2018-09-10 2020-03-17 北京市商汤科技开发有限公司 图像处理方法、装置、电子设备、服务器及系统
CN110910304A (zh) * 2019-11-08 2020-03-24 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及介质
CN111274852A (zh) * 2018-12-05 2020-06-12 北京猎户星空科技有限公司 目标对象关键点检测方法和装置
CN111325217A (zh) * 2018-12-14 2020-06-23 北京海益同展信息科技有限公司 数据处理方法、装置、系统和介质
CN111507896A (zh) * 2020-04-27 2020-08-07 北京字节跳动网络技术有限公司 图像液化处理方法、装置、设备和存储介质
CN111626084A (zh) * 2019-02-28 2020-09-04 北京京东尚科信息技术有限公司 信息标注方法、装置、设备、系统、介质及电子设备
CN111652796A (zh) * 2020-05-13 2020-09-11 上海连尚网络科技有限公司 图像处理方法、电子设备及计算机可读存储介质
CN111860304A (zh) * 2020-07-17 2020-10-30 北京百度网讯科技有限公司 一种图像标注方法、电子装置、设备及存储介质
CN112016508A (zh) * 2020-09-07 2020-12-01 杭州海康威视数字技术股份有限公司 人脸识别方法、装置、系统、计算设备及存储介质
CN112132913A (zh) * 2019-06-25 2020-12-25 北京字节跳动网络技术有限公司 图像处理方法、装置、介质和电子设备
CN112348035A (zh) * 2020-11-11 2021-02-09 东软睿驰汽车技术（沈阳）有限公司 车辆关键点检测方法、装置及电子设备
CN112364898A (zh) * 2020-10-27 2021-02-12 星火科技技术(深圳)有限责任公司 图像识别自动标注方法、装置、设备及存储介质
CN112487847A (zh) * 2019-09-11 2021-03-12 深圳云天励飞技术有限公司 图像检测方法及相关设备
CN112633305A (zh) * 2019-09-24 2021-04-09 深圳云天励飞技术有限公司 关键点标注方法及相关设备
CN112800878A (zh) * 2021-01-14 2021-05-14 北京迈格威科技有限公司 目标检测方法、装置、电子设备及可读存储介质
CN113012054A (zh) * 2019-12-20 2021-06-22 舜宇光学（浙江）研究院有限公司 基于抠图的样本增强方法和训练方法及其系统和电子设备
CN113129207A (zh) * 2019-12-30 2021-07-16 武汉Tcl集团工业研究院有限公司 一种图片的背景虚化方法及装置、计算机设备、存储介质
CN113379623A (zh) * 2021-05-31 2021-09-10 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113763511A (zh) * 2020-09-28 2021-12-07 北京沃东天骏信息技术有限公司 图像处理方法及装置
CN113869364A (zh) * 2021-08-26 2021-12-31 北京旷视科技有限公司 图像处理方法、装置、电子设备及介质
CN114119328A (zh) * 2021-10-25 2022-03-01 珠海格力电器股份有限公司 一种对象屏蔽方法、电子设备及存储介质
CN114143561A (zh) * 2021-11-12 2022-03-04 北京中联合超高清协同技术中心有限公司 一种超高清视频多视角漫游播放方法
CN114187326A (zh) * 2021-12-14 2022-03-15 维沃移动通信有限公司 图像处理方法、装置、电子设备和可读存储介质
CN114332201A (zh) * 2021-12-01 2022-04-12 北京三快在线科技有限公司 一种模型训练、目标检测的方法以及装置
CN115171034A (zh) * 2022-06-06 2022-10-11 阿里巴巴（中国）有限公司 道路异物检测方法、场景中异物检测方法及装置
CN115311148A (zh) * 2021-05-06 2022-11-08 Oppo广东移动通信有限公司 网格映射计算方法及装置、计算机可读介质和电子设备
CN111539443B (zh) * 2020-01-22 2024-02-09 北京小米松果电子有限公司 一种图像识别模型训练方法及装置、存储介质

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109147024A (zh) 2018-08-16 2019-01-04 Oppo广东移动通信有限公司 基于三维模型的表情更换方法和装置
CN113129312B (zh) 2018-10-15 2022-10-28 华为技术有限公司 一种图像处理方法、装置与设备
CN113298845A (zh) * 2018-10-15 2021-08-24 华为技术有限公司 一种图像处理方法、装置与设备
CN110033463B (zh) 2019-04-12 2021-06-04 腾讯科技（深圳）有限公司 一种前景数据生成及其应用方法、相关装置和系统
CN110310327B (zh) * 2019-06-28 2022-10-25 联想(北京)有限公司 图像处理方法及装置、计算机系统以及可读存储介质
CN112215762A (zh) * 2019-07-12 2021-01-12 阿里巴巴集团控股有限公司 视频图像处理方法、装置及电子设备
CN112204945A (zh) * 2019-08-14 2021-01-08 深圳市大疆创新科技有限公司 图像处理方法、设备、拍摄装置、可移动平台及存储介质
CN110728632B (zh) * 2019-09-04 2022-07-12 北京奇艺世纪科技有限公司 图像模糊处理方法、装置、计算机设备和存储介质
CN112785487B (zh) * 2019-11-06 2023-08-04 RealMe重庆移动通信有限公司 图像处理方法及装置、存储介质和电子设备
CN110991298B (zh) * 2019-11-26 2023-07-14 腾讯科技（深圳）有限公司 图像的处理方法和装置、存储介质及电子装置
CN113129241B (zh) * 2019-12-31 2023-02-07 RealMe重庆移动通信有限公司 图像处理方法及装置、计算机可读介质、电子设备
CN111639653B (zh) * 2020-05-08 2023-10-10 浙江大华技术股份有限公司 一种误检图像确定方法、装置、设备和介质
CN112669433B (zh) * 2020-12-22 2024-09-27 北京像素软件科技股份有限公司 轮廓渲染方法、装置、电子设备和计算机可读存储介质
CN113191348B (zh) * 2021-05-31 2023-02-03 山东新一代信息产业技术研究院有限公司 一种基于模板的文本结构化提取方法及工具
CN113784207A (zh) * 2021-07-30 2021-12-10 北京达佳互联信息技术有限公司 视频画面的显示方法和装置、电子设备及存储介质
CN114063858B (zh) * 2021-11-26 2023-03-17 北京百度网讯科技有限公司 图像处理方法、装置、电子设备及存储介质
CN114359077A (zh) * 2021-12-22 2022-04-15 北京旷视科技有限公司 图像处理方法、计算机程序产品和电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5870103A (en) * 1996-09-25 1999-02-09 Eastman Kodak Company Method for creating realistic-looking composite images
CN104751407A (zh) * 2015-03-11 2015-07-01 百度在线网络技术（北京）有限公司 一种用于对图像进行虚化的方法和装置
CN105979165A (zh) * 2016-06-02 2016-09-28 广东欧珀移动通信有限公司 虚化照片生成方法、装置和移动终端

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103119625B (zh) * 2011-09-16 2015-06-03 华为技术有限公司 一种视频人物分割的方法及装置
CN104408743A (zh) * 2014-11-05 2015-03-11 百度在线网络技术（北京）有限公司 图像分割方法和装置
CN104751406B (zh) * 2015-03-11 2018-05-01 百度在线网络技术（北京）有限公司 一种用于对图像进行虚化的方法和装置
CN105447458B (zh) * 2015-11-17 2018-02-27 深圳市商汤科技有限公司 一种大规模人群视频分析系统和方法
CN105701513B (zh) * 2016-01-14 2019-06-07 深圳市未来媒体技术研究院 快速提取掌纹感兴趣区域的方法
CN105678714B (zh) * 2016-02-05 2020-12-29 网易传媒科技（北京）有限公司 一种图像处理方法和装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5870103A (en) * 1996-09-25 1999-02-09 Eastman Kodak Company Method for creating realistic-looking composite images
CN104751407A (zh) * 2015-03-11 2015-07-01 百度在线网络技术（北京）有限公司 一种用于对图像进行虚化的方法和装置
CN105979165A (zh) * 2016-06-02 2016-09-28 广东欧珀移动通信有限公司 虚化照片生成方法、装置和移动终端

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109242763A (zh) * 2018-08-27 2019-01-18 Oppo广东移动通信有限公司 图片处理方法、图片处理装置、终端设备
CN109242763B (zh) * 2018-08-27 2023-09-01 Oppo广东移动通信有限公司 图片处理方法、图片处理装置、终端设备
CN110889314A (zh) * 2018-09-10 2020-03-17 北京市商汤科技开发有限公司 图像处理方法、装置、电子设备、服务器及系统
CN110147708A (zh) * 2018-10-30 2019-08-20 腾讯科技（深圳）有限公司 一种图像数据处理方法和相关装置
CN110147708B (zh) * 2018-10-30 2023-03-31 腾讯科技（深圳）有限公司 一种图像数据处理方法和相关装置
CN111274852B (zh) * 2018-12-05 2023-10-31 北京猎户星空科技有限公司 目标对象关键点检测方法和装置
CN111274852A (zh) * 2018-12-05 2020-06-12 北京猎户星空科技有限公司 目标对象关键点检测方法和装置
CN111325217A (zh) * 2018-12-14 2020-06-23 北京海益同展信息科技有限公司 数据处理方法、装置、系统和介质
CN111325217B (zh) * 2018-12-14 2024-02-06 京东科技信息技术有限公司 数据处理方法、装置、系统和介质
CN111626084A (zh) * 2019-02-28 2020-09-04 北京京东尚科信息技术有限公司 信息标注方法、装置、设备、系统、介质及电子设备
CN112132913A (zh) * 2019-06-25 2020-12-25 北京字节跳动网络技术有限公司 图像处理方法、装置、介质和电子设备
CN112487847A (zh) * 2019-09-11 2021-03-12 深圳云天励飞技术有限公司 图像检测方法及相关设备
CN112633305A (zh) * 2019-09-24 2021-04-09 深圳云天励飞技术有限公司 关键点标注方法及相关设备
CN110827371B (zh) * 2019-11-05 2023-04-28 厦门美图之家科技有限公司 证件照生成方法、装置、电子设备及存储介质
CN110827371A (zh) * 2019-11-05 2020-02-21 厦门美图之家科技有限公司 证件照生成方法、装置、电子设备及存储介质
CN110910304B (zh) * 2019-11-08 2023-12-22 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及介质
CN110910304A (zh) * 2019-11-08 2020-03-24 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及介质
CN113012054B (zh) * 2019-12-20 2023-12-05 舜宇光学（浙江）研究院有限公司 基于抠图的样本增强方法和训练方法及其系统和电子设备
CN113012054A (zh) * 2019-12-20 2021-06-22 舜宇光学（浙江）研究院有限公司 基于抠图的样本增强方法和训练方法及其系统和电子设备
CN113129207A (zh) * 2019-12-30 2021-07-16 武汉Tcl集团工业研究院有限公司 一种图片的背景虚化方法及装置、计算机设备、存储介质
CN111539443B (zh) * 2020-01-22 2024-02-09 北京小米松果电子有限公司 一种图像识别模型训练方法及装置、存储介质
CN111507896A (zh) * 2020-04-27 2020-08-07 北京字节跳动网络技术有限公司 图像液化处理方法、装置、设备和存储介质
CN111507896B (zh) * 2020-04-27 2023-09-05 抖音视界有限公司 图像液化处理方法、装置、设备和存储介质
CN111652796A (zh) * 2020-05-13 2020-09-11 上海连尚网络科技有限公司 图像处理方法、电子设备及计算机可读存储介质
CN111860304A (zh) * 2020-07-17 2020-10-30 北京百度网讯科技有限公司 一种图像标注方法、电子装置、设备及存储介质
CN111860304B (zh) * 2020-07-17 2024-04-30 北京百度网讯科技有限公司 一种图像标注方法、电子装置、设备及存储介质
CN112016508A (zh) * 2020-09-07 2020-12-01 杭州海康威视数字技术股份有限公司 人脸识别方法、装置、系统、计算设备及存储介质
CN112016508B (zh) * 2020-09-07 2023-08-29 杭州海康威视数字技术股份有限公司 人脸识别方法、装置、系统、计算设备及存储介质
CN113763511A (zh) * 2020-09-28 2021-12-07 北京沃东天骏信息技术有限公司 图像处理方法及装置
CN112364898B (zh) * 2020-10-27 2024-01-19 星火科技技术(深圳)有限责任公司 图像识别自动标注方法、装置、设备及存储介质
CN112364898A (zh) * 2020-10-27 2021-02-12 星火科技技术(深圳)有限责任公司 图像识别自动标注方法、装置、设备及存储介质
CN112348035B (zh) * 2020-11-11 2024-05-24 东软睿驰汽车技术（沈阳）有限公司 车辆关键点检测方法、装置及电子设备
CN112348035A (zh) * 2020-11-11 2021-02-09 东软睿驰汽车技术（沈阳）有限公司 车辆关键点检测方法、装置及电子设备
CN112800878A (zh) * 2021-01-14 2021-05-14 北京迈格威科技有限公司 目标检测方法、装置、电子设备及可读存储介质
CN115311148A (zh) * 2021-05-06 2022-11-08 Oppo广东移动通信有限公司 网格映射计算方法及装置、计算机可读介质和电子设备
CN113379623A (zh) * 2021-05-31 2021-09-10 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113379623B (zh) * 2021-05-31 2023-12-19 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113869364A (zh) * 2021-08-26 2021-12-31 北京旷视科技有限公司 图像处理方法、装置、电子设备及介质
CN114119328A (zh) * 2021-10-25 2022-03-01 珠海格力电器股份有限公司 一种对象屏蔽方法、电子设备及存储介质
CN114143561B (zh) * 2021-11-12 2023-11-07 北京中联合超高清协同技术中心有限公司 一种超高清视频多视角漫游播放方法
CN114143561A (zh) * 2021-11-12 2022-03-04 北京中联合超高清协同技术中心有限公司 一种超高清视频多视角漫游播放方法
CN114332201A (zh) * 2021-12-01 2022-04-12 北京三快在线科技有限公司 一种模型训练、目标检测的方法以及装置
CN114187326A (zh) * 2021-12-14 2022-03-15 维沃移动通信有限公司 图像处理方法、装置、电子设备和可读存储介质
CN115171034A (zh) * 2022-06-06 2022-10-11 阿里巴巴（中国）有限公司 道路异物检测方法、场景中异物检测方法及装置

Also Published As

Publication number Publication date
CN108230252A (zh) 2018-06-29
CN108230252B (zh) 2022-02-01

Similar Documents

Publication Publication Date Title
WO2018137623A1 (fr) Procédé et appareil de traitement d'image, et dispositif électronique
US11107232B2 (en) Method and apparatus for determining object posture in image, device, and storage medium
US10872420B2 (en) Electronic device and method for automatic human segmentation in image
US20210056360A1 (en) System and method using machine learning for iris tracking, measurement, and simulation
US9262671B2 (en) Systems, methods, and software for detecting an object in an image
US8391645B2 (en) Detecting orientation of digital images using face detection information
JP6368709B2 (ja) ３次元身体データを生成する方法
CN108229330A (zh) 人脸融合识别方法及装置、电子设备和存储介质
WO2018219180A1 (fr) Procédé et appareil pour déterminer une qualité d'image faciale, dispositif électronique et support d'informations d'ordinateur
WO2018121777A1 (fr) Procédé et appareil de détection de visage, et dispositif électronique
CN109086691A (zh) 一种三维脸部活体检测方法、脸部认证识别方法及装置
CN110570435B (zh) 用于对车辆损伤图像进行损伤分割的方法及装置
Edmunds et al. Motion-based countermeasure against photo and video spoofing attacks in face recognition
MX2013002904A (es) Aparato de proceso de imagenes de personas y metodo para procesar imagenes de personas.
CN111008935B (zh) 一种人脸图像增强方法、装置、系统及存储介质
CN112329663B (zh) 一种基于人脸图像序列的微表情时刻检测方法及装置
CN108961158A (zh) 一种图像合成方法及装置
WO2019061659A1 (fr) Procédé et dispositif permettant de supprimer des lunettes d'une image de visage, et support d'informations
JP5656768B2 (ja) 画像特徴量抽出装置およびそのプログラム
US9349038B2 (en) Method and apparatus for estimating position of head, computer readable storage medium thereof
JP2023512359A (ja) 関連対象検出方法、及び装置
CN106030658A (zh) 用于确定视频的方位的方法及装置
Halperin et al. Clear Skies Ahead: Towards Real‐Time Automatic Sky Replacement in Video
US20060010582A1 (en) Chin detecting method, chin detecting system and chin detecting program for a chin of a human face
CN113971671B (zh) 实例分割方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18743952

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21/11/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 18743952

Country of ref document: EP

Kind code of ref document: A1