[go: up one dir, main page]

CN109002801B - Face shielding detection method and system based on video monitoring - Google Patents

Face shielding detection method and system based on video monitoring Download PDF

Info

Publication number
CN109002801B
CN109002801B CN201810801599.7A CN201810801599A CN109002801B CN 109002801 B CN109002801 B CN 109002801B CN 201810801599 A CN201810801599 A CN 201810801599A CN 109002801 B CN109002801 B CN 109002801B
Authority
CN
China
Prior art keywords
image
area
moving object
face
vertical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810801599.7A
Other languages
Chinese (zh)
Other versions
CN109002801A (en
Inventor
顾广华
田季丰
崔冬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yanshan University
Original Assignee
Yanshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yanshan University filed Critical Yanshan University
Priority to CN201810801599.7A priority Critical patent/CN109002801B/en
Publication of CN109002801A publication Critical patent/CN109002801A/en
Application granted granted Critical
Publication of CN109002801B publication Critical patent/CN109002801B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/457—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F19/00—Complete banking systems; Coded card-freed arrangements adapted for dispensing or receiving monies or the like and posting such transactions to existing accounts, e.g. automatic teller machines
    • G07F19/20—Automatic teller machines [ATMs]
    • G07F19/207—Surveillance aspects at ATMs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Geometry (AREA)
  • General Engineering & Computer Science (AREA)
  • Psychiatry (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Social Psychology (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Image Analysis (AREA)
  • Alarm Systems (AREA)

Abstract

本发明公开了一种基于视频监控的人脸遮挡检测方法及系统，该方法包括获取视频监控设备采集的动态视频图像；采用三帧差法与混合高斯背景模型相结合的算法，检测所述动态视频图像的运动物体，并确定运动物体区域图像；提取所述运动物体区域图像中的各个连通区域并标记，确定最大连通区域；对所述最大连通区域进行垂直和水平投影处理，截取人脸区域图像；采用K最近邻分类算法和局部二值模式算法对所述人脸区域图像进行人脸遮挡检测。应用本发明提供的方法或者系统，能够耗时少、实时性高、精度高、误差低的实现人脸遮挡检测。

Figure 201810801599

The invention discloses a method and system for detecting face occlusion based on video surveillance. The method includes acquiring dynamic video images collected by video surveillance equipment; using an algorithm combining a three-frame difference method and a mixed Gaussian background model to detect the dynamic The moving object of the video image, and the image of the moving object area is determined; each connected area in the image of the moving object area is extracted and marked, and the maximum connected area is determined; the vertical and horizontal projection processing is performed on the maximum connected area, and the face area is intercepted Image; using K-nearest neighbor classification algorithm and local binary pattern algorithm to perform face occlusion detection on the face region image. By applying the method or system provided by the present invention, face occlusion detection can be realized with less time consumption, high real-time performance, high precision and low error.

Figure 201810801599

Description

Face shielding detection method and system based on video monitoring
Technical Field
The invention relates to the technical field of intelligent monitoring, in particular to a face occlusion detection method and system based on video monitoring.
Background
With the development of informatization, the application of the automatic teller machine (hereinafter referred to as an ATM machine) is more and more extensive, and due to the characteristics of unattended operation and electronic finance, the ATM machine brings convenience to people and simultaneously brings opportunity for criminals. According to actual cases, criminals in such cases often wear protective articles such as sunglasses, masks or hats to shield the face. Therefore, in order to prevent the occurrence of such cases in time, it is particularly important to detect and identify the face covered by the ornament.
There has been little research on detecting a human face having a mask and identifying a class of the mask. The correction statistical conversion (MCT) algorithm proposed by Froba et al. Dong and Soh detect a moving target by using a Gaussian mixture model, then segment a face region by moving skin color detection, and then provide facial features, and judge whether the face is blocked or not by a threshold value. In the positioned head image, YuanBaoHua judges whether the face is provided with the shielding ornaments or not by judging whether the five sense organs are missing or not, and most of the use of the face image is a simple image processing method. The lie proposes that on the basis of the detected possible face for the first time, the face is judged whether to be shielded by ornaments, and the lie utilizes a frame difference method and an Adaboost classifier to detect. Guo thought depression was detected using ellipse fitting and traditional skin color detection and five sense organs detection methods. However, in general, the existing research in China and abroad does not make the methods for detecting and judging the face with the shielding ornaments mature, and still has many defects. Summarizing, the method mainly comprises the following aspects: (1) by using a skin color detection method, for black and yellow people, the method cannot set a uniform threshold value to distinguish the face from the background, and ornaments such as a mask, a hat and the like can shield most of the face area, so that the skin is difficult to detect; (2) by using the method for detecting the five sense organs, the characteristics of the five sense organs can be shielded under the condition that the face is shielded by an ornament, and the false detection rate is relatively high in consideration of the fact that the posture of a customer changes greatly in the actual depositing and withdrawing process, such as the head is lowered, the side face is lowered and the like; (3) by using the template matching method, the difference of the human head outline is large, so that the detection rate cannot be ensured by using a simple template model, but the complex template model has large calculation amount, much time consumption and poor real-time property, and is difficult to be used for detecting a real video.
Disclosure of Invention
The invention aims to provide a face shielding detection method and system based on video monitoring, which can realize face shielding detection with less time consumption, high real-time performance, high precision and low error, particularly carry out real-time detection on a face with two shielding ornaments of sunglasses and scarves and judge the types of the two ornaments, and send out a warning when detecting the face with the sunglasses or the scarves, thereby actively preventing illegal criminal behaviors and being applied to monitoring videos of bank ATM machines.
In order to achieve the purpose, the invention provides the following scheme:
a face occlusion detection method based on video monitoring comprises the following steps:
acquiring a dynamic video image acquired by video monitoring equipment;
detecting a moving object of the dynamic video image by adopting an algorithm combining a three-frame difference method and a mixed Gaussian background model, and determining a region image of the moving object;
extracting and marking each connected region in the moving object region image, and determining the maximum connected region;
carrying out vertical and horizontal projection processing on the maximum communication area, and intercepting a face area image;
and carrying out face shielding detection on the face region image by adopting a K nearest neighbor classification algorithm and a local binary pattern algorithm.
Optionally, the face occlusion detection method further includes:
and when the face is detected to be shielded, the video monitoring equipment sends out an alarm prompt.
Optionally, the detecting a moving object of the dynamic video image and determining a region image of the moving object by using an algorithm combining a three-frame difference method and a gaussian mixture background model specifically includes:
judging whether a moving object enters the dynamic video image in real time by adopting an algorithm combining a three-frame difference method and a Gaussian mixture background model to obtain a first judgment result;
if the first judgment result shows that the moving object enters the dynamic video image, determining that the moving object exists in the dynamic video image, and extracting an image area corresponding to the moving object;
and if the first judgment result shows that the moving object is not entered into the dynamic video image, updating the background parameter of the Gaussian mixture background model.
Optionally, the extracting and marking each connected region in the moving object region image, and determining the maximum connected region specifically include:
carrying out mathematical morphology processing on the moving object region image;
extracting and marking each connected region of the processed moving object region image;
calculating the area of the connected region of each mark;
and determining the maximum connected region according to the area of the connected region of each mark.
Optionally, the performing vertical and horizontal projection processing on the maximum connected region to capture a face region image specifically includes:
acquiring the highest point vertical coordinate of the moving object profile in the maximum communication area by adopting a bwboundaries function;
performing vertical projection processing on the maximum communication area to obtain a vertical projection image;
determining two critical values of the vertical projection image, wherein the two critical values are a vertical first critical value and a vertical second critical value respectively; the vertical first critical value and the vertical second critical value are horizontal coordinates of left and right boundary points of the face area;
carrying out horizontal projection processing on the vertical projection image to obtain a horizontal projection image;
determining a horizontal critical value of the horizontal projection image; the horizontal critical value is a vertical coordinate of a critical point of a chin and a neck in the face area;
and intercepting a face region image from the moving object region image according to the vertical first critical value, the vertical second critical value, the horizontal critical value and the highest point vertical coordinate.
The invention also provides a face occlusion detection system based on video monitoring, which comprises:
the dynamic video image acquisition module is used for acquiring a dynamic video image acquired by the video monitoring equipment;
the moving object region image determining module is used for detecting a moving object of the dynamic video image by adopting an algorithm combining a three-frame difference method and a Gaussian mixture background model and determining a moving object region image;
the maximum connected region determining module is used for extracting and marking each connected region in the moving object region image and determining the maximum connected region;
the human face area image intercepting module is used for performing vertical and horizontal projection processing on the maximum communication area and intercepting a human face area image;
and the face shielding detection module is used for carrying out face shielding detection on the face region image by adopting a K nearest neighbor classification algorithm and a local binary pattern algorithm.
Optionally, the face occlusion detection system further includes:
and the alarm reminding module is used for sending out alarm reminding by the video monitoring equipment when the face is detected to be shielded.
Optionally, the moving object region image determining module specifically includes:
the first judgment result obtaining unit is used for judging whether the moving object enters the dynamic video image in real time by adopting an algorithm combining a three-frame difference method and a Gaussian mixture background model to obtain a first judgment result;
a moving object region image extraction unit, configured to determine that the moving object exists in the dynamic video image and extract an image region corresponding to the moving object if the first determination result indicates that the moving object enters the dynamic video image;
and the background parameter updating unit is used for updating the background parameter of the Gaussian mixture background model when the first judgment result shows that the moving object is not entered into the dynamic video image.
Optionally, the maximum connected region determining module specifically includes:
the mathematical morphology processing unit is used for carrying out mathematical morphology processing on the moving object region image;
a connected region extraction marking unit for extracting and marking each connected region of the processed moving object region image;
a connected region area calculation unit for calculating the area of the connected region of each mark;
and a maximum connected region determining unit configured to determine a maximum connected region according to an area of the connected region of each of the marks.
Optionally, the face region image intercepting module specifically includes:
the highest vertical coordinate acquisition unit is used for acquiring the highest vertical coordinate of the moving object profile in the maximum communication area by adopting a bwboundaries function;
a vertical projection image obtaining unit, configured to perform vertical projection processing on the maximum connected region to obtain a vertical projection image;
the vertical critical value determining unit is used for determining two critical values of the vertical projection image, wherein the two critical values are a vertical first critical value and a vertical second critical value respectively; the vertical first critical value and the vertical second critical value are horizontal coordinates of left and right boundary points of the face area;
a horizontal projection image obtaining unit for performing horizontal projection processing on the vertical projection image to obtain a horizontal projection image;
a horizontal critical value determining unit for determining a horizontal critical value of the horizontal projection image; the horizontal critical value is a vertical coordinate of a critical point of a chin and a neck in the face area;
and the human face region image intercepting unit is used for intercepting a human face region image from the moving object region image according to the vertical first critical value, the vertical second critical value, the horizontal critical value and the highest point vertical coordinate.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention provides a face shielding detection method and a face shielding detection system based on video monitoring, wherein the method comprises the steps of obtaining a dynamic video image collected by video monitoring equipment; detecting a moving object of the dynamic video image by adopting an algorithm combining a three-frame difference method and a mixed Gaussian background model, and determining a region image of the moving object; extracting and marking each connected region in the moving object region image, and determining the maximum connected region; carrying out vertical and horizontal projection processing on the maximum communication area, and intercepting a face area image; and carrying out face shielding detection on the face region image by adopting a K nearest neighbor classification algorithm and a local binary pattern algorithm. By adopting the algorithm and the face region image determining means, the face occlusion detection can be realized with less time consumption, high real-time performance, high precision and low error.
In addition, when the face is detected to be shielded, the video monitoring equipment sends out alarm reminding to actively prevent illegal criminal behaviors.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic flow chart of a face occlusion detection method based on video monitoring according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a method for determining an image of a moving object region according to the present invention;
FIG. 3 is a diagram illustrating the detection effect of a moving object according to the present invention;
FIG. 4 is a diagram showing the effect of the image of the moving object region after mathematical morphology processing according to the present invention;
FIG. 5 is a schematic diagram of the largest connected domain of the present invention;
FIG. 6 is a diagram illustrating the effect of vertically projecting an image according to the present invention;
FIG. 7 is a schematic illustration of a perpendicular projection image of the present invention marking critical points;
FIG. 8 is a diagram illustrating the effect of projecting an image horizontally in accordance with the present invention;
FIG. 9 is a schematic illustration of a horizontal projection image of a marked critical point of the present invention;
FIG. 10 is a schematic flow chart of a face occlusion detection method according to the present invention;
FIG. 11 is a schematic representation of a classical AR database of the present invention;
FIG. 12 is a schematic structural diagram of a face occlusion detection system based on video monitoring according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a face shielding detection method and system based on video monitoring, which can realize face shielding detection with less time consumption, high real-time performance, high precision and low error, particularly carry out real-time detection on a face with two shielding ornaments of sunglasses and scarves and judge the types of the two ornaments, and send out a warning when detecting the face with the sunglasses or the scarves, thereby actively preventing illegal criminal behaviors and being applied to monitoring videos of bank ATM machines.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of a face occlusion detection method based on video monitoring according to an embodiment of the present invention, and as shown in fig. 1, the face occlusion detection method based on video monitoring according to the embodiment of the present invention includes the following steps.
Step 101: and acquiring a dynamic video image acquired by the video monitoring equipment.
Step 102: and detecting a moving object of the dynamic video image by adopting an algorithm combining a three-frame difference method and a mixed Gaussian background model, and determining a region image of the moving object.
Step 103: and extracting and marking each connected region in the moving object region image, and determining the maximum connected region.
Step 104: and carrying out vertical and horizontal projection processing on the maximum connected region, and intercepting a face region image.
Step 105: and carrying out face shielding detection on the face region image by adopting a K nearest neighbor classification algorithm and a local binary pattern algorithm.
Step 106: and when the face is detected to be shielded, the video monitoring equipment sends out an alarm prompt.
Step 102 specifically includes:
because the external environment of the ATM changes, a method capable of updating the background is required to be used for detecting moving objects, and the code running speed is considered as much as possible, the method is realized by combining a three-frame difference method and a Gaussian mixture background model, so that the detection speed can be improved, a better detection effect can be obtained, the updating time interval of background parameters is limited, and the running speed is further improved.
Fig. 2 is a schematic flow chart of the method for determining the image of the moving object region according to the present invention, as shown in fig. 2, including:
step 201: judging whether a moving object enters the dynamic video image in real time by adopting an algorithm combining a three-frame difference method and a Gaussian mixture background model to obtain a first judgment result; if the first judgment result indicates that the moving object enters the dynamic video image, executing step 202; if the first determination result indicates that the moving object is not entered into the dynamic video image, step 203 is executed.
Step 202: and determining that the moving object exists in the dynamic video image, and extracting an image area corresponding to the moving object.
Step 203: and updating the background parameters of the Gaussian mixture background model.
The purpose of updating the background parameters is to solve the problem of interference on detection caused by changes (such as light rays and the like) in a video scene by modeling each pixel point and setting parameters such as weight, learning rate, standard deviation and the like, comparing a new pixel point with an established model in each frame to judge whether the pixel point is matched with the established model, classifying the pixel point into the model if the pixel point is matched with the established model, updating the model according to the new pixel value, and establishing a Gaussian model by using the pixel if the pixel point is not matched with the established model to initialize the parameters to replace the least probable model in the original model.
The update time interval of the background parameter is because the background is updated for a plurality of times first, which can effectively prevent the video from shaking; secondly, considering that most of the bank ATM machines are fixed, the background can be updated at any time (the background updating only occurs under the condition that no moving object enters), but a time period (the length is set according to the actual scene) can be set, and the updating is carried out once every other time period, so that the running speed is improved.
After the step 102, the detected moving object effect graph is shown in fig. 3, and it can be seen that white point noise appears in the detected moving object effect graph, and a hole appears in a required moving object region, and measures taken for this phenomenon are as follows: firstly, filling by using mathematical morphology and a method of expanding in a small area, filling a hole and then corroding in a large area, and the obtained effect graph is shown in fig. 4. Marking each connected domain, solving the size of each connected domain to obtain the index of the maximum connected domain, wherein the schematic diagram of the maximum connected domain is shown in fig. 5.
Therefore, step 103 specifically includes:
and carrying out mathematical morphology processing on the moving object region image.
And extracting and marking each connected region of the processed moving object region image.
The area of the connected region of each marker is calculated.
And determining the maximum connected region according to the area of the connected region of each mark.
In order to obtain an accurate face region, the maximum connected domain is respectively subjected to vertical projection and horizontal projection.
Firstly, vertical projection is carried out, and an effect diagram is shown in FIG. 6; two special points can be seen in fig. 6, which is schematically shown in fig. 7, and the marked points can reflect the horizontal coordinates of the left and right boundary points of the face region, which are respectively set as a1 and a 2.
Then, horizontal projection is performed, the effect diagram is shown in fig. 8, and similarly, a special point can be seen from fig. 8, the schematic diagram is shown in fig. 9, the point reflects the ordinate of the critical point of the chin and the neck of the detected face region, and is set as b 1.
The idea of finding out a1, a2 and b1 is that since a1, a2 and b1 are critical points at which the function corresponding to the projected image starts to change rapidly, the first derivative of each point of the function corresponding to the projected image can be calculated, and the larger the derivative is, the steeper the derivative is, so as to find out the three critical points.
And then finding the vertical coordinate min _ y of the highest point of the contour of the moving object by utilizing a bwbounderies function. Using the imcrop function: the imcrop (original image, [ a1(1), min _ y +30, a2(1) -a1(1), b1-min _ y ]), and a desired face region image is cut out on the original image (moving object region image).
That is, step 104 specifically includes:
and acquiring the highest point vertical coordinate of the contour of the moving object in the maximum communication area by adopting a bwbounderies function.
And carrying out vertical projection processing on the maximum communication area to obtain a vertical projection image.
Determining two critical values of the vertical projection image, wherein the two critical values are a vertical first critical value and a vertical second critical value respectively; the vertical first critical value and the vertical second critical value are horizontal coordinates of left and right boundary points of the face area.
And carrying out horizontal projection processing on the vertical projection image to obtain a horizontal projection image.
Determining a horizontal critical value of the horizontal projection image; the horizontal critical value is the ordinate of the critical point of the chin and the neck in the face area.
And intercepting a face region image from the moving object region image according to the vertical first critical value, the vertical second critical value, the horizontal critical value and the highest point vertical coordinate.
After the face region image is intercepted, face occlusion detection is realized by a K-nearest neighbor (KNN) classification algorithm and a Local Binary Pattern (LBP) algorithm, and a specific flow diagram is shown in fig. 10.
The classical AR database as shown in fig. 11 is used for sample training.
Firstly, training is carried out on two or hundred normal faces, faces with sunglasses and faces with scarves, and then detection is carried out to obtain a training model, wherein the detection results are shown in table 1.
TABLE 1 face occlusion training test result table
Occlusion type Detecting pictures Detection rate
Is normal 50 94％
Sunglasses 120 95.4％
Scarf
100 96％
Then, the played dynamic video image (the face region image obtained after processing) is detected by adopting a training model, and the detection of the scarf is changed into the detection of the mask due to the actual situation, and the detection result is shown in table 2.
Table 2 table of actual face shielding detection results
Occlusion type Number of entries Detection rate
Is normal 50 90％
Sunglasses
100 88％
Gauze mask
100 92％
In order to achieve the purpose, the invention also provides a face shielding detection system based on video monitoring.
Fig. 12 is a schematic structural diagram of a face occlusion detection system based on video monitoring according to an embodiment of the present invention, as shown in fig. 12, the face occlusion detection system according to the embodiment of the present invention includes:
the dynamic video image obtaining module 100 is configured to obtain a dynamic video image collected by a video monitoring device.
And the moving object region image determining module 200 is configured to detect a moving object of the dynamic video image by using an algorithm combining a three-frame difference method and a gaussian mixture background model, and determine a moving object region image.
And a maximum connected region determining module 300, configured to extract and mark each connected region in the moving object region image, and determine a maximum connected region.
And a face region image intercepting module 400, configured to perform vertical and horizontal projection processing on the maximum connected region, and intercept a face region image.
And the face shielding detection module 500 is configured to perform face shielding detection on the face region image by using a K-nearest neighbor classification algorithm and a local binary pattern algorithm.
And the alarm reminding module 600 is configured to send an alarm reminding by the video monitoring device when the face is detected to be blocked.
The moving object region image determining module 200 specifically includes:
and the first judgment result obtaining unit is used for judging whether the moving object enters the dynamic video image in real time by adopting an algorithm combining a three-frame difference method and a Gaussian mixture background model to obtain a first judgment result.
And the moving object region image extraction unit is used for determining that the moving object exists in the dynamic video image and extracting an image region corresponding to the moving object if the first judgment result shows that the moving object enters the dynamic video image.
And the background parameter updating unit is used for updating the background parameter of the Gaussian mixture background model when the first judgment result shows that the moving object is not entered into the dynamic video image.
The maximum connected region determining module 300 specifically includes:
and the mathematical morphology processing unit is used for performing mathematical morphology processing on the moving object region image.
And the connected region extraction marking unit is used for extracting and marking each connected region of the processed moving object region image.
And a connected region area calculating unit for calculating the area of the connected region of each mark.
And a maximum connected region determining unit configured to determine a maximum connected region according to an area of the connected region of each of the marks.
The face region image intercepting module 400 specifically includes:
and the highest vertical coordinate acquisition unit is used for acquiring the highest vertical coordinate of the contour of the moving object in the maximum communication area by adopting a bwboundaries function.
And the vertical projection image obtaining unit is used for performing vertical projection processing on the maximum communication area to obtain a vertical projection image.
The vertical critical value determining unit is used for determining two critical values of the vertical projection image, wherein the two critical values are a vertical first critical value and a vertical second critical value respectively; the vertical first critical value and the vertical second critical value are horizontal coordinates of left and right boundary points of the face area.
And the horizontal projection image obtaining unit is used for carrying out horizontal projection processing on the vertical projection image to obtain a horizontal projection image.
A horizontal critical value determining unit for determining a horizontal critical value of the horizontal projection image; the horizontal critical value is the ordinate of the critical point of the chin and the neck in the face area.
And the human face region image intercepting unit is used for intercepting a human face region image from the moving object region image according to the vertical first critical value, the vertical second critical value, the horizontal critical value and the highest point vertical coordinate.
Compared with the prior art, the invention has the advantages that;
the method of combining the three-frame difference method with the Gaussian mixture background model is utilized, so that the detection precision is improved, and the calculation time of the system is saved.
The updating time interval of the background parameters in the Gaussian mixture background model is limited, so that the adaptability to the actual scene is not reduced, and the computing time of the system is saved.
The human face is divided by adopting a projection method, so that the time is saved, and the detection effect is good.
The method based on the K nearest neighbor classification algorithm (approach algorithm) and the local binary pattern is used for detecting and judging and identifying the shielded human face, and has a good detection effect.
The method or the system provided by the invention relates to the field of intelligent monitoring, in particular to the technologies of image processing, machine learning and the like, and aiming at the crime cases of the ATM, the face shielding detection method and the system based on video monitoring are designed through an algorithm, so that a good detection effect can be achieved on the action that criminals often wear shielding ornaments to hide the facial features of the criminals, and the illegal criminal actions can be actively prevented.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.

Claims (4)

1.一种基于视频监控的人脸遮挡检测方法，其特征在于，所述人脸遮挡检测方法包括：1. a face occlusion detection method based on video surveillance, is characterized in that, described face occlusion detection method comprises: 获取视频监控设备采集的动态视频图像；Obtain dynamic video images collected by video surveillance equipment; 采用三帧差法与混合高斯背景模型相结合的算法，检测所述动态视频图像的运动物体，并确定运动物体区域图像；An algorithm combining the three-frame difference method and the mixed Gaussian background model is used to detect the moving object of the dynamic video image, and determine the area image of the moving object; 提取所述运动物体区域图像中的各个连通区域并标记，确定最大连通区域；对所述运动物体区域图像进行数学形态学处理，其中对所述运动物体区域图像进行数学形态学处理，处理方式为先小面积膨胀、填洞再大面积腐蚀进行填充；提取处理后的运动物体区域图像的各个连通区域并标记；计算各个标记的连通区域的面积；根据各个所述标记的连通区域的面积，确定最大连通区域；Extracting and marking each connected area in the moving object area image to determine the maximum connected area; performing mathematical morphological processing on the moving object area image, wherein the moving object area image is subjected to mathematical morphological processing, and the processing method is as follows First dilate a small area, fill a hole, and then corrode a large area for filling; extract each connected area of the processed moving object area image and mark it; calculate the area of each marked connected area; according to the area of each marked connected area, determine maximum connected area; 对所述最大连通区域进行垂直和水平投影处理，截取人脸区域图像；采用bwboundaries函数获取所述最大连通区域中运动物体轮廓的最高点纵坐标；对所述最大连通区域进行垂直投影处理，得到垂直投影图像；确定所述垂直投影图像的两个临界值，分别为垂直第一临界值和垂直第二临界值；所述垂直第一临界值、所述垂直第二临界值为人脸区域的左、右界限点的横坐标；对所述垂直投影图像进行水平投影处理，得到水平投影图像；确定所述水平投影图像的水平临界值；所述水平临界值为人脸区域中下巴与脖子的临界点的纵坐标；根据所述垂直第一临界值、所述垂直第二临界值、所述水平临界值以及所述最高点纵坐标，从所述运动物体区域图像中截取人脸区域图像；Perform vertical and horizontal projection processing on the maximum connected area, and intercept the face area image; use the bwboundaries function to obtain the highest point ordinate of the contour of the moving object in the maximum connected area; perform vertical projection processing on the maximum connected area to obtain Vertical projection image; determine two critical values of the vertical projection image, which are respectively the vertical first critical value and the vertical second critical value; the vertical first critical value and the vertical second critical value are the left side of the face area. , the abscissa of the right limit point; perform horizontal projection processing on the vertical projection image to obtain a horizontal projection image; determine the horizontal critical value of the horizontal projected image; the horizontal critical value is the critical point of the chin and neck in the face area The ordinate; according to the vertical first critical value, the vertical second critical value, the horizontal critical value and the vertical coordinate of the highest point, the face area image is intercepted from the moving object area image; 将左、右界限点的所述横坐标分别设置为a1和a2，所述纵坐标设置为b1，找出所述a1、所述a2和所述b1的思路为：所述a1、所述a2和所述b1为投影之后的图像所对应的函数开始急剧变化的临界点，计算投影之后的图像所对应的函数每个点的一阶导数，导数越大越陡峭，从而找出这三个临界点；Set the abscissas of the left and right limit points as a1 and a2, respectively, and the ordinate as b1. The idea of finding the a1, the a2, and the b1 is: the a1, the a2 and the b1 is the critical point where the function corresponding to the projected image begins to change sharply, calculate the first-order derivative of each point of the function corresponding to the projected image, the larger the derivative, the steeper, so as to find these three critical points ; 采用K最近邻分类算法和局部二值模式算法对所述人脸区域图像进行人脸遮挡检测；The K-nearest neighbor classification algorithm and the local binary pattern algorithm are used to perform face occlusion detection on the face region image; 所述采用三帧差法与混合高斯背景模型相结合的算法，检测所述动态视频图像的运动物体，并确定运动物体区域图像，具体包括：The algorithm that adopts the combination of the three-frame difference method and the mixed Gaussian background model to detect the moving object of the dynamic video image, and determine the region image of the moving object, specifically includes: 采用三帧差法与混合高斯背景模型相结合的算法实时判断所述动态视频图像中是否进入运动物体，得到第一判断结果；若所述第一判断结果表示所述动态视频图像中进入所述运动物体，则确定所述动态视频图像存在所述运动物体，提取所述运动物体对应的图像区域；若所述第一判断结果表示所述动态视频图像中未进入所述运动物体，则更新所述混合高斯背景模型的背景参数。An algorithm combining the three-frame difference method and the mixed Gaussian background model is used to judge in real time whether a moving object enters the dynamic video image, and a first judgment result is obtained; if the first judgment result indicates that the dynamic video image enters the moving object If there is a moving object in the dynamic video image, it is determined that the moving object exists in the dynamic video image, and the image area corresponding to the moving object is extracted; if the first judgment result indicates that the moving object does not enter the dynamic video image, update the The background parameters of the Gaussian mixture background model described above. 2.根据权利要求1所述的人脸遮挡检测方法，其特征在于，所述人脸遮挡检测方法还包括：2. The face occlusion detection method according to claim 1, wherein the face occlusion detection method further comprises: 当检测到人脸被遮挡时，所述视频监控设备发出报警提醒。When detecting that the face is blocked, the video surveillance device sends out an alarm reminder. 3.一种基于视频监控的人脸遮挡检测系统，其特征在于，所述人脸遮挡检测系统包括：3. a face occlusion detection system based on video surveillance, is characterized in that, described face occlusion detection system comprises: 动态视频图像获取模块，用于获取视频监控设备采集的动态视频图像；A dynamic video image acquisition module is used to acquire dynamic video images collected by video surveillance equipment; 运动物体区域图像确定模块，用于采用三帧差法与混合高斯背景模型相结合的算法，检测所述动态视频图像的运动物体，并确定运动物体区域图像；A moving object area image determination module, used for detecting the moving object in the dynamic video image by using the algorithm combining the three frame difference method and the mixed Gaussian background model, and determining the moving object area image; 最大连通区域确定模块，用于提取所述运动物体区域图像中的各个连通区域并标记，确定最大连通区域；所述最大连通区域确定模块，具体包括：数学形态学处理单元，用于对所述运动物体区域图像进行数学形态学处理；连通区域提取标记单元，用于提取处理后的运动物体区域图像的各个连通区域并标记；连通区域面积计算单元，用于计算各个标记的连通区域的面积；最大连通区域确定单元，用于根据各个所述标记的连通区域的面积，确定最大连通区域；The maximum connected area determination module is used to extract and mark each connected area in the moving object area image to determine the maximum connected area; the maximum connected area determination module specifically includes: a mathematical morphology processing unit, used for The moving object area image is subjected to mathematical morphological processing; the connected area extraction and marking unit is used to extract and mark each connected area of the processed moving object area image; the connected area area calculation unit is used to calculate the area of each marked connected area; a maximum connected area determination unit, configured to determine the maximum connected area according to the area of each of the marked connected areas; 人脸区域图像截取模块，用于对所述最大连通区域进行垂直和水平投影处理，截取人脸区域图像；所述人脸区域图像截取模块，具体包括：最高纵坐标获取单元，用于采用bwboundaries函数获取所述最大连通区域中运动物体轮廓的最高点纵坐标；垂直投影图像得到单元，用于对所述最大连通区域进行垂直投影处理，得到垂直投影图像；垂直临界值确定单元，用于确定所述垂直投影图像的两个临界值，分别为垂直第一临界值和垂直第二临界值；所述垂直第一临界值、所述垂直第二临界值为人脸区域的左、右界限点的横坐标；水平投影图像得到单元，用于对所述垂直投影图像进行水平投影处理，得到水平投影图像；水平临界值确定单元，用于确定所述水平投影图像的水平临界值；所述水平临界值为人脸区域中下巴与脖子的临界点的纵坐标；人脸区域图像截取单元，根据所述垂直第一临界值、所述垂直第二临界值、所述水平临界值以及所述最高点纵坐标，从所述运动物体区域图像中截取人脸区域图像；The face area image interception module is used to perform vertical and horizontal projection processing on the maximum connected area, and intercept the face area image; the face area image interception module specifically includes: the highest ordinate acquisition unit, used for using bwboundaries The function obtains the ordinate of the highest point of the contour of the moving object in the maximum connected area; the vertical projection image obtaining unit is used to perform vertical projection processing on the maximum connected area to obtain a vertical projection image; the vertical critical value determination unit is used to determine The two critical values of the vertical projection image are respectively the vertical first critical value and the vertical second critical value; the vertical first critical value and the vertical second critical value are the difference between the left and right boundary points of the face area. abscissa; a horizontal projection image obtaining unit for performing horizontal projection processing on the vertical projection image to obtain a horizontal projection image; a horizontal critical value determining unit for determining the horizontal critical value of the horizontal projected image; the horizontal critical value The value is the ordinate of the critical point of the chin and neck in the face area; the face area image interception unit, according to the vertical first critical value, the vertical second critical value, the horizontal critical value and the vertical value of the highest point coordinates, and intercept the face area image from the moving object area image; 将左、右界限点的所述横坐标分别设置为a1和a2，所述纵坐标设置为b1，找出所述a1、所述a2和所述b1的思路为：所述a1、所述a2和所述b1为投影之后的图像所对应的函数开始急剧变化的临界点，计算投影之后的图像所对应的函数每个点的一阶导数，导数越大越陡峭，从而找出这三个临界点；Set the abscissas of the left and right limit points as a1 and a2, respectively, and the ordinate as b1. The idea of finding the a1, the a2, and the b1 is: the a1, the a2 and the b1 is the critical point where the function corresponding to the projected image begins to change sharply, calculate the first-order derivative of each point of the function corresponding to the projected image, the larger the derivative, the steeper, so as to find these three critical points ; 人脸遮挡检测模块，用于采用K最近邻分类算法和局部二值模式算法对所述人脸区域图像进行人脸遮挡检测；a face occlusion detection module, which is used for performing face occlusion detection on the image of the face region by using the K nearest neighbor classification algorithm and the local binary pattern algorithm; 所述运动物体区域图像确定模块，具体包括：The moving object area image determination module specifically includes: 第一判断结果得到单元，用于采用三帧差法与混合高斯背景模型相结合的算法实时判断所述动态视频图像中是否进入运动物体，得到第一判断结果；运动物体区域图像提取单元，用于若所述第一判断结果表示所述动态视频图像中进入所述运动物体时，确定所述动态视频图像存在所述运动物体，提取所述运动物体对应的图像区域；背景参数更新单元，用于当所述第一判断结果表示所述动态视频图像中未进入所述运动物体时，更新所述混合高斯背景模型的背景参数。The first judgment result obtaining unit is used to judge in real time whether there is a moving object in the dynamic video image by using the algorithm combining the three frame difference method and the mixed Gaussian background model, and obtain the first judgment result; If the first judgment result indicates that the moving object is entered in the dynamic video image, determine that the moving object exists in the dynamic video image, and extract the image area corresponding to the moving object; the background parameter updating unit, using When the first judgment result indicates that the moving object does not enter the dynamic video image, the background parameters of the mixed Gaussian background model are updated. 4.根据权利要求3所述的人脸遮挡检测系统，其特征在于，所述人脸遮挡检测系统还包括：4. The face occlusion detection system according to claim 3, wherein the face occlusion detection system further comprises: 报警提醒模块，用于当检测到人脸被遮挡时，所述视频监控设备发出报警提醒。The alarm reminder module is used for the video surveillance device to issue an alarm reminder when it is detected that the face is blocked.
CN201810801599.7A 2018-07-20 2018-07-20 Face shielding detection method and system based on video monitoring Active CN109002801B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810801599.7A CN109002801B (en) 2018-07-20 2018-07-20 Face shielding detection method and system based on video monitoring

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810801599.7A CN109002801B (en) 2018-07-20 2018-07-20 Face shielding detection method and system based on video monitoring

Publications (2)

Publication Number Publication Date
CN109002801A CN109002801A (en) 2018-12-14
CN109002801B true CN109002801B (en) 2021-01-15

Family

ID=64597322

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810801599.7A Active CN109002801B (en) 2018-07-20 2018-07-20 Face shielding detection method and system based on video monitoring

Country Status (1)

Country Link
CN (1) CN109002801B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109902720B (en) * 2019-01-25 2020-11-27 同济大学 Image Classification and Recognition Method Based on Subspace Decomposition for Depth Feature Estimation
CN110519515B (en) * 2019-08-28 2021-03-19 联想(北京)有限公司 Information processing method and electronic equipment
CN110913209B (en) * 2019-12-05 2021-06-04 杭州飞步科技有限公司 Camera occlusion detection method, device, electronic device and monitoring system
CN113096059B (en) * 2019-12-19 2023-10-31 合肥君正科技有限公司 Method for eliminating interference shielding detection of night light source by in-vehicle monitoring camera
CN111428581B (en) * 2020-03-05 2023-11-21 平安科技（深圳）有限公司 Face shielding detection method and system
CN112287823A (en) * 2020-10-28 2021-01-29 怀化学院 A method of facial mask recognition based on video surveillance
CN112507926B (en) * 2020-12-16 2022-07-01 电子科技大学 Go game repeating method based on video image analysis
CN114708626A (en) * 2021-12-17 2022-07-05 武汉众智数字技术有限公司 Method for identifying and early warning behaviors of key personnel
CN114821795B (en) * 2022-05-05 2022-10-28 北京容联易通信息技术有限公司 Personnel running detection and early warning method and system based on ReiD technology

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102184401A (en) * 2011-04-29 2011-09-14 苏州两江科技有限公司 Facial feature extraction method
CN103927519A (en) * 2014-04-14 2014-07-16 中国华戎控股有限公司 Real-time face detection and filtration method

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8150155B2 (en) * 2006-02-07 2012-04-03 Qualcomm Incorporated Multi-mode region-of-interest video object segmentation
CN101883209B (en) * 2010-05-31 2012-09-12 中山大学 Method for integrating background model and three-frame difference to detect video background
CN102147861A (en) * 2011-05-17 2011-08-10 北京邮电大学 Moving target detection method for carrying out Bayes judgment based on color-texture dual characteristic vectors
CN102663400B (en) * 2012-04-16 2014-06-04 北京博研新创数码科技有限公司 LBP (length between perpendiculars) characteristic extraction method combined with preprocessing
CN102855496B (en) * 2012-08-24 2016-05-25 苏州大学 Block face authentication method and system
CN103400110B (en) * 2013-07-10 2016-11-23 上海交通大学 Abnormal face detecting method before ATM cash dispenser
CN107992864A (en) * 2018-01-15 2018-05-04 武汉神目信息技术有限公司 A kind of vivo identification method and device based on image texture

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102184401A (en) * 2011-04-29 2011-09-14 苏州两江科技有限公司 Facial feature extraction method
CN103927519A (en) * 2014-04-14 2014-07-16 中国华戎控股有限公司 Real-time face detection and filtration method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Research on the Feature Displacement Map of the Facial Expression;Wenbai C.等;《2010 Asia-Pacific Conference on Wearable Computing Systems》;20100607;第319-321 *

Also Published As

Publication number Publication date
CN109002801A (en) 2018-12-14

Similar Documents

Publication Publication Date Title
CN109002801B (en) Face shielding detection method and system based on video monitoring
CN110648352B (en) Abnormal event detection method and device and electronic equipment
CN113011385B (en) Face silence living body detection method, face silence living body detection device, computer equipment and storage medium
CN104063722B (en) A kind of detection of fusion HOG human body targets and the safety cap recognition methods of SVM classifier
JP6549797B2 (en) Method and system for identifying head of passerby
WO2021003824A1 (en) Image recognition-based illegal building identification method and device
CN103942539B (en) A kind of oval accurate high efficiency extraction of head part and masking method for detecting human face
CN101188743A (en) A video-based intelligent counting system and its processing method
CN105469105A (en) Cigarette smoke detection method based on video monitoring
CN102867179A (en) Method for detecting acquisition quality of digital certificate photo
CN106355154B (en) Method for detecting frequent passing of people in surveillance video
CN112184773A (en) Helmet wearing detection method and system based on deep learning
CN112434578A (en) Mask wearing normative detection method and device, computer equipment and storage medium
CN112464850B (en) Image processing method, device, computer equipment and medium
WO2013075295A1 (en) Clothing identification method and system for low-resolution video
CN105513053A (en) Background modeling method for video analysis
CN112287823A (en) A method of facial mask recognition based on video surveillance
CN115830719B (en) Building site dangerous behavior identification method based on image processing
CN111985331B (en) Detection method and device for preventing trade secret from being stolen
CN106570447A (en) Face photo sunglass automatic removing method based on gray histogram matching
CN103927518A (en) Facial feature extraction method for facial analysis system
CN107862298B (en) A living body detection method based on blinking under infrared camera device
CN111274888B (en) Helmet and work clothes intelligent identification method based on wearable mobile glasses
CN113792629B (en) A safety helmet wearing detection method and system based on deep neural network
CN111178167B (en) Method and device for checking lasting lens, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant