CN116935165A - Method for acquiring offline quantization calibration set of face detection model - Google Patents
Method for acquiring offline quantization calibration set of face detection model Download PDFInfo
- Publication number
- CN116935165A CN116935165A CN202310965151.XA CN202310965151A CN116935165A CN 116935165 A CN116935165 A CN 116935165A CN 202310965151 A CN202310965151 A CN 202310965151A CN 116935165 A CN116935165 A CN 116935165A
- Authority
- CN
- China
- Prior art keywords
- face
- sample
- average
- quality
- threshold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0495—Quantised networks; Sparse networks; Compressed networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/164—Detection; Localisation; Normalisation using holistic features
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
本发明涉及计算机视觉领域，具体涉及一种用于获取人脸检测模型离线量化校准集的方法，在获取人脸检测数据集之后，将人脸检测数据集中的各个样本按不同场景进行分类，并确定人脸检测数据集中各个样本对应的场景质量标签、人脸密度标签、人脸平均面积标签、人脸平均朝向标签、人脸平均质量标签后，在人脸检测数据集中筛选出符合要求的若干样本，形成人脸检测模型离线量化校准集。本发明通过对人脸检测数据集中各个样本进行打标存储，自动选择具有代表性、多样性的若干样本形成人脸检测模型离线量化校准集，相对于人工制作人脸检测模型离线量化校准集来说，大大的提高了工作效率，操作简单，可扩展性强。
The invention relates to the field of computer vision, and specifically relates to a method for obtaining an offline quantitative calibration set of a face detection model. After obtaining a face detection data set, each sample in the face detection data set is classified according to different scenarios, and After determining the scene quality label, face density label, average face area label, average face orientation label, and average face quality label corresponding to each sample in the face detection data set, select several that meet the requirements in the face detection data set. Samples form an offline quantitative calibration set for the face detection model. The present invention marks and stores each sample in the face detection data set, and automatically selects a number of representative and diverse samples to form an offline quantitative calibration set of the face detection model. Compared with the manually produced offline quantitative calibration set of the face detection model, Said that it greatly improves work efficiency, is simple to operate and has strong scalability.
Description
Technical Field
The invention relates to the field of computer vision, in particular to a method for acquiring an offline quantization calibration set of a face detection model.
Background
Model quantization refers to converting floating point operation of a neural network into fixed point operation, so as to accelerate reasoning speed, reduce equipment power consumption and reduce storage space, and is mainly divided into offline quantization (Post Training Quantization, PTQ) and quantized perception training (Quantization Aware Training, QAT). Typically, the chip manufacturer performs quantization operation before deploying the deep learning model to the edge device operation, and provides an offline quantization scheme based on a calibration set, which requires a chip user to select 50-100 pictures of a typical scene as the calibration set according to a specific quantization scheme to collect information of data distribution, including minimum/maximum values, optimal threshold values based on entropy theory, quantization factors based on symmetric quantization, and the like. Eventually, these quantization parameters will be recorded in the newly generated quantization model, and a reasonable calibration set can reduce the accuracy loss of the quantization process.
However, how to quickly and accurately select a typical scene picture sample as an offline quantization calibration set of a face detection model can only depend on manual selection at present, and is low in efficiency.
Disclosure of Invention
Aiming at the corresponding defects of the prior art, the invention provides a method for acquiring an offline quantization calibration set of a face detection model, which marks samples in a face detection data set according to different dimensions, such as scene ID, scene quality, face density, face average orientation, face average quality, face average size and the like, and automatically screens more representative and diversified sample calibration sets.
The invention aims at realizing the following scheme, namely a method for acquiring an offline quantization calibration set of a face detection model, which classifies each sample in the face detection data set after acquiring the face detection data set and processes each sample in the face detection data set according to the following steps:
1) Determining scene quality labels corresponding to all samples in the face detection data set;
2) Determining face density labels corresponding to all samples in a face detection data set;
3) Determining face average area labels corresponding to all samples in a face detection data set;
4) Determining face average orientation labels corresponding to all samples in the face detection data set;
5) Determining face average quality labels corresponding to all samples in a face detection data set;
and finally, screening out a plurality of samples meeting the requirements from the face detection data set to form an offline quantization calibration set of the face detection model.
Preferably, after the face detection data set is acquired, the type of the face detection data set is determined, and each sample in the face detection data set is classified in the following manner:
(1) if the face detection data set is a public data set, extracting the characteristics of each sample, clustering through a characteristic matching algorithm to obtain a plurality of clustering centers, setting a corresponding scene ID label for each clustering center, and classifying each sample in the face detection data set by using different scene ID labels;
(2) If the face detection data set is a self-collection data set, setting a corresponding scene ID label for each sample collection device, and classifying each sample in the face detection data set by using different scene ID labels.
Preferably, the scene quality label comprises black and white and color, and the specific steps of detection are as follows:
(1) setting a black-and-white picture judgment threshold;
(2) splitting the sample into three channels R, G and B, calculating the difference value of each pixel coordinate of the sample between the color information of each channel, counting the number of pixel coordinates with zero difference value between the color information of each channel, and judging the size relation between the number and a black-and-white picture judgment threshold value:
if the number is more than or equal to the black-and-white picture judgment threshold value, setting the scene quality label of the sample to be black-and-white;
if the number is less than the black-and-white picture judgment threshold value, setting the scene quality label of the sample to be 'color';
preferably, the scene quality label further comprises "blurring", "sharpness", and the specific steps of detecting are as follows:
(1) setting a fuzzy picture judgment threshold;
(2) converting the sample into a gray picture, carrying out Laplace transformation, solving the variance of pixels in the gray picture, and judging the size relation between the variance and a fuzzy picture judgment threshold value:
If the variance is less than or equal to the fuzzy picture judgment threshold value, setting the scene quality label of the sample as 'fuzzy';
if the variance is larger than the fuzzy picture judgment threshold value, setting the scene quality label of the sample as clear;
preferably, the scene quality label further includes "color cast" and "unbiased", and the specific steps of determining whether the scene quality label corresponding to each sample in the face detection data set is "color cast" or "unbiased" are as follows:
(1) setting a color cast picture judgment threshold;
(2) the samples are converted from the RGB color space to the CIELAB color space and the bias values of the samples are calculated according to the following formula:
where da is the sample in CIELAB color space a * The mean value of the on-axis components, ma, is the CIELAB color space a for the sample * The variance of the on-axis component, db, is the sample in the CIELAB color space b * The mean value of the on-axis component, mb, is the CIELAB color space b for the sample * The variance of the component on the axis, K, is the bias value of the sample;
(3) judging the magnitude relation between the color cast value of the sample and the color cast picture judgment threshold value:
if the color cast value of the sample is more than or equal to the color cast picture judgment threshold value, setting the scene quality label of the sample as 'color cast';
If the color cast value of the sample is less than the color cast image judgment threshold value, setting the scene quality label of the sample as 'unbiased';
preferably, the scene quality label further comprises "normal brightness", "over-dark", "over-bright", and the specific steps of detecting are as follows:
(1) setting a normal brightness judgment threshold, wherein the normal brightness judgment threshold comprises a normal brightness judgment upper limit value and a normal brightness judgment lower limit value;
(2) converting the sample into a gray picture, and calculating the average value of pixels in the gray picture;
(3) judging the size relation between the average value of pixels in the gray level picture and the normal brightness judgment threshold value:
if the average value of pixels in the gray level picture is less than or equal to the normal brightness judgment lower limit value, setting the scene quality label of the sample to be 'excessively dark';
if the average value of pixels in the gray level picture is within the normal brightness judgment threshold value range, setting the scene quality label of the sample as normal brightness;
if the average value of pixels in the gray level picture is more than or equal to the normal brightness judgment upper limit value, setting the scene quality label of the sample to be 'over bright'.
Preferably, the specific manner of determining the face density label corresponding to each sample in the face detection data set is as follows:
2-1) setting a face density threshold, wherein the face density threshold comprises an upper limit value and a lower limit value of the face density;
2-2) reading labeling information of a sample, counting the total number of faces in the sample, and judging the size relation between the total number of faces in the sample and a face density threshold value:
if the total number of faces is smaller than the lower limit value of the face density, setting the face density label of the sample as low density;
if the total number of faces is within the face density threshold range, setting the face density label of the sample as medium density;
if the total number of faces is greater than the upper limit value of the face density, the face density label of the sample is set as 'high density'.
Preferably, the specific manner of determining the face average area label corresponding to each sample in the face detection data set is as follows:
3-1) setting a face average area threshold, wherein the face average area threshold comprises an upper limit value and a lower limit value of a face average area;
3-2) reading labeling information of a sample, counting the area of each face in the sample, and judging the size relation between the average face area in the sample and the average face area threshold value:
if the average face area is smaller than the lower limit value of the average face area, setting the average face area label of the sample as a small face;
If the average face area is within the average face area threshold range, setting the average face area label of the sample as a medium face;
and if the average face area is larger than the upper limit value of the average face area, setting the average face area label of the sample as a large face.
Preferably, the specific manner of determining the average face orientation label corresponding to each sample in the face detection data set is as follows:
4-1) setting an orientation deflection threshold;
4-2) reading labeling information of a sample, and calculating a pitch angle, a yaw angle and a roll angle corresponding to each labeling face image in the sample by using a head posture estimation network;
4-3) calculating an average value of the sum of absolute values of each pitch angle, an average value of the sum of absolute values of each yaw angle and an average value of the sum of absolute values of each roll angle, and judging the magnitude relation between each average value and the deflection threshold value:
if each average value is smaller than the orientation deflection threshold value, setting the face average orientation label of the sample as 'front orientation';
if any average value is greater than or equal to the orientation deflection threshold, the face average orientation label of the sample is set to be "side orientation".
Preferably, the specific manner of determining the face average quality label corresponding to each sample in the face detection data set is as follows:
5-1) setting a face quality threshold, wherein the face quality threshold comprises an upper limit value and a lower limit value of the face quality;
5-2) reading labeling information of any sample, and respectively sending each labeling face image in the sample into a face quality detection network to obtain corresponding face quality scores;
5-3) summing the face quality scores of the marked face images, then averaging, and comparing the obtained average value with a face quality threshold value:
if the average value is smaller than the lower limit value of the face quality, setting the face average quality label of the sample as a low-quality face;
if the average value is within the face quality threshold value range, setting the face average quality label of the sample as a 'medium quality face';
if the average value is larger than the upper limit value of the face quality, the face average quality label of the sample is set as a 'high-quality face'.
Preferably, a sample simplifying threshold is set, and before a plurality of samples meeting the requirements are screened out from the face detection data set to form an offline quantization calibration set of the face detection model, all samples corresponding to scene types with the number of samples lower than the sample simplifying threshold are deleted to simplify the face detection data set.
Preferably, a plurality of samples meeting the requirements are screened out from the face detection data set, and the specific mode for forming the off-line quantization calibration set of the face detection model is as follows:
Setting a scene quality control threshold, a face density control threshold, a face average area control threshold, a face average orientation control threshold and a face average quality control threshold;
preferably, the scene quality control threshold includes a normal color sample number threshold, a normal black and white sample number threshold;
preferably, the face density control threshold includes a low density sample number threshold, a medium density sample number threshold, a high density sample number threshold;
preferably, the face average area control threshold includes a small face sample number threshold, a medium face sample number threshold, and a large face sample number threshold;
preferably, the face average orientation control threshold includes a front orientation sample number threshold and a side orientation sample number threshold;
preferably, the face average quality control threshold includes a low quality sample number threshold, a medium quality sample number threshold, a high quality sample number threshold;
randomly screening a plurality of normal color samples and normal black-and-white samples from the face detection data set according to a scene quality control threshold value to form a scene quality sample set;
the normal color sample is a sample with four scene quality labels of color, clear, unbiased and normal brightness at the same time;
The normal black-and-white sample is a sample with four scene quality labels of black-and-white, clear, unbiased and normal brightness at the same time;
preferably, the number of normal color samples in the scene quality sample set is a normal color sample number threshold, and the number of normal black-and-white samples in the scene quality sample set is a normal black-and-white sample number threshold;
randomly screening samples with the face density labels of low density, medium density and high density from a face detection data set according to a face density control threshold value to form a face density sample set;
preferably, the number of samples with the face density label of "low density" in the face density sample set is a low density sample number threshold, the number of samples with the face density label of "medium density" is a medium density sample number threshold, and the number of samples with the face density label of "high density" is a high density sample number threshold;
fourthly, randomly screening samples of which the face average area labels are 'small faces', 'medium faces' and 'large faces' from a face detection data set according to a face average area control threshold value to form a face average area sample set;
Preferably, the number of samples with the face average area label of "small face" in the face average area sample set is a small face sample number threshold, the number of samples with the face average area label of "medium density" is a medium face sample number threshold, and the number of samples with the face average area label of "high density" is a large face sample number threshold;
fifthly, randomly screening samples with face average orientation labels of 'front orientation' and 'side orientation' from the face detection data set according to a face average orientation control threshold value to form a face average orientation sample set;
preferably, the number of samples with the face average orientation label of "front orientation" in the face average orientation sample set is a front orientation sample number threshold, and the number of samples with the face average orientation label of "side orientation" is a side orientation sample number threshold;
the method comprises the steps of (a) randomly screening samples with face average quality labels of low quality, medium quality and high quality from a face detection data set according to a face average quality control threshold value to form a face average quality sample set;
preferably, the number of samples with the face average quality label of "low quality" in the face average quality sample set is a low quality sample number threshold, the number of samples with the face average quality label of "medium quality" is a medium quality sample number threshold, and the number of samples with the face average quality label of "high quality" is a high quality sample number threshold;
And screening out repeated samples in the scene quality sample set, the face density sample set, the face average area sample set, the face average orientation sample set and the face average quality sample set, and combining the rest samples into a face detection model offline quantization calibration set.
The invention has the advantages that:
(1) the method and the device have the advantages that corresponding attribute labels are arranged on each sample in the face detection data set, the operation is simple, and the expandability is high;
(2) the invention utilizes the computer to automatically select a plurality of representative and diversified samples to form the offline quantization calibration set of the face detection model, and compared with the process of manually manufacturing the offline quantization calibration set of the face detection model, the invention has simple operation, strong expandability and extremely high working efficiency.
Drawings
FIG. 1 is a flow chart of the present invention.
Detailed Description
As shown in fig. 1, a method for acquiring an offline quantization calibration set of a face detection model, after acquiring a face detection data set, first determines the type of the face detection data set, and classifies each sample in the face detection data set according to the following manner:
(1) if the face detection data set is a public data set, extracting the characteristics of each sample, clustering through a characteristic matching algorithm to obtain a plurality of clustering centers, setting a corresponding scene ID label for each clustering center, and classifying each sample in the face detection data set by using different scene ID labels;
For example, the face detection dataset is a public dataset, firstly, a feature point set of each sample picture in the public dataset is extracted in batches, then one sample picture is randomly selected as a clustering center, the number of feature points matched with the rest sample pictures in the public dataset is calculated by using a feature matching algorithm, if the quotient of the number of the matched feature points divided by the number of the feature points of the sample picture is greater than 0.1, the two sample pictures are considered to belong to one category, the same scene ID label is marked for the sample pictures, after all samples are traversed, one sample picture is randomly selected in the rest samples as a clustering center, and the process is repeated until all the sample pictures are marked.
In this embodiment, the feature point extraction algorithm includes a sift algorithm, a kaze/akaze algorithm, a superpoint (Self-Supervised Interest Point Detection and Description) algorithm, and the like; the feature matching algorithm includes nearest neighbor matching algorithm, violent matching algorithm, superglue (Learning Feature Matching with Graph Neural Networks) matching algorithm, etc.
(2) If the face detection data set is a self-collection data set, setting a corresponding scene ID label for each sample collection device, and classifying each sample in the face detection data set by using different scene ID labels.
After classifying each sample in the face detection data set, processing each sample in the face detection data set according to the following steps:
1) Determining scene quality labels corresponding to all samples in the face detection data set;
in this embodiment, the scene quality label includes "black and white" and "color", and the judgment mode is specifically as follows:
(1) setting a black-and-white picture judgment threshold, wherein in the embodiment, the black-and-white picture judgment threshold is 5% of the total number of pixels of a sample to be processed;
(2) splitting the sample into three channels R, G and B, calculating the difference value of each pixel coordinate of the sample between the color information of each channel, counting the number of pixel coordinates with zero difference value between the color information of each channel, and judging the size relation between the number and a black-and-white picture judgment threshold value:
if the number is more than or equal to the black-and-white picture judgment threshold value, setting the scene quality label of the sample to be black-and-white;
if the number is less than the black-and-white picture judgment threshold value, setting the scene quality label of the sample to be 'color';
in this embodiment, the scene quality tag further includes "blur", "clear", and the specific steps of detection are as follows:
(1) Setting a fuzzy picture judgment threshold, wherein in the embodiment, the fuzzy picture judgment threshold is set to be 100;
(2) converting the sample into a gray picture, carrying out Laplace transformation, solving the variance of pixels in the gray picture, and judging the size relation between the variance and a fuzzy picture judgment threshold value:
if the variance is less than or equal to the fuzzy picture judgment threshold value, setting the scene quality label of the sample as 'fuzzy';
if the variance is larger than the fuzzy picture judgment threshold value, setting the scene quality label of the sample as clear;
in this embodiment, the scene quality label further includes "color cast" and "unbiased", and the specific steps of determining whether the scene quality label corresponding to each sample in the face detection dataset is "color cast" or "unbiased" are as follows:
(1) setting a color-shifting picture judgment threshold, wherein in the embodiment, the color-shifting picture judgment threshold is set to be 1.5;
(2) the samples are converted from the RGB color space to the CIELAB color space and the bias values of the samples are calculated according to the following formula:
where da is the sample in CIELAB color space a * The mean value of the on-axis components, ma, is the sample's color empty at CIELABInterval a * The variance of the on-axis component, db, is the sample in the CIELAB color space b * The mean value of the on-axis component, mb, is the CIELAB color space b for the sample * The variance of the component on the axis, K, is the bias value of the sample;
(3) judging the magnitude relation between the color cast value of the sample and the color cast picture judgment threshold value:
if the color cast value of the sample is more than or equal to the color cast picture judgment threshold value, setting the scene quality label of the sample as 'color cast';
if the color cast value of the sample is less than the color cast image judgment threshold value, setting the scene quality label of the sample as 'unbiased';
in this embodiment, the scene quality label further includes "normal brightness", "too dark", "too bright", and the specific steps of detection are as follows:
(1) setting a normal brightness judgment threshold, wherein the normal brightness judgment threshold comprises a normal brightness judgment upper limit value and a normal brightness judgment lower limit value;
in this embodiment, the normal brightness determination upper limit value is 200, and the normal brightness determination lower limit value is 40;
(2) converting the sample into a gray picture, and calculating the average value of pixels in the gray picture;
(3) judging the size relation between the average value of pixels in the gray level picture and the normal brightness judgment threshold value:
if the average value of pixels in the gray level picture is less than or equal to the normal brightness judgment lower limit value, setting the scene quality label of the sample to be 'excessively dark';
If the average value of pixels in the gray level picture is within the normal brightness judgment threshold value range, setting the scene quality label of the sample as normal brightness;
if the average value of pixels in the gray level picture is more than or equal to the normal brightness judgment upper limit value, setting the scene quality label of the sample to be 'over bright'.
2) Face density labels corresponding to the samples in the face detection data set are determined according to the following mode:
2-1) setting a face density threshold, wherein the face density threshold comprises an upper limit value and a lower limit value of the face density;
in this embodiment, the upper limit value of the face density is 10, and the lower limit value of the face density is 5;
2-2) reading labeling information of a sample, counting the total number of faces in the sample, and judging the size relation between the total number of faces in the sample and a face density threshold value:
if the total number of faces is smaller than the lower limit value of the face density, setting the face density label of the sample as low density;
if the total number of faces is within the face density threshold range, setting the face density label of the sample as medium density;
if the total number of faces is greater than the upper limit value of the face density, the face density label of the sample is set as 'high density'.
3) The average face area label corresponding to each sample in the face detection data set is determined according to the following mode:
3-1) setting a face average area threshold, wherein the face average area threshold comprises an upper limit value and a lower limit value of a face average area;
in this embodiment, the upper limit value of the average face area is 9216 pixels, and the lower limit value of the average face area is 1024 pixels;
3-2) reading labeling information of a sample, counting the area of each face in the sample, calculating the average face area of the sample (namely, the sum of the areas of each face divided by the number of faces of the sample), and judging the size relation between the average face area in the sample and the average face area threshold value:
if the average face area is smaller than the lower limit value of the average face area, setting the average face area label of the sample as a small face;
if the average face area is within the average face area threshold range, setting the average face area label of the sample as a medium face;
and if the average face area is larger than the upper limit value of the average face area, setting the average face area label of the sample as a large face.
4) The average face orientation label corresponding to each sample in the face detection data set is determined according to the following mode:
4-1) setting a threshold of deflection towards, in this embodiment, 20 °;
4-2) reading labeling information of a sample, and calculating a pitch angle, a yaw angle and a roll angle corresponding to each labeling face image in the sample by using a head posture estimation network;
in this embodiment, the head pose estimation network used includes HopeNet (Fine-Grained Head Pose Estimation Without Keypoints), img2 post (Face Alignment and Detection via 6DoF,Face Pose Estimation), FSA-Net (Learning Fine-Grained Structure Aggregation for Head Pose Estimation from a Single Image), and the like.
4-3) calculating an average value of the sum of absolute values of each pitch angle, an average value of the sum of absolute values of each yaw angle and an average value of the sum of absolute values of each roll angle, and judging the magnitude relation between each average value and the deflection threshold value:
if each average value is smaller than the orientation deflection threshold value, setting the face average orientation label of the sample as 'front orientation';
if any average value is greater than or equal to the orientation deflection threshold, the face average orientation label of the sample is set to be "side orientation".
5) The average face quality label corresponding to each sample in the face detection data set is determined according to the following mode:
5-1) setting a face quality threshold, wherein the face quality threshold comprises an upper limit value and a lower limit value of the face quality;
in this embodiment, the upper limit value of the face quality is 0.6, and the lower limit value of the face quality is 0.3;
5-2) reading labeling information of any sample, and respectively sending each labeling face image in the sample into a face quality detection network to obtain corresponding face quality scores;
in this embodiment, the face quality detection network includes EQFace (A Simple Explicit Quality Network for Face Recognition), SER-FIQ (Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness), PLQ (Pixel-Level Face Image Quality Assessment for Explainable Face Recognition) and the like.
5-3) summing the face quality scores of the marked face images, then averaging, and comparing the obtained average value with a face quality threshold value:
if the average value is smaller than the lower limit value of the face quality, setting the face average quality label of the sample as a low-quality face;
if the average value is within the face quality threshold value range, setting the face average quality label of the sample as a 'medium quality face';
If the average value is larger than the upper limit value of the face quality, the face average quality label of the sample is set as a 'high-quality face'.
And finally, screening out a plurality of samples meeting the requirements from the face detection data set to form an offline quantization calibration set of the face detection model, wherein the method comprises the following specific steps of:
setting a scene quality control threshold, a face density control threshold, a face average area control threshold, a face average orientation control threshold and a face average quality control threshold;
in this embodiment, the scene quality control threshold includes a normal color sample number threshold and a normal black-and-white sample number threshold;
the face density control threshold comprises a low-density sample number threshold, a medium-density sample number threshold and a high-density sample number threshold;
the face average area control threshold comprises a small face sample number threshold, a medium face sample number threshold and a large face sample number threshold;
the face average orientation control threshold comprises a front orientation sample number threshold and a side orientation sample number threshold;
the face average quality control threshold comprises a low quality sample number threshold, a medium quality sample number threshold and a high quality sample number threshold;
Randomly screening a plurality of normal color samples and normal black-and-white samples from the face detection data set according to a scene quality control threshold value to form a scene quality sample set;
the normal color sample is a sample with four scene quality labels of color, clear, unbiased and normal brightness at the same time;
the normal black-and-white sample is a sample with four scene quality labels of black-and-white, clear, unbiased and normal brightness at the same time;
in this embodiment, the number of normal color samples in the scene quality sample set is a normal color sample number threshold, and the number of normal black-and-white samples in the scene quality sample set is a normal black-and-white sample number threshold;
randomly screening samples with the face density labels of low density, medium density and high density from a face detection data set according to a face density control threshold value to form a face density sample set;
in this embodiment, the number of samples with a face density label of "low density" in the face density sample set is a low density sample number threshold, the number of samples with a face density label of "medium density" is a medium density sample number threshold, and the number of samples with a face density label of "high density" is a high density sample number threshold;
Fourthly, randomly screening samples of which the face average area labels are 'small faces', 'medium faces' and 'large faces' from a face detection data set according to a face average area control threshold value to form a face average area sample set;
in this embodiment, the number of samples with the face average area label of "small face" in the face average area sample set is a small face sample number threshold, the number of samples with the face average area label of "medium density" is a medium face sample number threshold, and the number of samples with the face average area label of "high density" is a large face sample number threshold;
fifthly, randomly screening samples with face average orientation labels of 'front orientation' and 'side orientation' from the face detection data set according to a face average orientation control threshold value to form a face average orientation sample set;
in this embodiment, the number of samples with the face average orientation label of "front orientation" in the face average orientation sample set is a front orientation sample number threshold, and the number of samples with the face average orientation label of "side orientation" is a side orientation sample number threshold;
the method comprises the steps of (a) randomly screening samples with face average quality labels of low quality, medium quality and high quality from a face detection data set according to a face average quality control threshold value to form a face average quality sample set;
In this embodiment, the number of samples with a face average quality label of "low quality" in the face average quality sample set is a low quality sample number threshold, the number of samples with a face average quality label of "medium quality" is a medium quality sample number threshold, and the number of samples with a face average quality label of "high quality" is a high quality sample number threshold;
and screening out repeated samples in the scene quality sample set, the face density sample set, the face average area sample set, the face average orientation sample set and the face average quality sample set, and combining the rest samples into a face detection model offline quantization calibration set.
It should be noted that, in order to simplify the face detection data set and improve the quantization precision, a sample simplifying threshold is usually set, before a plurality of samples meeting the requirements are screened out from the face detection data set to form an offline quantization calibration set of the face detection model, all samples corresponding to scene types with the number of samples lower than the sample simplifying threshold are deleted, that is, the samples in the offline quantization calibration set of the finally generated face detection model cannot have scene quality labels of "blurs", "color bias", "darkness" and "brightness", and the samples belong to extreme samples, so that the quantization precision is affected and need to be deleted, so as to ensure that all the scene types corresponding to all the samples in the offline quantization calibration set of the finally obtained face detection model are representative.
In addition, each threshold related to the embodiment is an empirical value, and the steps of determining each threshold are as follows:
(1) determining initial set values of all thresholds according to an offline quantization scheme provided by a chip manufacturer;
(2) according to the method, an offline quantization calibration set of the face detection model is obtained;
(3) and (3) carrying out model quantization processing on the face detection model offline quantization calibration set according to an offline quantization scheme provided by a chip manufacturer, judging whether the precision loss in the quantization process meets the requirement of the offline quantization scheme, if not, adjusting the value of each threshold value, and repeating the step (2) until the precision loss in the quantization process meets the requirement of the offline quantization scheme.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the invention, and those skilled in the art will appreciate that the modifications made to the invention fall within the scope of the invention without departing from the spirit of the invention.
Claims (9)
1. A method for acquiring an offline quantization calibration set of a face detection model, characterized in that after acquiring a face detection dataset, each sample in the face detection dataset is classified and processed according to the following steps:
1) Determining scene quality labels corresponding to all samples in the face detection data set;
2) Determining face density labels corresponding to all samples in a face detection data set;
3) Determining face average area labels corresponding to all samples in a face detection data set;
4) Determining face average orientation labels corresponding to all samples in the face detection data set;
5) Determining face average quality labels corresponding to all samples in a face detection data set;
and finally, screening out a plurality of samples meeting the requirements from the face detection data set to form an offline quantization calibration set of the face detection model.
2. The method for acquiring an offline quantization calibration set of a face detection model according to claim 1, wherein after acquiring the face detection dataset, the type of the face detection dataset is determined and the individual samples in the face detection dataset are classified as follows:
(1) if the face detection data set is a public data set, extracting the characteristics of each sample, clustering through a characteristic matching algorithm to obtain a plurality of clustering centers, setting a corresponding scene ID label for each clustering center, and classifying each sample in the face detection data set by using different scene ID labels;
(2) If the face detection data set is a self-collection data set, setting a corresponding scene ID label for each sample collection device, and classifying each sample in the face detection data set by using different scene ID labels.
3. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein the scene quality tags include "black and white", "color", the specific steps of detecting are as follows:
(1) setting a black-and-white picture judgment threshold;
(2) splitting the sample into three channels R, G and B, calculating the difference value of each pixel coordinate of the sample between the color information of each channel, counting the number of pixel coordinates with zero difference value between the color information of each channel, and judging the size relation between the number and a black-and-white picture judgment threshold value:
if the number is more than or equal to the black-and-white picture judgment threshold value, setting the scene quality label of the sample to be black-and-white;
if the number is less than the black-and-white picture judgment threshold value, setting the scene quality label of the sample to be 'color';
preferably, the scene quality label further comprises "blurring", "sharpness", and the specific steps of detecting are as follows:
(1) setting a fuzzy picture judgment threshold;
(2) Converting the sample into a gray picture, carrying out Laplace transformation, solving the variance of pixels in the gray picture, and judging the size relation between the variance and a fuzzy picture judgment threshold value:
if the variance is less than or equal to the fuzzy picture judgment threshold value, setting the scene quality label of the sample as 'fuzzy';
if the variance is larger than the fuzzy picture judgment threshold value, setting the scene quality label of the sample as clear;
preferably, the scene quality label further comprises "off-color", "no off-color", and the specific steps of detecting are as follows:
(1) setting a color cast picture judgment threshold;
(2) the samples are converted from the RGB color space to the CIELAB color space and the bias values of the samples are calculated according to the following formula:
where da is the sample in CIELAB color space a * The mean value of the on-axis components, ma, is the CIELAB color space a for the sample * The variance of the on-axis component, db, is the sample in the CIELAB color space b * The mean value of the on-axis component, mb, is the CIELAB color space b for the sample * The variance of the component on the axis, K, is the bias value of the sample;
(3) judging the magnitude relation between the color cast value of the sample and the color cast picture judgment threshold value:
if the color cast value of the sample is more than or equal to the color cast picture judgment threshold value, setting the scene quality label of the sample as 'color cast';
If the color cast value of the sample is less than the color cast image judgment threshold value, setting the scene quality label of the sample as 'unbiased';
preferably, the scene quality label further comprises "normal brightness", "over-dark", "over-bright", and the specific steps of detecting are as follows:
(1) setting a normal brightness judgment threshold, wherein the normal brightness judgment threshold comprises a normal brightness judgment upper limit value and a normal brightness judgment lower limit value;
(2) converting the sample into a gray picture, and calculating the average value of pixels in the gray picture;
(3) judging the size relation between the average value of pixels in the gray level picture and the normal brightness judgment threshold value:
if the average value of pixels in the gray level picture is less than or equal to the normal brightness judgment lower limit value, setting the scene quality label of the sample to be 'excessively dark';
if the average value of pixels in the gray level picture is within the normal brightness judgment threshold value range, setting the scene quality label of the sample as normal brightness;
if the average value of pixels in the gray level picture is more than or equal to the normal brightness judgment upper limit value, setting the scene quality label of the sample to be 'over bright'.
4. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein the specific manner of determining the face density label corresponding to each sample in the face detection dataset is as follows:
2-1) setting a face density threshold, wherein the face density threshold comprises an upper limit value and a lower limit value of the face density;
2-2) reading labeling information of a sample, counting the total number of faces in the sample, and judging the size relation between the total number of faces in the sample and a face density threshold value:
if the total number of faces is smaller than the lower limit value of the face density, setting the face density label of the sample as low density;
if the total number of faces is within the face density threshold range, setting the face density label of the sample as medium density;
if the total number of faces is greater than the upper limit value of the face density, the face density label of the sample is set as 'high density'.
5. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein the specific manner of determining the face average area label corresponding to each sample in the face detection data set is as follows:
3-1) setting a face average area threshold, wherein the face average area threshold comprises an upper limit value and a lower limit value of a face average area;
3-2) reading labeling information of a sample, counting the area of each face in the sample, and judging the size relation between the average face area in the sample and the average face area threshold value:
If the average face area is smaller than the lower limit value of the average face area, setting the average face area label of the sample as a small face;
if the average face area is within the average face area threshold range, setting the average face area label of the sample as a medium face;
and if the average face area is larger than the upper limit value of the average face area, setting the average face area label of the sample as a large face.
6. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein the specific manner of determining the average face orientation label corresponding to each sample in the face detection data set is as follows:
4-1) setting an orientation deflection threshold;
4-2) reading labeling information of a sample, and calculating a pitch angle, a yaw angle and a roll angle corresponding to each labeling face image in the sample by using a head posture estimation network;
4-3) calculating an average value of the sum of absolute values of each pitch angle, an average value of the sum of absolute values of each yaw angle and an average value of the sum of absolute values of each roll angle, and judging the magnitude relation between each average value and the deflection threshold value:
if each average value is smaller than the orientation deflection threshold value, setting the face average orientation label of the sample as 'front orientation';
If any average value is greater than or equal to the orientation deflection threshold, the face average orientation label of the sample is set to be "side orientation".
7. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein the specific manner of determining the face average quality label corresponding to each sample in the face detection dataset is as follows:
5-1) setting a face quality threshold, wherein the face quality threshold comprises an upper limit value and a lower limit value of the face quality;
5-2) reading labeling information of any sample, and respectively sending each labeling face image in the sample into a face quality detection network to obtain corresponding face quality scores;
5-3) summing the face quality scores of the marked face images, then averaging, and comparing the obtained average value with a face quality threshold value:
if the average value is smaller than the lower limit value of the face quality, setting the face average quality label of the sample as a low-quality face;
if the average value is within the face quality threshold value range, setting the face average quality label of the sample as a 'medium quality face';
if the average value is larger than the upper limit value of the face quality, the face average quality label of the sample is set as a 'high-quality face'.
8. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein a sample reduction threshold is set, and all samples corresponding to scene types having a number of samples lower than the sample reduction threshold are deleted before a plurality of samples meeting requirements are screened out of the face detection data set to form the offline quantization calibration set of the face detection model, so as to reduce the face detection data set.
9. The method for obtaining an offline quantization calibration set of a face detection model according to claim 1, wherein a plurality of samples meeting requirements are screened out from a face detection data set, and the specific manner of forming the offline quantization calibration set of the face detection model is as follows:
setting a scene quality control threshold, a face density control threshold, a face average area control threshold, a face average orientation control threshold and a face average quality control threshold;
preferably, the scene quality control threshold includes a normal color sample number threshold, a normal black and white sample number threshold;
preferably, the face density control threshold includes a low density sample number threshold, a medium density sample number threshold, a high density sample number threshold;
Preferably, the face average area control threshold includes a small face sample number threshold, a medium face sample number threshold, and a large face sample number threshold;
preferably, the face average orientation control threshold includes a front orientation sample number threshold and a side orientation sample number threshold;
preferably, the face average quality control threshold includes a low quality sample number threshold, a medium quality sample number threshold, a high quality sample number threshold;
randomly screening a plurality of normal color samples and normal black-and-white samples from the face detection data set according to a scene quality control threshold value to form a scene quality sample set;
the normal color sample is a sample with four scene quality labels of color, clear, unbiased and normal brightness at the same time;
the normal black-and-white sample is a sample with four scene quality labels of black-and-white, clear, unbiased and normal brightness at the same time;
preferably, the number of normal color samples in the scene quality sample set is a normal color sample number threshold, and the number of normal black-and-white samples in the scene quality sample set is a normal black-and-white sample number threshold;
Randomly screening samples with the face density labels of low density, medium density and high density from a face detection data set according to a face density control threshold value to form a face density sample set;
preferably, the number of samples with the face density label of "low density" in the face density sample set is a low density sample number threshold, the number of samples with the face density label of "medium density" is a medium density sample number threshold, and the number of samples with the face density label of "high density" is a high density sample number threshold;
fourthly, randomly screening samples of which the face average area labels are 'small faces', 'medium faces' and 'large faces' from a face detection data set according to a face average area control threshold value to form a face average area sample set;
preferably, the number of samples with the face average area label of "small face" in the face average area sample set is a small face sample number threshold, the number of samples with the face average area label of "medium density" is a medium face sample number threshold, and the number of samples with the face average area label of "high density" is a large face sample number threshold;
Fifthly, randomly screening samples with face average orientation labels of 'front orientation' and 'side orientation' from the face detection data set according to a face average orientation control threshold value to form a face average orientation sample set;
preferably, the number of samples with the face average orientation label of "front orientation" in the face average orientation sample set is a front orientation sample number threshold, and the number of samples with the face average orientation label of "side orientation" is a side orientation sample number threshold;
the method comprises the steps of (a) randomly screening samples with face average quality labels of low quality, medium quality and high quality from a face detection data set according to a face average quality control threshold value to form a face average quality sample set;
preferably, the number of samples with the face average quality label of "low quality" in the face average quality sample set is a low quality sample number threshold, the number of samples with the face average quality label of "medium quality" is a medium quality sample number threshold, and the number of samples with the face average quality label of "high quality" is a high quality sample number threshold;
and screening out repeated samples in the scene quality sample set, the face density sample set, the face average area sample set, the face average orientation sample set and the face average quality sample set, and combining the rest samples into a face detection model offline quantization calibration set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310965151.XA CN116935165A (en) | 2023-07-31 | 2023-07-31 | Method for acquiring offline quantization calibration set of face detection model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310965151.XA CN116935165A (en) | 2023-07-31 | 2023-07-31 | Method for acquiring offline quantization calibration set of face detection model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116935165A true CN116935165A (en) | 2023-10-24 |
Family
ID=88386057
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310965151.XA Pending CN116935165A (en) | 2023-07-31 | 2023-07-31 | Method for acquiring offline quantization calibration set of face detection model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116935165A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110909568A (en) * | 2018-09-17 | 2020-03-24 | 北京京东尚科信息技术有限公司 | Image detection method, apparatus, electronic device, and medium for face recognition |
| CN113705650A (en) * | 2021-08-20 | 2021-11-26 | 网易（杭州）网络有限公司 | Processing method, device, medium and computing equipment for face picture set |
| CN114648788A (en) * | 2022-02-23 | 2022-06-21 | 北京市农林科学院 | Quality evaluation method and device for image sample set and storage medium |
| CN115661894A (en) * | 2022-10-13 | 2023-01-31 | 福建捷宇电脑科技有限公司 | A face image quality filtering method |
| WO2023125654A1 (en) * | 2021-12-29 | 2023-07-06 | 杭州海康威视数字技术股份有限公司 | Training method and apparatus for face recognition model, electronic device and storage medium |
-
2023
- 2023-07-31 CN CN202310965151.XA patent/CN116935165A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110909568A (en) * | 2018-09-17 | 2020-03-24 | 北京京东尚科信息技术有限公司 | Image detection method, apparatus, electronic device, and medium for face recognition |
| CN113705650A (en) * | 2021-08-20 | 2021-11-26 | 网易（杭州）网络有限公司 | Processing method, device, medium and computing equipment for face picture set |
| WO2023125654A1 (en) * | 2021-12-29 | 2023-07-06 | 杭州海康威视数字技术股份有限公司 | Training method and apparatus for face recognition model, electronic device and storage medium |
| CN114648788A (en) * | 2022-02-23 | 2022-06-21 | 北京市农林科学院 | Quality evaluation method and device for image sample set and storage medium |
| CN115661894A (en) * | 2022-10-13 | 2023-01-31 | 福建捷宇电脑科技有限公司 | A face image quality filtering method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1374168B1 (en) | Method and apparatus for determining regions of interest in images and for image transmission | |
| CN106803257B (en) | Method for segmenting disease spots in crop disease leaf image | |
| CN110443800B (en) | Video image quality evaluation method | |
| US6996277B2 (en) | Image type classification using color discreteness features | |
| CN109948566B (en) | Double-flow face anti-fraud detection method based on weight fusion and feature selection | |
| JP2007336562A (en) | Digital image region separation method, region separation system, and labeling method | |
| US20030128396A1 (en) | Image type classification using edge features | |
| CN107729812B (en) | A method suitable for vehicle color recognition in surveillance scenes | |
| US10438376B2 (en) | Image processing apparatus replacing color of portion in image into single color, image processing method, and storage medium | |
| Tong et al. | Learning no-reference quality metric by examples | |
| CN104361357B (en) | Photo album categorizing system and sorting technique based on image content analysis | |
| CN114240925A (en) | Method and system for detecting document image definition | |
| Yang et al. | EHNQ: Subjective and objective quality evaluation of enhanced night-time images | |
| CN108769543B (en) | Method and device for determining exposure time | |
| CN119444739A (en) | A method, system, electronic device and storage medium for improving Mura defect classification accuracy | |
| Oszust | No-reference image quality assessment with local features and high-order derivatives | |
| KR102102403B1 (en) | Code authentication method of counterfeit print image and its application system | |
| CN104268509B (en) | Method and system for license plate detection of dump truck | |
| Fernández-Caballero et al. | Display text segmentation after learning best-fitted OCR binarization parameters | |
| CN117994156A (en) | Presswork pattern matching process regulation and control method based on computer vision | |
| KR20190058753A (en) | Image processing method and image processor performing the same | |
| Li et al. | Two-layer average-to-peak ratio based saliency detection | |
| CN112532938B (en) | A video surveillance system based on big data technology | |
| JP2018022416A (en) | Face direction estimation apparatus and program thereof | |
| JP5887242B2 (en) | Image processing apparatus, image processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |