CN106778695A - A kind of many people's examing heartbeat fastly methods based on video - Google Patents

Abstract

The invention relates to the technical field of human health monitoring, and discloses a video-based rapid heart rate detection method for multiple people, which uses a non-contact method to collect video data of the subject through a camera, and converts the color space of the video data from RGB to HSV to realize Quickly measure the heart rate of multiple people. It is characterized in that the method detects and tracks the video input by the camera or multiple face areas in the existing video and segments the cheek area, extracts the time domain data sequence of the cheek area and performs preprocessing, and then converts to the frequency domain Extract the heart rate. Compared with the prior art, the present invention is based on an improved face detection algorithm and an improved tracking algorithm and is accelerated by a multi-thread method, which can realize rapid heart rate detection of multiple people, shorten the time of heart rate measurement, and improve the detection of physiological signals efficiency.

Description

A video-based fast heart rate detection method for multiple people

technical field

The patent of the present invention relates to the technical field of human health monitoring, in particular to a method that can realize rapid heart rate measurement of multiple people.

Background technique

Heart rate refers to the number of times the heart beats per minute and varies with age, gender, and other physiological conditions. The heart rate of newborns is very fast, reaching more than 130 beats/min. There are significant individual differences in the resting heart rate of normal adults, with an average of about 75 beats/min (60-100 beats/min). For the same person, the heart rate slows down when resting or sleeping, and increases when exercising or having mood swings. Therefore, heart rate can fully reflect a person's physical condition, is an important physiological parameter for self-health monitoring, and is also an important basis for doctors to diagnose diseases for patients.

Heart rate can be divided into contact measurement and non-contact measurement according to different measurement methods. One of the representatives of contact measurement is the gold standard of heart rate measurement—electrocardiogram (EEG). In addition, there are contact measurement methods that wrap chest straps, cuffs or electrodes on wrists, fingertips, earlobes, etc. The biggest disadvantage is that the measurement operation is complicated, the measurement period is long, and the contact with the skin will bring discomfort to the subject. The non-contact measurement method based on photoplethysmography (IPPG) overcomes the disadvantages of the contact heart rate measurement method. The disadvantage is that it is a non-contact measurement method that is widely used at present.

However, the non-contact measurement method based on the existing IPPG is sensitive to light changes, the measurement speed is low, the measurement results are easily affected by motion artifacts, the accuracy of the heart rate measurement results is low, and most of them are based on the heart rate measurement of a single person. And other issues.

Contents of the invention

1. The present invention aims to solve at least one of the above-mentioned technical problems.

2. For this reason, the object of the present invention is to propose a kind of non-contact multi-person fast heart rate detection method based on video, this method can be to the video that comprises a plurality of human faces or the people in the video that contains a plurality of human faces obtained by camera. Face automatic detection and fast tracking, after analyzing and processing the obtained face area, the heart rate of each subject can be obtained.

3. The method includes the following parts: video acquisition part, face detection part, face tracking part, ROI tone frame extraction part, time domain signal acquisition part, time domain signal processing part, heart rate calculation part, face number and heart rate display part;

4. The video acquisition part is used to select the working mode: one is to turn on the camera, and in the indoor environment of daily lighting, fix the camera after determining the position where the imaging device can clearly and completely image the face area; the other is to select a section of local Existing videos containing face regions;

5. The face detection part is used to detect the face area from the video, returns the face number and the heart rate display part to each face area number, and initializes the face tracking part;

6. The face tracking part tracks each face area detected by the face detection part;

7. The ROI tone sub-frame extraction part is used to extract the color space of each frame image ROI region in the video by the value of the Hue (hue) component after RGB is converted to HSV;

8. The time-domain signal acquisition part is used to divide the face in the ROI area image of each frame into the cheek area, and get the grayscale mean value of the pixel for the Hue (tone) component in the three color components of HSV in this area, The time-domain signal value X(t) of the H component in the cheek area;

9. The time-domain signal processing part is used to perform noise suppression and signal detrending on the obtained time-domain signal value X(t), and obtain the processed time-domain signal value

10. The heart rate calculation part is used for time domain signal value Perform spectrum analysis and generate a spectrogram, extract the peak frequency in the specified frequency band from the spectrogram for heart rate calculation;

11. The face number and heart rate display part is used to display the face area number marked by the face detection part and the heart rate value corresponding to the face area number obtained by the heart rate calculation part in the video.

Preferably, the video acquisition part is realized by controlling the imaging device through a PC.

Preferably, the face detection part is implemented by simultaneously loading the nose, mouth, and frontal face classifiers.

Preferably, the face tracking part is realized by an improved compressed tracking method suitable for multiple people tracking.

Preferably, the ROI hue frame extraction part is realized by converting the color space from RGB to HSV and extracting the H component.

Preferably, the time-domain signal acquisition part is realized by extracting the gray mean value of the cheek area.

Preferably, the time-domain signal processing part is realized by means of mean filtering, wavelet denoising, and moving average.

Preferably, the heart rate calculation part is realized by fast Fourier transform method.

Preferably, the face number and the heart rate display are partially implemented by a multi-threaded method.

Description of drawings

1. Fig. 1 is a heart rate measurement method step diagram of the present invention

2. Fig. 2 is the block diagram of each part that the present invention comprises

3. Fig. 3 is a flow chart of the heart rate measurement method of the present invention and a multi-thread schematic diagram

4. Fig. 4 is a flow chart of the human face detection part of the heart rate measurement method of the present invention

5. Fig. 5 is the part flow chart of face tracking of heart rate measurement method of the present invention

6. Figure 6 is a positional diagram of the heart rate measurement method of the present invention in the face tracking part of the target detection frame and its adjacent detection frames

detailed description

1. In order to clearly illustrate the purpose, technical solutions and advantages of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

2. As shown in Figure 1, it is a step diagram of the video-based rapid heart rate measurement method for multiple people of the present invention.

3. As shown in FIG. 2 , it is a block diagram of various parts included in the video-based rapid heart rate measurement method for multiple people of the present invention.

4. The first step is to select the working mode. The present invention provides two working modes: one is to directly detect the real-time heart rate of the subject through the camera, and the other is to perform real-time heart rate detection on the subject in the local video. For videos in both working modes, it is necessary to select an environment with suitable lighting, and make sure that the imaging device can clearly and completely image the face area. The following steps take working mode 1 as an example.

5. The second step is to start the imaging device, collect the video of the face of the subject to be tested, and then decompose the video input by the camera into an image sequence and convert the RGB image into a grayscale image. During the acquisition process, the face is allowed to appear in the range of the imaging scene Internal movement and deflection.

6. The third step is to judge the current face detection status. If no face is detected, start the face detection part until a face is detected, return the face number to the face number and heart rate display part, and initialize with the detection result In the tracking part, the face detection status is marked as a detected face at the same time. The workflow of this part is shown in Figure 4.

7. The fourth step is to start the face tracking part to track all the currently detected faces separately, and the tracking module returns to the tracking status. If the tracking is successful, continue to the next step, otherwise return to the third step. The tracking process is shown in the figure 5. The search radius in tracking is updated according to Figure 6: among them, the 8 frames numbered 1 to 8 represent the 8 adjacent frames that may exist around the current target frame, and the area of the frame represents the tracked people at different positions The size of the face area. As shown by the coordinate axes in the figure, assuming that the parameters of the target frame are (x ₀ , y ₀ , w ₀ , h ₀ ), the four parameters in turn indicate the abscissa, ordinate, and rectangle width of the top left corner of the rectangle corresponding to the target frame , the height of the rectangle; similarly, it can be assumed that the parameters of the eight tracking frames are ( _xi , y _i , w _i , h _i ), where i takes 1, 2, 3, 4, 5, 6, 7, 8 in sequence; The following is an example of calculating the distance between the target frame and the tracking frame in the upper right corner. Other situations are similar, and the conditions are met at this time:

The minimum distance obtained is:

Then the search radius is:

r _search =l _min *0.8

8. The fifth step is to perform mean filtering on each tracked face region (ROI), segment the cheek region, extract the gray mean value of the H component of the cheek region, and perform wavelet denoising on the obtained time domain signal X(t) , sliding filtering, and signal detrending to obtain the final time domain signal

9. The sixth step is to perform fast Fourier transform on the time domain signal obtained in the previous step, and select the frequency corresponding to the largest spectrum value between 0.5Hz and 3Hz, which corresponds to the value per minute, which is the heart rate. The 0.5Hz~3Hz selected here means that the heart rate is in the range of 30~180 beats/min, which covers most possible heart rate ranges and eliminates the interference of other physiological signals. For the implementation process of this step, see the thread in Figure 3 2.

10. The seventh step is to enable thread 3 to display the heart rate data corresponding to different face numbers in the video in real time.

11. Beneficial effects: Compared with the prior art, the present invention proposes a fast multi-person heart rate measurement method based on image processing, and realizes the acceleration of multi-person heart rate detection through a multi-thread method and an improved tracking algorithm. The detection algorithm and the tracking algorithm reduce the false detection rate and improve the detection efficiency.

Claims (11)

1. A video-based multi-person fast heart rate detection method, the method may further comprise the steps: S100, select the working mode, if it is the direct working mode, then call the program to automatically open the camera, if it is the indirect working mode, then call the program to automatically read the video, and then call the face detection part to detect the face area in the video; S200. Transfer the detected face area position information to the face tracking part, start the face tracking, and simultaneously obtain image information such as the average gray value of the tracked face area; S300. Process the obtained image information and convert it into a time-domain signal, and then filter, denoise, detrend, etc. the time-domain signal to obtain pre-processed data and start thread 2, transform it into the frequency domain, and obtain the frequency domain data, and calculate the heart rate according to the frequency domain data; S400, start the thread 3, and display the number of each detected object and its heart rate value in the video in real time. 2. A video-based rapid heart rate detection method for multiple people. This method is mainly used in daily non-contact heart rate measurement systems. Imaging devices such as web cameras and mobile phone cameras are used to shoot videos in areas containing faces to realize automatic heart rate measurement; It is characterized in that the method includes a video acquisition part, a face detection part, a face tracking part, a frame extraction ROI part, a time domain signal acquisition part, a time domain signal processing part, a heart rate calculation part, a face number and a heart rate display part ; The video collection part is used to obtain a section of color video images that include a plurality of human face regions by means of a camera or select a local video file; the face detection part is used to detect faces on the video images collected; The face tracking part is used to track the detected face to speed up the processing speed of each frame of image; the ROI tone sub-frame extraction part is used to extract the color space of each frame of picture from RGB to H after HSV conversion. The value of the (Hue) component; the time-domain signal acquisition part is used to divide the cheek region from the ROI region of each frame, and obtain the gray mean value of the H component of the cheek region as the feature value of the frame image, and generate the time domain Signal X(t) in the domain; the time domain signal processing part is used to perform noise suppression on the obtained time domain signal X(t) to obtain the processed time domain signal The heart rate calculation part is used for the time domain signal value Perform spectrum analysis and generate a spectrogram, and extract the peak frequency within the specified frequency band from the spectrogram for heart rate calculation. 3. The multi-person rapid heart rate detection method according to claim 1, characterized in that: in the step S200, specifically comprising the following steps: S201. Set a new search radius according to the distance and positional relationship between the tracking frame and its nearest neighbor in the last image frame; S202. Extract features from samples collected near the tracking position according to the new search radius and map them to a low-dimensional space to obtain a region to be classified; S203. Use the two Bayesian classifiers obtained in the previous frame to classify the regions to be classified respectively, and select the most likely rectangular frame as the target as the current tracking result; If it is judged that the current tracking object has moved to the edge of the entire video frame, then the marker tracking is unsuccessful, the face detection module is restarted, and the current heart rate detection result is cleared, otherwise step S204 is performed; S204. Taking the target area of the current frame as the center, take two groups of positive and negative samples, which are: 1) Take 4 pixels as the radius in the target area, take 45 positive samples, and randomly select 50 negative samples in a circle with 8 as the inner radius and 12 as the outer radius outside the target area; 2) Take 4 pixels as the inner diameter and 6 pixels as the outer diameter in the target area, take out 60 positive samples, and randomly select 60 negative samples from a circle with 12 as the inner radius and 16 as the outer radius outside the target area. sample; S205, calculating the integral map of the original image and the Haar feature extraction template; S206. According to the integral map and the obtained Haar feature extraction template, extract the features of the positive and negative samples, update the Bayesian separator, obtain a new classifier, and use the classifier to track the target in the current frame image. 4. The rapid heart rate detection method for multiple people according to claim 1, characterized in that: the heart rate measurement is realized by using imaging devices commonly used in daily life such as webcams or mobile phone cameras. 5. The multi-person rapid heart rate detection method according to claim 1, characterized in that: there are two optional working modes, which can directly detect the heart rate of the person under test through the camera or can detect the face area in the local video. Heart rate detection. 6. the many people's fast heart rate detection method according to claim 1 is characterized in that: described human face detection module reduces misdetection rate by loading face classifier, nose classifier, mouth classifier three classifiers, The missed detection rate is reduced by performing histogram equalization on the image to be detected. 7. The multiple people's fast heart rate detection method according to claim 1 is characterized in that: the face tracking module utilizes the improved Compression Tracking (Compress Tracking) algorithm of the present invention to suppress tracking in the tracking process by two methods The drift of the frame is: 1) According to the 8 kinds of positional relationships between the current detection frame and its adjacent detection frames, calculate the distance under each positional relationship and select the minimum distance l _min from it, and set the search radius r _search = l _min *0.8, to avoid Too large a search radius results in aliasing of feature extraction between tracked targets. 2) Using the idea of ensemble learning, 45 positive samples are taken in the target area with a radius of 4 pixels, and 50 negative samples are randomly selected in a circle with an inner radius of 8 and an outer radius of 12 outside the target area. Sample; take 4 pixels as the inner diameter and 6 pixels as the outer diameter, take 60 positive samples, and randomly select 60 negative samples in a circle with 12 as the inner radius and 16 as the outer radius outside the target area; The two groups of positive and negative samples are sent to the classifier, and the mean value of the maximum position returned by the classifier in these two cases is used as the position of the tracking target. 8. The multi-person rapid heart rate detection method according to claim 1, characterized in that: the face tracking module detects that the detected object moves out of the detection field of view, clears the heart rate data of the object, exits the face tracking part and Restart the face detection part. 9. The rapid heart rate detection method for many people according to claim 1, characterized in that: divide the cheek area from the face area, and extract the heart rate signal, avoiding the blinking of the eyes and the blocking of the measured forehead part of the hair The impact on the gray value of ROI improves the accuracy of heart rate measurement. 10. A video-based fast heart rate detection method for multiple people, which uses multi-threading to accelerate heart rate detection. The heart rate calculation part, face number and heart rate display part occupy separate threads, without affecting the work of other modules, and realize multiple Rapid detection of human heart rate. 11. The multi-person rapid heart rate detection method according to claim 1, characterized in that: said face number and heart rate display part are marked on each face tracking frame in sequence according to the order of faces detected by the face detection part The corresponding number is displayed, and the heart rate value corresponding to each subject is displayed in sequence in the order of the number from small to large in the video.