WO2020042419A1 - Gait-based identity recognition method and apparatus, and electronic device - Google Patents

Abstract

A gait-based identity recognition method and apparatus, and an electronic device, relating to the technical field of identity recognition, and solving the problem in the prior art of the accuracy of gait-based identity recognition methods in multiple pedestrian scenarios. The method comprises: implementing pedestrian detection, tracking, and pedestrian re-recognition for a video image sequence to determine target pedestrians in the video image sequence (110); determining a silhouette image sequence of the target pedestrians (120); on the basis of the silhouette image sequence of each target pedestrian, respectively acquiring a gait energy map of the target pedestrian (130); and, on the basis of the gait energy map of the target pedestrians, implementing identity recognition of the target pedestrians (140). The present method incorporates pedestrian detection, tracking, and pedestrian re-recognition technology to implement recognition of the same or different target pedestrians in a video image sequence, increasing the accuracy of the recognition of target pedestrians in a video image sequence, and thereby increasing the accuracy of gait-based identity recognition in multi-person scenarios.

Description

Gait-based identity recognition method, device and electronic equipment Technical field

The present application relates to the field of computer vision technology, and in particular, to a gait-based identity recognition method, device, electronic device, and storage medium.

Background technique

Biometric identification is the identification of individuals through various high-tech information detection methods, using the inherent physiological or behavioral characteristics of the human body. Biological characteristics mainly include physiological characteristics and behavioral characteristics. Behavioral characteristics refer to the characteristics extracted from the movements performed by a person, such as gait, handwriting and other characteristics. These characteristics are mostly acquired characteristics. In recent years, with the rapid development of biometric authentication technology, gait-based identification technology has been widely used in access control systems, security monitoring, human-computer interaction, and medical diagnosis. However, the inventor's research on the prior art found that the security monitoring environment is usually complex. Because a person is a non-rigid body, it is more flexible than a vehicle when moving, its contour features are constantly changing, and pedestrian occlusion is easy to occur in a multi-person environment, and pedestrian gait features are not easy to extract. The accuracy of pedestrian gait information is low, and the accuracy of gait-based identification in multi-person scenarios is still low.

It can be seen that the accuracy of the gait-based identity recognition method in the multi-pedestrian scene in the prior art needs to be improved.

Summary of the Invention

This application provides a gait-based identity recognition method, which helps to improve the accuracy of gait-based identity recognition in a multi-person scenario.

In order to solve the above problem, according to one aspect of the present application, a gait-based identity recognition method is provided, including:

Performing pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence;

Determining a silhouette image sequence of the target pedestrian;

Obtaining a gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian;

Based on the gait energy map of the target pedestrian, the target pedestrian is identified.

Further, the steps of performing pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence include:

Determining video images corresponding to consecutive detection and tracking periods in a video image sequence according to a preset number of video image frames corresponding to each detection and tracking period;

In the order from front to back, based on the video images corresponding to each of the detection and tracking periods, the target pedestrian in the video image sequence is detected, tracked, and pedestrian re-identified, and the corresponding one of each of the detection and tracking periods is determined. Tracking results of target pedestrians;

Determining the target pedestrian in the video image sequence according to the tracking result of the target pedestrian corresponding to the last detection and tracking cycle.

Further, based on the video image corresponding to each of the detection and tracking periods, detecting, tracking, and re-identifying a target pedestrian in the video image sequence, and determining a target pedestrian corresponding to each of the detection and tracking periods. Steps to track results, including:

Do the following for each of the detection tracking cycles:

Tracking a predetermined target pedestrian in the video image sequence based on a tracking frame image corresponding to the current detection tracking period, and determining a tracking result corresponding to the current detection tracking period;

Performing pedestrian detection on a detection frame image of a next detection tracking period relative to the current detection tracking period to determine a detection result;

Perform pedestrian re-identification according to the tracking result and the detection result, and determine a target pedestrian to be tracked in the next detection and tracking cycle;

Wherein, the detection frame image is a first frame video image in a preset number of video image sequences corresponding to each detection tracking cycle, and the tracking frame image is a first frame video image in a video image sequence corresponding to each detection and tracking cycle. Video images other than video images.

Further, the predetermined target pedestrian is determined in at least one of the following ways:

If the current detection and tracking period is the first detection and tracking period, it is determined according to the detection result of the detection frame image of the detection and tracking period;

If the current detection tracking period is not the first detection tracking period, it is determined according to the tracking result corresponding to the previous detection tracking period with respect to the current detection tracking period and the detection result of the detection frame image of the current detection tracking period.

Further, the tracking result includes at least: a target pedestrian to be tracked in a normal tracking state, a tracker of the target pedestrian, and a target pedestrian to be tracked in a tracking loss state;

The detection result includes: a detection frame of a currently detected target pedestrian;

The step of re-identifying a pedestrian based on the tracking result and the detection result and determining a target pedestrian to be tracked in the next detection and tracking cycle includes:

Matching the area of the target pedestrian to be tracked in the normal tracking state with the currently detected target pedestrian to determine the target pedestrian pair;

For each of the target pedestrian pairs, perform pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked;

Performing pedestrian re-identification on each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the target pedestrians to be tracked in a tracking loss state to update the information of the target pedestrians to be tracked;

For a target pedestrian that needs to be tracked in a normal tracking state that does not constitute a target pedestrian pair, set it to a lost state.

Further, the step of performing pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked includes:

Determine the similarity between the two target pedestrians according to the pedestrian characteristics of the two target pedestrians included in the target pedestrian pair, wherein the two target pedestrians include: the target pedestrian to be tracked and the currently detected Target pedestrian

If the two target pedestrians are determined to be the same person according to the similarity, updating a tracker of the target pedestrian to be tracked through a detection frame of the currently detected target pedestrian;

If it is determined that the two target pedestrians are not the same person according to the similarity, setting the target pedestrian to be tracked included in the target pedestrian pair to a lost state, and deleting the tracker of the target pedestrian to be tracked, and Adding the currently detected target pedestrian to the target pedestrian to be tracked;

Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology.

Further, the currently detected target pedestrians that do not constitute a target pedestrian pair are re-identified with the target pedestrians to be tracked in a tracking loss state to update the information of the target pedestrians to be tracked. Steps, including:

Determining the pairwise similarity between each of the currently detected target pedestrians that do not constitute a target pedestrian pair and each target pedestrian that needs to be tracked in a tracking loss state according to the pedestrian characteristics obtained in advance;

If it is determined according to the pairwise similarity that the corresponding currently detected target pedestrian that does not constitute a target pedestrian pair is the same person as the target pedestrian in the tracking loss state, the demand in the tracking loss state is determined. The tracked target pedestrian is restored to a normal tracking state, and a tracker of the tracked target pedestrian to be restored to a normal tracking state is created by using a detection frame of the currently detected target pedestrian;

If it is determined according to the pairwise similarity that the currently detected target pedestrian is not a target pedestrian to be tracked in a tracking loss state, adding the currently detected target pedestrian to the target pedestrian to be tracked;

Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology.

Further, after the step of adding the currently detected target pedestrian to the target pedestrian to be tracked, the method further includes: if the number of times that the currently detected target pedestrian is detected is less than a preset number, then Remove from target pedestrians to be tracked.

Further, in the step of performing pedestrian detection and tracking on the video image sequence, the target pedestrians in the video image sequence are detected and tracked in cycles.

Further, in the step of performing pedestrian detection and tracking on the video image sequence, target pedestrian detection is performed through the first frame of video image in each detection and tracking cycle, and pedestrian characteristics of the target pedestrian are extracted; through each detection tracking cycle The second and subsequent frames of video images are used to track the detected target pedestrian.

Further, for two consecutive detection and tracking periods, the target pedestrian tracking result in the previous detection and tracking period is updated by detecting the target pedestrian from the first frame of the video image of the subsequent detection and tracking period.

Further, the step of determining a silhouette image sequence of the target pedestrian includes:

Determining the image area of the target pedestrian in the detection frame image according to the detection frame of the target pedestrian, or determining the image area of the target pedestrian in the tracking frame image according to the tracking frame of the target pedestrian;

Input the image area of the target pedestrian in the detection frame image or the tracking frame image to the pre-trained network model to obtain the silhouette image of the target pedestrian;

The silhouette images corresponding to the same target pedestrian are arranged in the sequence of the video image frames to which the silhouette image belongs, to obtain the silhouette image sequence corresponding to the same target pedestrian.

Further, the step of identifying the target pedestrian based on the gait energy map of the target pedestrian includes:

Identify the gait energy map of the target pedestrian through a pre-trained neural network model to determine the gait characteristics of the target pedestrian;

Match the gait characteristics of the target pedestrian with the gait characteristics in the preset database, and determine the identity information of the user who matches the gait characteristics of the target pedestrian as the identity information of the target user.

According to another aspect of the present disclosure, a gait-based identity recognition device is provided, including:

A target pedestrian determination module, configured to perform pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence;

A silhouette image sequence determination module, configured to determine a silhouette image sequence of the target pedestrian;

A gait energy map determination module, configured to obtain a gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian;

An identity recognition module is configured to identify the target pedestrian based on the gait energy map of the target pedestrian.

Further, when performing pedestrian detection, tracking, and pedestrian recognition on a video image sequence to determine a target pedestrian in the video image sequence, the target pedestrian determination module further includes:

The detection tracking period determination submodule is configured to determine video images corresponding to consecutive detection and tracking periods in a video image sequence according to a preset number of video image frames corresponding to each detection and tracking period;

The periodic target pedestrian determination sub-module is configured to detect, track and re-identify the target pedestrian in the video image sequence based on the video images corresponding to each of the detection and tracking periods in the order from front to back. A tracking result of a target pedestrian corresponding to each of the detection and tracking periods;

The tracking result determination submodule is configured to determine a target pedestrian in the video image sequence according to a tracking result of a target pedestrian corresponding to the last detection and tracking period.

Further, based on the video image corresponding to each of the detection and tracking periods, the target pedestrian in the video image sequence is detected, tracked and re-identified, and the tracking of the target pedestrian corresponding to each of the detection and tracking periods is determined. In the event, the periodic target pedestrian determination sub-module further includes:

A tracking unit is configured to track, for each of the detection and tracking cycles, a predetermined target pedestrian in the video image sequence based on a tracking frame image corresponding to the current detection and tracking cycle, and determine a tracking corresponding to the current detection and tracking cycle. result;

A detection unit, configured to perform a pedestrian detection on a detection frame image of a next detection and tracking period relative to the current detection and tracking period for each of the detection and tracking periods to determine a detection result;

A pedestrian re-identification unit, configured to perform pedestrian re-identification according to the tracking result and the detection result for each of the detection and tracking cycles, and determine a target pedestrian to be tracked in the next detection and tracking cycle;

Wherein, the detection frame image is a first frame video image in a preset number of video image sequences corresponding to each detection tracking cycle, and the tracking frame image is a first frame video image in a video image sequence corresponding to each detection and tracking cycle. Video images other than video images.

Further, the predetermined target pedestrian is determined in at least one of the following ways:

If the current detection and tracking period is the first detection and tracking period, it is determined according to the detection result of the detection frame image of the detection and tracking period;

If the current detection tracking period is not the first detection tracking period, it is determined according to a tracking result corresponding to a previous detection tracking period with respect to the current detection tracking period and a detection result of a detection frame image of the current detection tracking period.

Further, the tracking result includes at least: a target pedestrian to be tracked in a normal tracking state, a tracker of the target pedestrian, and a target pedestrian to be tracked in a tracking loss state;

The detection result includes: a detection frame of a currently detected target pedestrian;

When pedestrian re-identification is performed according to the tracking result and the detection result, and a target pedestrian to be tracked in the next detection and tracking cycle is determined, the pedestrian re-identification unit is further configured to:

Matching the area of the target pedestrian to be tracked in the normal tracking state with the currently detected target pedestrian to determine the target pedestrian pair;

For each of the target pedestrian pairs, performing pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked;

Performing pedestrian re-identification on each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the target pedestrians to be tracked in a tracking loss state to update the information of the target pedestrians to be tracked;

For a target pedestrian that needs to be tracked in a normal tracking state that does not constitute a target pedestrian pair, set it to a lost state.

Further, the pedestrian re-identification unit is configured to perform pedestrian re-identification in the following manner according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian, and update the target to be tracked Pedestrian information:

Determine the similarity between the two target pedestrians according to the pedestrian characteristics of the two target pedestrians included in the target pedestrian pair, wherein the two target pedestrians include: the target pedestrian to be tracked and the currently detected Target pedestrian

If the two target pedestrians are determined to be the same person according to the similarity, updating a tracker of the target pedestrian to be tracked through a detection frame of the currently detected target pedestrian;

If it is determined that the two target pedestrians are not the same person according to the similarity, setting the target pedestrian to be tracked included in the target pedestrian pair to a lost state, and deleting the tracker of the target pedestrian to be tracked, and Adding the currently detected target pedestrian to the target pedestrian to be tracked;

Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology.

Further, the pedestrian re-identification unit is configured to perform pedestrian re-identification on each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the target pedestrians that need to be tracked in a tracking loss state, and update the Information of target pedestrians to be tracked:

Determining the pairwise similarity between each of the currently detected target pedestrians that do not constitute a target pedestrian pair and each target pedestrian that needs to be tracked in a tracking loss state according to the pedestrian characteristics obtained in advance;

If it is determined according to the pairwise similarity that the corresponding currently detected target pedestrian that does not constitute a target pedestrian pair is the same person as the target pedestrian in the tracking loss state, the demand in the tracking loss state is determined. The tracked target pedestrian is restored to a normal tracking state, and a tracker of the tracked target pedestrian to be restored to a normal tracking state is created by using a detection frame of the currently detected target pedestrian;

If it is determined according to the pairwise similarity that the currently detected target pedestrian is not a target pedestrian to be tracked in a tracking loss state, adding the currently detected target pedestrian to the target pedestrian to be tracked;

Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology.

Further, after the currently detected target pedestrian is added to the target pedestrians to be tracked, if the number of times that the currently detected target pedestrians are detected is less than a preset number, it is removed from the targets to be tracked. Removed from pedestrians.

Further, the target pedestrian determination module is configured to perform pedestrian detection and tracking on a video image sequence in a periodical manner.

Further, the target pedestrian determination module performs target pedestrian detection through the first frame video image in each detection and tracking cycle, and extracts pedestrian characteristics of the target pedestrian; through the second and subsequent frame video images in each detection and tracking cycle, Detected target pedestrians are tracked.

Further, for two consecutive detection and tracking cycles, the target pedestrian determination module updates the target pedestrian tracking result in the previous detection and tracking cycle by detecting the target pedestrian from the first frame of the video image in the subsequent detection and tracking cycle. .

Further, the silhouette image sequence determination module is configured to determine the silhouette image sequence of the target pedestrian in the following manner:

Determining the image area of the target pedestrian in the detection frame image according to the detection frame of the target pedestrian, or determining the image area of the target pedestrian in the tracking frame image according to the tracking frame of the target pedestrian;

Input the image area of the target pedestrian in the detection frame image or the tracking frame image to a pre-trained network model to obtain the silhouette image of the target pedestrian;

The silhouette images corresponding to the same target pedestrian are arranged in the sequence of the video image frames to which the silhouette image belongs, to obtain the silhouette image sequence corresponding to the same target pedestrian.

Further, the identity recognition module is used for identity recognition in the following ways:

Identify the gait energy map of the target pedestrian through a pre-trained neural network model to determine the gait characteristics of the target pedestrian;

Match the gait characteristics of the target pedestrian with the gait characteristics in the preset database, and determine the identity information of the user who matches the gait characteristics of the target pedestrian as the identity information of the target user.

According to still another aspect of the present disclosure, there is provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor implements the computer program when the processor executes the computer program. The gait-based identification method described above.

According to still another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the steps of the gait-based identification method.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some of the present application. For those of ordinary skill in the art, other embodiments may be obtained based on these drawings without paying creative labor.

FIG. 1 is a flowchart of a gait-based identification method according to the first embodiment of the present application; FIG.

2 is a flowchart of steps of detecting and tracking and re-identifying a pedestrian in a gait-based identification method according to a second embodiment of the present application;

FIG. 3 is one of the schematic structural diagrams of a gait-based identification device according to Embodiment 3 of the present application; FIG.

4 is a second schematic structural diagram of a gait-based identification device according to Embodiment 3 of the present application;

FIG. 5 is a schematic structural diagram of a sub-module of a gait-based identification device according to Embodiment 3 of the present application.

detailed description

In the following, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

Example one

A gait-based identification method disclosed in this embodiment is shown in FIG. 1. The method includes steps 110 to 140.

Step 110: Perform pedestrian detection, tracking, and pedestrian recognition on the video image sequence to determine a target pedestrian in the video image sequence.

In the specific application process, the input video image sequence may be a real-time camera surveillance video sequence, or a video in a video stream that is played. A video image sequence consists of a sequence of frame-by-frame video images. Generally, the video images in the video image sequence include images of multiple target pedestrians.

For the input video image sequence, in order to improve the efficiency of pedestrian recognition, it is preferable to detect and track the target pedestrian in the video image sequence in cycles.

For example, every 20 frames of video images are used as a detection tracking cycle. Target pedestrian detection is performed through the first frame of video images in each detection tracking cycle, and the pedestrian characteristics of the target pedestrian are extracted. Then, subsequent frames of video in each detection tracking cycle are used. Image to track detected pedestrians.

Further, for two consecutive detection and tracking periods, the target pedestrian tracking result in the previous detection and tracking period is updated by detecting the target pedestrian from the first frame of the video image of the subsequent detection and tracking period.

For example, for a target pedestrian detected in both detection and tracking cycles, tracking continues in the next detection and tracking cycle; for those that were lost during the previous detection and tracking cycle, the first frame of video in the next detection and tracking cycle The target pedestrians re-detected in the image will continue to be tracked in the next detection and tracking cycle; for the target pedestrians not detected in the previous detection and tracking cycle and detected in the later detection and tracking cycle, the next detection and tracking will be performed. Tracking continues in the cycle; target pedestrians detected in the previous detection and tracking cycle but not detected in the later detection and tracking cycle will be temporarily identified as lost target pedestrians, and will be judged in the subsequent detection and tracking cycle The lost target pedestrian was detected again.

Until the video image in the last detection and tracking cycle of the input video image sequence is processed, each target pedestrian detected in the video image sequence and the area where the target pedestrian is in the frame video image can be obtained. Information and pedestrian characteristics, and can determine the final tracking status of each target pedestrian, that is, normal tracking status or missing status.

Step 120: Determine a silhouette image sequence of the target pedestrian.

In the specific implementation of the present application, when pedestrian detection is performed on a detection frame image, a unique identifier is assigned to each detected target pedestrian, and the same target pedestrian detected by different detection frame images uses the same identifier. Therefore, the target pedestrians detected in the last detection and tracking cycle also have unique identifiers, and the target pedestrians who have confirmed tracking loss in the last detection and tracking cycle also have unique identifiers.

When pedestrian detection is performed on a detection frame image in each detection and tracking cycle, a detection frame for each target pedestrian detected is also determined, and the detection frame is used to indicate an image area where the target pedestrian is located in the detection frame image. When pedestrian detection is performed on the detected target pedestrian based on the tracking frame image of the detection tracking period, the tracker is initialized through the detection frame, and the area where the target pedestrian is located in each tracking frame image is further tracked by the tracker.

Then, the image area of the target pedestrian in the detection frame image is further determined according to the detection frame of a target pedestrian, or the image area of the target pedestrian in the tracking frame image is further determined according to the tracking frame of the target pedestrian; The image area in the detection frame image or in the tracking frame image is input to a pre-trained U-shaped network (Unity Networking, UNet for short) or a U-shaped network, a timing-based network model, and a silhouette image of the target pedestrian can be obtained .

Through this method, a silhouette image of each target pedestrian based on different video image frames can be determined.

Finally, the silhouette images corresponding to the same target pedestrian are arranged according to the sequence of the video image frames to which the silhouette image belongs to obtain the silhouette image sequence corresponding to the same target pedestrian.

In an embodiment of the present application, when training the U-shaped network, the training data is color image data, and the labels are corresponding binary images. The prepared training data set is sent to the U-shaped network, and then forward calculation is performed. Loss function method, and error back propagation to modify the weights, optimize the network, repeatedly train the network to minimize the loss, and finally get an accurate U-shaped network for pedestrian segmentation.

When extracting silhouette images, because the size of the detection frame or tracking frame of each target pedestrian is different, and the size of the corresponding detection frame and tracking frame of the same target pedestrian in different video image frames may also be different, Before inputting the image of the image area determined by the detection frame or tracking frame into the U-shaped network, you first need to normalize the image corresponding to the input image area and normalize the input image to a specified size, such as 473 × 473 × 3 size image.

Step 130: Obtain a gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian.

Generally, 20 to 25 different pose sequences can well characterize the gait characteristics of a person. Therefore, the threshold Th = 20 of the silhouette image sequence is set in the specific implementation of this application. In the process of detecting and tracking the video image sequence, when the number of silhouette images included in the determined silhouette image sequence of a target pedestrian is greater than the threshold value Th of the silhouette image sequence, gait recognition can be further performed according to the silhouette image sequence .

For example, the gait energy map of the target pedestrian is determined by the following formula:

其中，ID为目标行人的标识，x和y是剪影图像坐标，

表示标识为ID的目标行人的剪影图像序列中第t个剪影图像的(x，y)点的像素值，Count _ID表示标识为ID的目标行人的剪影图像序列中Th个剪影图像的(x，y)点的像素值累加和；GEI(x，y) _ID表示标识为ID的目标行人的剪影图像序列(x，y)点对应的步态能量图。

Among them, ID is the identification of the target pedestrian, x and y are the silhouette image coordinates,

Represents the pixel value of the (x, y) point of the t-th silhouette image in the silhouette image sequence of the target pedestrian identified as ID, and Count _ID represents the (x, The pixel values of y) are accumulated and sum; GEI (x, y) _ID represents the gait energy map corresponding to the (x, y) points of the silhouette image sequence of the target pedestrian identified as ID.

Step 140: Identify the target pedestrian based on the gait energy map of the target pedestrian.

In some embodiments of the present application, the gait energy map of the target pedestrian may be identified through a pre-trained neural network model to determine the gait characteristics of the target pedestrian.

Then, the gait feature of the target pedestrian is matched with the gait feature in a preset database, and the identity information of the user matching the gait feature of the target pedestrian is determined as the identity information of the target user. So far, the target user is identified based on the gait information.

In some embodiments of the present application, a convolutional neural network including 1 convolutional layer, 2 residual blocks, and 2 fully connected layers may be used to extract gait features. During the training phase, gait energy maps of several pedestrians are selected as training samples, where the sample data is the user's gait energy map and the sample labels are user categories.

In the training process, the training samples are first sent to a convolutional neural network, and then a conv operation is performed first, and then the convolution results are down-sampled. After that, two residual blocks and two fully connected The layer performs forward transfer to find the error loss, and finally, performs the back propagation error training network. The gait energy map used during training may be a gait energy map in a preset database or a gait energy map in a public database. In the recognition process, the feature vector of a feature output layer of the convolutional neural network can be used as the gait feature of the input gait energy map.

The gait-based identification method disclosed in the embodiments of the present application determines a target pedestrian in the video image sequence by performing pedestrian detection, tracking, and pedestrian re-identification on the video image sequence; determining a silhouette image sequence of the target pedestrian; Gait energy maps of the target pedestrians are obtained based on the silhouette image sequences of the target pedestrians respectively; the target pedestrians are identified based on the gait energy maps of the target pedestrians, which solves the problem of The problem of low accuracy of gait recognition. In the embodiment of the present application, the identification of the same and different target pedestrians in the video image sequence is performed by combining pedestrian detection, tracking, and pedestrian re-identification technologies, which improves the accuracy of target pedestrian recognition in the video image sequence and helps to improve the multi-person scene The accuracy of gait-based identification.

Example two

Based on the foregoing embodiments, a specific implementation of a gait-based identity recognition method disclosed in this embodiment.

In an embodiment of the present application, the steps of performing pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence include: pre-corresponding to a video image frame corresponding to each detection and tracking cycle. Set the number to determine the video images corresponding to the consecutive detection and tracking periods in the video image sequence; in the order from front to back, based on the video images corresponding to each of the detection and tracking periods, target pedestrians in the video image sequence Performing detection, tracking, and pedestrian re-identification to determine the tracking result of the target pedestrian corresponding to each of the detection and tracking cycles; and determining the target in the video image sequence according to the tracking result of the target pedestrian corresponding to the last detection and tracking cycle pedestrian.

Taking the identification disclosed in this application in the field of video surveillance as an example, the input video image sequence includes several frames of video images, the time interval between shooting each frame of video images is short, and the content does not change much. Video images are all detected by pedestrians, and pedestrian features are extracted, which will cause pedestrian data redundancy and reduce the efficiency of identity recognition. Therefore, this application performs pedestrian detection based on the detection and tracking cycle to improve the efficiency of identity recognition and avoid pedestrian features. redundancy.

For example, each 20-frame video image corresponds to one detection and tracking period, and pedestrian detection is performed only on the first frame of video images in each detection and tracking period to determine a target pedestrian included therein.

Further, in order to avoid that when the pedestrian detection is performed only on the first frame of the video image of each detection and tracking cycle, the target pedestrian is lost due to the rapid movement of a detected target pedestrian. This application is based on each detection The second and subsequent video image frames in the tracking cycle track the detected target pedestrian.

As shown in FIG. 2, optionally, based on the video image corresponding to each detection and tracking cycle, target pedestrians in the video image sequence are detected, tracked, and pedestrian re-identified to determine the tracking of the target pedestrian corresponding to each detection and tracking cycle The step of the result includes performing the operations of step 210 to step 240 for each of the detection tracking cycles.

Step 210: Based on the tracking frame image corresponding to the current detection tracking period, track a predetermined target pedestrian in the video image sequence to determine a tracking result corresponding to the current detection tracking period.

In the embodiment of the present application, the detection frame image is a first frame video image in a preset number of video image sequences corresponding to each detection tracking period, and the tracking frame image is a video image sequence corresponding to each detection tracking period. Video image except the first frame video image.

In this embodiment, a video image sequence includes three detection and tracking periods, and each detection and tracking period corresponds to 20 frames of video images as an example. The detailed description is based on the target pedestrian in the video image sequence based on the video image corresponding to each detection and tracking period. Detection, tracking, and pedestrian re-identification, and a specific implementation scheme for determining a tracking result of a target pedestrian corresponding to each detection and tracking cycle.

In specific implementation, the predetermined target pedestrian is determined by at least one of the following methods: if the current detection and tracking period is the first detection and tracking period, it is determined according to the detection result of the detection frame image of the detection and tracking period; if the current detection and tracking period is not the first The detection tracking period is determined according to the tracking result corresponding to the previous detection tracking period relative to the current detection tracking period and the detection result of the detection frame image of the current detection tracking period.

For the detection and tracking period being the first detection and tracking period, the target pedestrian is determined according to the detection result of the detection frame image of the detection and tracking period.

First, pedestrian detection is performed on a detection frame image corresponding to a first detection and tracking period, and a target pedestrian included in the detection frame image is determined. In specific implementation, in order to identify the target pedestrian detected in the video image sequence, a unique identifier is assigned to the target pedestrian detected in the detection frame image. For example, if three target pedestrians are detected in the detection frame image of the first detection tracking cycle, the identifiers of the three target pedestrians can be set to 1, 2, and 3 respectively. In the embodiment of the present application, the target pedestrian detected in the detection frame image is determined as the target pedestrian to be tracked.

Further, in order to perform target pedestrian comparison and generate a silhouette image sequence of the target pedestrian in the subsequent implementation, when the present application is implemented, a target information queue needs to be created to store the detected target pedestrian information, for example, to store the target pedestrian. Information such as the identity of the pedestrian, pedestrian characteristics and silhouette images corresponding to the identity.

For example, after pedestrian detection is performed on the detection frame image in the first detection and tracking cycle, for the determined target pedestrian 1, according to the image area of the target pedestrian 1 in the detection frame image, that is, according to the detection frame of the target pedestrian 1 The corresponding image area determines the image of the target pedestrian 1. Then, the image of the target pedestrian 1 is input to a pre-trained pedestrian re-identification feature extraction model, the pedestrian feature of the target pedestrian 1 is extracted based on the pedestrian re-identification technology, and the extracted pedestrian feature is stored in the target information queue corresponding to the target pedestrian 1 Information.

Further, the image of the target pedestrian 1 can also be input to a pre-trained U-shaped network, the silhouette image of the first frame of the video image in the video image sequence corresponding to the target pedestrian 1 is determined, and the determined silhouette image is stored in the target information queue. In the information corresponding to the target pedestrian 1.

According to the above method, the detected pedestrian characteristics of the target pedestrian 1, target pedestrian 2, and target pedestrian 3 and the silhouette image of the first frame video image in the corresponding video image sequence can be obtained and stored in the target information queue.

Step 220: Perform a pedestrian detection on a detection frame image of a next detection tracking period relative to the current detection tracking period to determine a detection result.

Then, based on the tracking frame image corresponding to the detection tracking period, a predetermined target pedestrian in the video image sequence is tracked to determine a tracking result corresponding to the detection tracking period.

For the first detection and tracking cycle, based on the 2nd to 20th tracking frame images, 3 target pedestrians detected by the first detection frame image are tracked. When tracking the target pedestrian, the tracker is first initialized according to the detection frame, and then the target pedestrian in each tracking frame image is tracked by the tracker. For a specific method for tracking the target, refer to the prior art, which is not described in this embodiment.

When tracking the target pedestrian, the area where the tracker of each target pedestrian in each tracking frame image is located, that is, the image area where each target pedestrian is located in the corresponding tracking frame image. Further, according to an image region where each target pedestrian is located in the corresponding tracking frame image, an image of each target pedestrian in the corresponding tracking frame image may be determined.

Then, by inputting the image of each target pedestrian in the corresponding tracking frame image into a pre-trained U-shaped network, the silhouette images of the second frame video to the 20th frame video image in the corresponding video image sequence of each target pedestrian are determined. And stored in the information queue of each target pedestrian.

In the specific implementation process, if a detected pedestrian such as target pedestrian 2 is out of the video surveillance range, the tracker will not be able to track target pedestrian 2 in the corresponding tracking frame image. At this time, the target pedestrian is determined 2 is the tracking loss state. In some embodiments of the present application, a tracking loss queue may be created to store the information of the target pedestrian who lost track.

For example, when the tracking loss of the target pedestrian 2 is determined, the information of the target pedestrian 2 is transferred from the target information queue to the tracking loss queue. At the same time, the tracker of the target pedestrian 2 is deleted. In the current detection and tracking cycle, the target new pedestrian 2 will no longer be tracked. Target pedestrians tracked in the detection tracking frame will continue to be tracked.

During the tracking process, the confidence scores of the tracking results of all target pedestrians to be tracked in the current tracking frame are scored. If there are certain target pedestrians whose confidence scores are lower than a preset confidence threshold, the target pedestrians are determined. Tracking is lost.

当这个评分大于预设的置信度阈值时，认为是高置信度的跟踪结果，其中，F _max，F _min分别表示反映响应图中的像素值最大值和最小值；w，h表示反映响应图的水平垂直像素数量；F _w，h表示反映响应图中每一个位置的像素值。 In specific implementation, the tracking algorithm needs to be able to reflect the reliability of each tracking result, otherwise it may cause the pedestrian target to be lost or followed. The confidence index is the average peak-to correlation energy (APCE), which reflects the degree of fluctuation of the response graph and the confidence level of the detection target. In an embodiment of the present application, the calculation formula of the confidence score is:

When this score is greater than the preset confidence threshold, it is considered to be a tracking result with high confidence, where F _max and F _min respectively represent the maximum and minimum values of the pixel values in the response map; w and h represent the response map The number of horizontal and vertical pixels; F _{w, h} represents the pixel value reflecting each position in the response map.

Based on the 19 tracking frame images of the first detection tracking cycle, after tracking the 3 target pedestrians that first appeared in the video image sequence, the tracking results of the 3 target pedestrians will be determined. The tracking results include: a target pedestrian to be tracked in a normal tracking state, a tracker, pedestrian characteristics, at least one frame of a silhouette image, and a target pedestrian to be tracked in a tracking loss state, pedestrian characteristics, at least one Frame silhouette image.

Step 230: Perform pedestrian re-identification according to the above tracking result and detection result, and determine a target pedestrian to be tracked in the next detection and tracking cycle.

Next, pedestrian detection is performed on the detection frame image of the second detection tracking cycle to determine the detection result.

For a specific implementation manner of performing pedestrian detection on a video image frame, refer to the prior art, which is not repeated in the embodiments of the present application. By performing pedestrian detection on the detection frame image of the second detection and tracking cycle, the determined detection result includes: the target pedestrian included in the detection frame image, and a detection frame corresponding to each target pedestrian.

Further, in the same manner as in the first detection and tracking cycle, it is also possible to determine pedestrian characteristics and silhouette images corresponding to the second detection frame image (that is, the 21st frame video image in the video image sequence) of each target pedestrian. Assume that pedestrian detection is performed on the detection frame image of the second detection and tracking cycle, and it is determined that three target pedestrians are included, and the detected three target pedestrians are respectively assigned different identifiers from the target pedestrians detected in the previous detection and tracking cycle. For example, three target pedestrians detected in the detection frame image of the second detection tracking cycle are identified as target pedestrians -1, -2, and -3, respectively.

After that, the pedestrian is re-identified according to the tracking result of the first detection and tracking cycle and the detection result of the detection frame image in the second detection and tracking cycle, and the target pedestrian to be tracked in the second detection and tracking cycle is determined.

In some embodiments of the present application, the tracking result includes at least: a target pedestrian to be tracked in a normal tracking state and a tracker of the target pedestrian, and a target pedestrian to be tracked in a tracking loss state; the detection result includes: currently detected Detection frame for the target pedestrian.

In specific implementation, the step of re-identifying pedestrians based on the tracking results and detection results to determine the target pedestrians to be tracked in the next detection and tracking cycle includes the following: the target pedestrians in the normal tracking state that need to be tracked and the currently detected targets Pedestrians are matched to determine the target pedestrian pair; for each target pedestrian pair, pedestrian re-identification is performed based on the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the required pedestrians. Information of the tracked target pedestrians; re-identifying each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the tracked target pedestrians in a tracking loss state to update the information of the tracked target pedestrians; For a target pedestrian that needs to be tracked in a normal tracking state that does not constitute a target pedestrian pair, set it to a lost state.

In some embodiments of the present application, matching the area where the target pedestrian to be tracked in the normal tracking state with the currently detected target pedestrian includes determining the tracking of the target pedestrian to be tracked in the normal tracking state. The size of the overlap between the detector and the detection frame of the currently detected target pedestrian, for example, determines the ratio of the overlap between the area where the tracker is located and the area where the detection frame is located to the minimum area. It can be known from the tracking results of the first detection and tracking cycle that the target pedestrians in the normal tracking state include target pedestrians 1 and 3, and from the detection results of the second detection and tracking cycle that the currently detected target pedestrians include target pedestrians. -1. Target pedestrian-2 and target pedestrian-3. First, according to the size of the overlap between the trackers of target pedestrian 1 and target pedestrian 3 and the detection frame of target pedestrian-1, target pedestrian-2 and target pedestrian-3, respectively , Determine the target pedestrian pair. Assuming that the size of the overlapping portion of the tracker of the target pedestrian 1 and the detection frame of the target pedestrian-1 satisfies a preset condition, it is determined that the target pedestrian 1 and the target pedestrian-1 are target pedestrian pairs. Assuming that the tracker of target pedestrian 3, the target pedestrian-1, target pedestrian-2, and the overlapping portion of the detection frame of target pedestrian-3 do not meet the preset conditions, determine target pedestrian 3, target pedestrian-2, Target pedestrian-3 does not constitute a target pedestrian pair.

After matching the target pedestrians based on their areas, the target pedestrians to be tracked and the currently detected target pedestrians with high overlap in their areas will be matched as target pedestrian pairs. Further, in order to improve the judgment accuracy rate of the same pedestrian, it is also necessary to re-identify the two target pedestrians in the target pedestrian pair based on the pedestrian characteristics.

In some embodiments of the present application, the step of re-identifying the pedestrian according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked includes: According to the pedestrian characteristics of two target pedestrians included in the target pedestrian pair, determine the similarity between the two target pedestrians, where the two target pedestrians include: the target pedestrian to be tracked and the currently detected target pedestrian; if If the two target pedestrians are determined to be the same person based on the similarity, the tracker of the target pedestrians to be tracked is updated through the detection frame of the currently detected target pedestrians; if the two target pedestrians are not the same person based on the similarity, The target pedestrian to be tracked included in the target pedestrian pair is set to a lost state, and the tracker of the target pedestrian to be tracked is deleted, and the currently detected target pedestrian is added to the target pedestrian to be tracked; wherein, Pedestrian characteristics are obtained through pedestrian re-identification technology.

For example, for the target pedestrian pair to be tracked, the target pedestrian 1 and the currently detected target pedestrian-1, further determine the target pedestrian 1 and the target by calculating the similarity distance between the pedestrian characteristics of the target pedestrian 1 and the target pedestrian-1. Similarity of Pedestrian-1. If the similarity satisfies the preset similarity condition, it is determined that the target pedestrian 1 and the target pedestrian-1 are the same person, and the tracker of the target pedestrian 1 to be tracked is updated through the detection frame of the currently detected target pedestrian-1, so that The target pedestrian 1 continues to be tracked during the two detection and tracking cycles. If the similarity does not satisfy the preset similarity conditions, it is determined that the target pedestrian 1 and the target pedestrian-1 are not the same person, the target pedestrian 1 who believes that the tracking needs to be out of the video surveillance scope, sets the target pedestrian 1 to the missing state, and deletes the tracking required Tracker for target pedestrian 1. At the same time, the currently detected target pedestrian-1 is added to the target pedestrian to be tracked, and information such as the identification, pedestrian characteristics, and silhouette image of the target pedestrian-1 is stored in the target information queue. Preferably, the pedestrian characteristics in the embodiments of the present application are obtained through pedestrian re-identification technology.

In some other examples of the present application, after matching the target pedestrians based on their location, each currently detected target pedestrian that does not constitute a target pedestrian pair is pedestrian-revised with the target pedestrian to be tracked in a tracking loss state. The step of identifying and updating the information of the target pedestrians to be tracked includes: determining, according to the pedestrian characteristics obtained in advance, each target pedestrian currently detected that does not constitute a target pedestrian pair and each target to be tracked in a tracking loss state Pairwise similarity between pedestrians; if the corresponding currently detected target pedestrian that does not constitute a target pedestrian pair is determined to be the same person as the target pedestrian that needs to be tracked in the state of tracking loss based on the pairwise similarity, it will be in tracking loss The state of the target pedestrian to be tracked is restored to the normal tracking state, and a tracker of the target pedestrian to be tracked is restored to the normal tracking state through the detection frame of the currently detected target pedestrian; if the current detection is determined according to the pairwise similarity The target pedestrian is not the target pedestrian to be tracked in the state of tracking loss, that is, the current Pedestrian detected target and the target track in the track pedestrians have lost state does not match, then the current detected target complementary to the target pedestrian pedestrians have tracked; wherein pedestrian pedestrians characterized by a weight acquired recognition technology.

For example, for the currently detected target pedestrians -2 and -3, they are further re-identified with the target pedestrian 2 in the tracking loss queue. First, according to the pedestrian characteristics of target pedestrian 2 and the pedestrian characteristics of target pedestrians 2 and -3, determine the similarity between target pedestrian 2 and target pedestrian-2, and the similarity between target pedestrian 2 and target pedestrian-3, respectively.

Then, according to the determined similarity, it is further determined whether the currently detected target pedestrian is a target pedestrian who has been tracked and lost.

For example, if the similarity between target pedestrian 2 and target pedestrian-2 satisfies a preset similarity condition, it is determined that target pedestrian 2 and target pedestrian-2 are the same target pedestrian. At this time, the target to be tracked will be in a state of tracking loss. Pedestrian 2 returns to the normal tracking state, and a tracker of target pedestrian 2 is created by the detection frame of the currently detected target pedestrian.

For another example, if the similarity between target pedestrian 2 and target pedestrian-3 does not meet the preset similarity condition, it is determined that target pedestrian 2 and target pedestrian-3 are not the same target pedestrian. At this time, the currently detected target pedestrian is added to Among the target pedestrians to be tracked, the target pedestrian-3's identification, pedestrian characteristics, and silhouette images are saved to the target information queue. Among them, pedestrian characteristics are obtained through pedestrian re-identification technology.

In some embodiments of the present application, there may be zero, one, or more target pedestrians to be tracked in a tracking loss state. When matching the areas of the target pedestrians to be tracked and the currently detected target pedestrians, if all the target pedestrians to be tracked and the currently detected target pedestrians are matched as target pedestrian pairs, it can be determined that there is currently no Target pedestrians to be tracked in a lost state. Therefore, after matching the target pedestrians based on their area, there is no need to perform pedestrian re-identification of each currently detected target pedestrian that does not constitute a target pedestrian pair and the target pedestrians to be tracked in a tracking loss state to update Steps to track the information of the target pedestrian.

In other embodiments of the present application, after the target pedestrian is matched based on the area in which it is located, the target pedestrian 3 to be tracked in a normal tracking state that does not constitute the target pedestrian pair is set to a lost state. The tracker of the target pedestrian 3 is deleted, and the information of the target pedestrian 3 is transferred to the tracking loss queue.

In other embodiments of the present application, after adding the currently detected target pedestrian to the target pedestrian to be tracked, the method further includes: if the number of times that the currently detected target pedestrian is detected is less than a preset number , Remove it from the targeted pedestrians to track. For example, if the currently detected target pedestrian is detected only in a few detection and tracking cycles, the target pedestrian is considered to be a pedestrian who temporarily broke into the video surveillance range, or a tracking error has occurred, and the target pedestrian will not be allowed to do so. For identification. If the currently detected target pedestrian is continuously detected in a subsequent preset number of detection and tracking cycles, the target pedestrian is considered to be a pedestrian newly appearing in the video surveillance range, and it is continuously tracked.

In step 240, it is judged whether all the detection and tracking cycle processing ends, and if so, it ends; otherwise, go to step 210.

In the specific implementation of this application, for the continuous detection and tracking cycle, the above steps 210 to 230 are performed cyclically until the last detection and tracking cycle ends.

In the embodiment of the present application, the pedestrian re-recognition feature extraction model used to extract pedestrian features from the image of each target pedestrian in each frame image can be trained by the following methods.

First, build a network structure.

Because the image of the target pedestrian is usually small, and the target pedestrian image is not square, it is not suitable to directly use ImageNet with high resolution and rich details (the ImageNet project is a large visualization database for visual object recognition software research) pre-training The convolutional neural network model structure used in the embodiment of the present application includes: three layers of convolutional layers (the size of the convolution kernel is 3 × 3), six Inception modules, and two fully connected layers. As shown in Table 1, the structure of the Inception module, where Conv represents the convolution layer, Pooling represents the pooling layer, and each row in the table represents a branch.

Second, construct training samples.

In the prior art, the number of single data sets is small and the sample types are relatively single. In order to ensure that the Reid-CNN network obtained by training has a good generalization ability, multiple pedestrian re-identification data sets are selected for the network training data. , Such as CUHK03 data set, CUHK01 data set, PRID, VIPer, 3DPeS, iLIDS and Shinpuhkan, and then combine the pedestrian re-identification data set into a large data set as the network input, so that the network can learn more rich scenarios Feature information.

The sample data is an image, and the sample label is the identity or attribute of a pedestrian.

Finally, the convolutional neural network model is trained based on the generated training samples.

The embodiment of the present application improves the accuracy of identifying target pedestrians in a video image sequence by first performing pedestrian matching based on the area in which they are located, and then performing target pedestrian matching based on pedestrian characteristics to determine the same target pedestrian in different video image frames. Improve the accuracy of identity recognition based on gait.

By matching target pedestrians detected in subsequent video detection cycles with target pedestrians with missing tracking, it is possible to avoid inaccurate pedestrian recognition in the video image sequence caused by tracking errors.

While performing target pedestrian detection and tracking, the present application uses pedestrian re-identification technology to perform accurate pedestrian identification in the context of multiple pedestrians or across cameras, which is conducive to accurately extracting pedestrian silhouette image sequences. In addition, the present application performs adaptive gait feature extraction based on a convolutional neural network, which makes up for the drawback that it is difficult for traditional gait feature extraction methods to accurately extract representative gait features. The combination of the two well implements pedestrian identification based on gait under real-time camera surveillance conditions.

In some embodiments of the present application, the step of determining the silhouette image sequence of the target pedestrian is specifically: determining the silhouette image sequence of the target pedestrian according to the obtained silhouette image of each frame of the video image in the video image sequence of the target pedestrian, The method comprises: sorting the pre-obtained silhouette images of each target pedestrian according to the forward and backward positions of the video image frames corresponding to the silhouette images in the video image sequence to determine the silhouette image sequence of each target pedestrian; wherein each target pedestrian The silhouette image of is obtained by: acquiring the image of each target pedestrian in the detection frame image according to the detection frame of each target pedestrian determined during the pedestrian detection of the detection frame image; and when pedestrian tracking is performed on the tracking frame image The determined tracker of each target pedestrian obtains the images of the target pedestrians in the tracking frame images; the images of the target pedestrians in the video images of each frame are input to a pre-trained U-shaped network, respectively, and the identified The target pedestrian corresponds to a silhouette image of each frame of video image.

For example, after detecting and tracking the video image corresponding to the last detection and tracking cycle, the target pedestrians to be tracked and the target pedestrians who have been tracked to be lost in the normal tracking state are determined. Target pedestrians who have been lost to tracking are uniquely identified, and corresponding silhouette images are stored in either the target information queue or the tracking loss queue. By arranging all the silhouette images corresponding to the same identifier in the order of the video image frames to which the silhouette image belongs in the video image queue, the silhouette image sequence of the target pedestrian corresponding to the same identifier is obtained. In specific implementation, when obtaining silhouette images, the silhouette images of the target pedestrians corresponding to each identifier can be stored in the order of acquisition, and the silhouette image sequence of the target pedestrians can be obtained more quickly.

Example three

A gait-based identification device disclosed in this embodiment, as shown in FIG. 3, the device includes:

A target pedestrian determination module 310, configured to perform pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence;

A silhouette image sequence determination module 320, configured to determine a silhouette image sequence of the target pedestrian;

Gait energy map determination module 330 is configured to obtain a gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian;

The identity recognition module 340 is configured to identify the target pedestrian based on the gait energy map of the target pedestrian.

Optionally, when pedestrian detection, tracking, and pedestrian recognition are performed on the video image sequence to determine a target pedestrian in the video image sequence, as shown in FIG. 4, the target pedestrian determination module 310 further includes:

The detection tracking period determination submodule 3101 is configured to determine video images corresponding to consecutive detection tracking periods in a video image sequence according to a preset number of video image frames corresponding to each detection tracking period;

A periodic target pedestrian determination sub-module 3102, configured to detect, track and re-identify a target pedestrian in the video image sequence based on a video image corresponding to each of the detection and tracking periods in a front-to-back sequence, Determining a tracking result of a target pedestrian corresponding to each of the detection and tracking periods;

The tracking result determination submodule 3103 is configured to determine a target pedestrian in the video image sequence according to a tracking result of the target pedestrian corresponding to the last detection and tracking cycle.

Optionally, based on the video image corresponding to each of the detection and tracking periods, the target pedestrian in the video image sequence is detected, tracked and re-recognized to determine the target pedestrian corresponding to each of the detection and tracking periods. When tracking results, as shown in FIG. 5, the periodic target pedestrian determination sub-module 3102 further includes:

A tracking unit 31021 is configured to track, for each of the detection and tracking cycles, a predetermined target pedestrian in the video image sequence based on a tracking frame image corresponding to the current detection and tracking cycle, and determine a target corresponding to the current detection and tracking cycle. Tracking Results;

A detection unit 31022, configured to perform, for each of the detection and tracking periods, pedestrian detection on a detection frame image of a next detection and tracking period relative to the current detection and tracking period to determine a detection result;

A pedestrian re-identification unit 31023, configured to perform pedestrian re-identification according to the tracking result and the detection result for each of the detection and tracking cycles to determine a target pedestrian to be tracked in the next detection and tracking cycle;

Wherein, the detection frame image is a first frame video image in a preset number of video image sequences corresponding to each detection tracking cycle, and the tracking frame image is a first frame video image in a video image sequence corresponding to each detection and tracking cycle. Video images other than video images.

Optionally, the predetermined target pedestrian is determined in at least one of the following ways:

If the current detection and tracking period is the first detection and tracking period, it is determined according to the detection result of the detection frame image of the detection and tracking period;

If the current detection tracking period is not the first detection tracking period, it is determined according to a tracking result corresponding to a previous detection tracking period with respect to the current detection tracking period and a detection result of a detection frame image of the current detection tracking period.

Optionally, the tracking result includes at least: a target pedestrian to be tracked in a normal tracking state, a tracker of the target pedestrian, and a target pedestrian to be tracked in a tracking loss state; the detection result includes: the currently detected The detection frame of the target pedestrian, when the pedestrian re-identification is performed according to the tracking result and the detection result, and the target pedestrian to be tracked in the next detection and tracking cycle is determined, the pedestrian re-identification unit 31023 is further configured to:

Matching the area of the target pedestrian to be tracked in the normal tracking state with the currently detected target pedestrian to determine the target pedestrian pair;

For each of the target pedestrian pairs, perform pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked;

Performing pedestrian re-identification on each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the target pedestrians to be tracked in a tracking loss state to update the information of the target pedestrians to be tracked;

For a target pedestrian that needs to be tracked in a normal tracking state that does not constitute a target pedestrian pair, set it to a lost state.

Optionally, the step of performing pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked includes: :

Determine the similarity between the two target pedestrians according to the pedestrian characteristics of the two target pedestrians included in the target pedestrian pair, wherein the two target pedestrians include: the target pedestrian to be tracked and the currently detected Target pedestrian

If the two target pedestrians are determined to be the same person according to the similarity, updating a tracker of the target pedestrian to be tracked through a detection frame of the currently detected target pedestrian;

If it is determined that the two target pedestrians are not the same person according to the similarity, setting the target pedestrian to be tracked included in the target pedestrian pair to a lost state, and deleting the tracker of the target pedestrian to be tracked, and Adding the currently detected target pedestrians to the target pedestrians to be tracked;

Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology.

Optionally, each of the currently detected target pedestrians that do not form a target pedestrian pair is re-identified with a target pedestrian that needs to be tracked in a tracking loss state to update the information of the target pedestrian that needs to be tracked Steps, including:

Determining the pairwise similarity between each of the currently detected target pedestrians that do not constitute a target pedestrian pair and each target pedestrian that needs to be tracked in a tracking loss state according to the pedestrian characteristics obtained in advance;

If it is determined according to the pairwise similarity that the corresponding currently detected target pedestrian that does not constitute a target pedestrian pair is the same person as the target pedestrian in the tracking loss state, the demand in the tracking loss state is determined. The tracked target pedestrian is restored to a normal tracking state, and a tracker of the tracked target pedestrian to be restored to a normal tracking state is created by using a detection frame of the currently detected target pedestrian;

If it is determined according to the pairwise similarity that the currently detected target pedestrian is not a target pedestrian to be tracked in a tracking loss state, that is, the currently detected target pedestrian and each target row to be tracked in a tracking loss state If there is no match, add the currently detected target pedestrian to the target pedestrian to be tracked;

Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology.

In other embodiments of the present application, after adding the currently detected target pedestrian to the target pedestrian to be tracked, the method further includes: if the number of times that the currently detected target pedestrian is detected is less than a preset number , Remove it from the targeted pedestrians to track.

The gait-based identification device disclosed in this embodiment is used to implement the gait-based identification method described in Embodiment 1. For specific implementations of each module of the device, refer to the specific implementation of corresponding steps in Embodiment 1. This embodiment will not repeat them.

The gait-based identity recognition device disclosed in the embodiments of the present application determines a target pedestrian in the video image sequence by performing pedestrian detection, tracking, and pedestrian re-identification on the video image sequence; according to the obtained target pedestrian in the video image, The silhouette image in each frame of the video image in the sequence determines the silhouette image sequence of the target pedestrian; obtains the gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian; based on the gait energy map of the target pedestrian The identity recognition of the target pedestrian solves the problem of low accuracy of identity recognition based on gait recognition in a multi-person scene in the prior art. In the embodiment of the present application, the identification of the same and different target pedestrians in the video image sequence is performed by combining pedestrian detection, tracking, and pedestrian re-identification technologies, which improves the accuracy of target pedestrian recognition in the video image sequence and helps to improve the multi-person scene The accuracy of gait-based identification.

The embodiment of the present application improves the accuracy of identifying target pedestrians in a video image sequence by first performing pedestrian matching based on the area in which they are located, and then performing target pedestrian matching based on pedestrian characteristics to determine the same target pedestrian in different video image frames. Improve the accuracy of identity recognition based on gait.

By matching target pedestrians detected in subsequent video detection cycles with target pedestrians with missing tracking, it is possible to avoid inaccurate pedestrian recognition in the video image sequence caused by tracking errors.

While performing target pedestrian detection and tracking, the present application uses pedestrian re-identification technology to perform accurate pedestrian identification in the context of multiple pedestrians or across cameras, which is conducive to accurately extracting pedestrian silhouette image sequences. In addition, the present application performs adaptive gait feature extraction based on a convolutional neural network, which makes up for the drawback that it is difficult for traditional gait feature extraction methods to accurately extract representative gait features. The combination of the two well implements pedestrian identification based on gait under real-time camera surveillance conditions.

Correspondingly, the present application also discloses an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the processor is implemented as in the present application. The gait-based identification method according to the first embodiment. The electronic device may be a PC, a mobile terminal, a personal digital assistant, a tablet computer, or the like.

The present application also discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the gait-based identification method according to the first embodiment of the present application are implemented.

Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may refer to each other. As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the related parts, refer to the description of the method embodiment.

The gait-based identification method and device provided in this application have been described in detail above. Specific examples are used in this article to explain the principle and implementation of this application. The description of the above embodiments is only used to help understand this. The method of application and its core ideas; at the same time, for those of ordinary skill in the art, according to the ideas of this application, there will be changes in the specific implementation and application scope. In summary, the contents of this specification should not be understood Is a limitation on this application.

Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a necessary universal hardware platform, and of course, they can also be implemented by hardware. Based on such an understanding, the above-mentioned technical solution essentially or part that contributes to the existing technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic A disc, an optical disc, and the like include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments. In addition, the above definitions of the elements and methods are not limited to the various specific structures, shapes, or manners mentioned in the embodiments, and those skilled in the art can simply modify or replace them.

Similarly, it should be understood that, in order to streamline the present disclosure and help understand one or more of the various disclosed aspects, in the above description of exemplary embodiments of the present disclosure, various features of the present disclosure are sometimes grouped together into a single embodiment, Figure, or description of it. However, this disclosed method should not be construed as reflecting the intention that the claimed present disclosure claims more features than are expressly recited in each claim. Rather, as reflected in the following claims, the disclosure aspect is less than all features of the single embodiment disclosed previously. Thus, claims following a specific embodiment are hereby expressly incorporated into that specific embodiment, with each claim standing on its own as a separate embodiment of the present disclosure.

The specific embodiments described above further describe the objectives, technical solutions, and beneficial effects of the present disclosure in detail. It should be understood that the above are only specific embodiments of the present disclosure and are not intended to limit the present disclosure. Any modification, equivalent replacement, or improvement made within the spirit and principle of this disclosure shall be included in the protection scope of this disclosure.

Claims (28)

A gait-based identification method includes: Performing pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence; Determining a silhouette image sequence of the target pedestrian; Obtaining a gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian; Based on the gait energy map of the target pedestrian, the target pedestrian is identified. The method according to claim 1, wherein the steps of performing pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence include: Determining video images corresponding to consecutive detection and tracking periods in a video image sequence according to a preset number of video image frames corresponding to each detection and tracking period; In the order from front to back, based on the video images corresponding to each of the detection and tracking periods, the target pedestrian in the video image sequence is detected, tracked, and pedestrian re-identified, and the corresponding one of each of the detection and tracking periods is determined. Tracking results of target pedestrians; Determining the target pedestrian in the video image sequence according to the tracking result of the target pedestrian corresponding to the last detection and tracking cycle. The method according to claim 2, characterized in that, based on the video image corresponding to each of the detection and tracking periods, the target pedestrian in the video image sequence is detected, tracked, and pedestrian re-identified to determine each The step of detecting a tracking result of a target pedestrian corresponding to a tracking cycle includes: Do the following for each of the detection tracking cycles: Tracking a predetermined target pedestrian in the video image sequence based on a tracking frame image corresponding to the current detection tracking period, and determining a tracking result corresponding to the current detection tracking period; Performing pedestrian detection on a detection frame image of a next detection tracking period relative to the current detection tracking period to determine a detection result; Perform pedestrian re-identification according to the tracking result and the detection result, and determine a target pedestrian to be tracked in the next detection and tracking cycle; Wherein, the detection frame image is a first frame video image in a preset number of video image sequences corresponding to each detection tracking cycle, and the tracking frame image is a first frame video image in a video image sequence corresponding to each detection and tracking cycle. Video images other than video images. The method according to claim 3, wherein the predetermined target pedestrian is determined in at least one of the following ways: If the current detection and tracking period is the first detection and tracking period, it is determined according to the detection result of the detection frame image of the detection and tracking period; If the current detection tracking period is not the first detection tracking period, it is determined according to a tracking result corresponding to a previous detection tracking period with respect to the current detection tracking period and a detection result of a detection frame image of the current detection tracking period. The method according to claim 3, wherein the tracking result comprises at least: a target pedestrian to be tracked in a normal tracking state and a tracker of the target pedestrian, and a target pedestrian to be tracked in a tracking loss state; The detection result includes: a detection frame of a currently detected target pedestrian; The step of re-identifying a pedestrian based on the tracking result and the detection result and determining a target pedestrian to be tracked in the next detection and tracking cycle includes: Matching the area of the target pedestrian to be tracked in the normal tracking state with the currently detected target pedestrian to determine the target pedestrian pair; For each of the target pedestrian pairs, perform pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked; Performing pedestrian re-identification on each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the target pedestrians to be tracked in a tracking loss state to update the information of the target pedestrians to be tracked; For a target pedestrian that needs to be tracked in a normal tracking state that does not constitute a target pedestrian pair, set it to a lost state. The method according to claim 5, wherein the pedestrian re-identification is performed according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the tracked to be tracked Steps to target pedestrian information, including: Determine the similarity between the two target pedestrians according to the pedestrian characteristics of the two target pedestrians included in the target pedestrian pair, wherein the two target pedestrians include: the target pedestrian to be tracked and the currently detected Target pedestrian If the two target pedestrians are determined to be the same person according to the similarity, updating a tracker of the target pedestrian to be tracked through a detection frame of the currently detected target pedestrian; If it is determined that the two target pedestrians are not the same person according to the similarity, setting the target pedestrian to be tracked included in the target pedestrian pair to a lost state, and deleting the tracker of the target pedestrian to be tracked, and Adding the currently detected target pedestrian to the target pedestrian to be tracked; Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology. The method according to claim 5, wherein each of the currently detected target pedestrians that do not constitute a target pedestrian pair is re-identified with a target pedestrian that needs to be tracked in a tracking loss state to update The step of tracking the information of the target pedestrian includes: Determining the pairwise similarity between each of the currently detected target pedestrians that do not constitute a target pedestrian pair and each target pedestrian that needs to be tracked in a tracking loss state according to the pedestrian characteristics obtained in advance; If it is determined according to the pairwise similarity that the corresponding currently detected target pedestrian that does not constitute a target pedestrian pair is the same person as the target pedestrian in the tracking loss state, the demand in the tracking loss state is determined. The tracked target pedestrian is restored to a normal tracking state, and a tracker of the tracked target pedestrian to be restored to a normal tracking state is created by using a detection frame of the currently detected target pedestrian; If it is determined according to the pairwise similarity that the currently detected target pedestrian is not a target pedestrian to be tracked in a tracking loss state, adding the currently detected target pedestrian to the target pedestrian to be tracked; Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology. The method according to claim 6 or 7, wherein after the step of adding the currently detected target pedestrian to the target pedestrian to be tracked, further comprising: if the currently detected target pedestrian is If the number of detections is less than the preset number, it is removed from the target pedestrians to be tracked. The method according to claim 1, characterized in that in the step of detecting and tracking a video image sequence, the target pedestrians in the video image sequence are detected and tracked in cycles. The method according to claim 9, wherein in the step of detecting and tracking a video image sequence, target pedestrian detection is performed by using the first frame of video images in each detection and tracking cycle, and target pedestrians are extracted Pedestrian characteristics; track the detected target pedestrian through the second and subsequent frame video images of each detection and tracking cycle. The method according to claim 10, characterized in that, for two consecutive detection and tracking periods, the target pedestrians detected in the first frame of the video image of the subsequent detection and tracking period are updated in the previous detection and tracking period. Target pedestrian tracking results. The method according to claim 1, wherein the step of determining a silhouette image sequence of the target pedestrian comprises: Determining the image area of the target pedestrian in the detection frame image according to the detection frame of the target pedestrian, or determining the image area of the target pedestrian in the tracking frame image according to the tracking frame of the target pedestrian; Input the image area of the target pedestrian in the detection frame image or the tracking frame image to a pre-trained network model to obtain the silhouette image of the target pedestrian; The silhouette images corresponding to the same target pedestrian are arranged in the sequence of the video image frames to which the silhouette image belongs, to obtain the silhouette image sequence corresponding to the same target pedestrian. The method according to claim 1, wherein the step of identifying the target pedestrian based on the gait energy map of the target pedestrian includes: Identify the gait energy map of the target pedestrian through a pre-trained neural network model to determine the gait characteristics of the target pedestrian; Match the gait characteristics of the target pedestrian with the gait characteristics in the preset database, and determine the identity information of the user who matches the gait characteristics of the target pedestrian as the identity information of the target user. A gait-based identity recognition device, comprising: A target pedestrian determination module, configured to perform pedestrian detection, tracking, and pedestrian re-identification on a video image sequence to determine a target pedestrian in the video image sequence; A silhouette image sequence determination module, configured to determine a silhouette image sequence of the target pedestrian; A gait energy map determination module, configured to obtain a gait energy map of the target pedestrian based on the silhouette image sequence of each target pedestrian; An identity recognition module is configured to identify the target pedestrian based on the gait energy map of the target pedestrian. The device according to claim 14, characterized in that when performing pedestrian detection, tracking and pedestrian recognition on a video image sequence to determine a target pedestrian in the video image sequence, the target pedestrian determination module further comprises: The detection tracking period determination submodule is configured to determine video images corresponding to consecutive detection and tracking periods in a video image sequence according to a preset number of video image frames corresponding to each detection and tracking period; The periodic target pedestrian determination sub-module is configured to detect, track and re-identify the target pedestrian in the video image sequence based on the video images corresponding to each of the detection and tracking periods in the order from front to back. A tracking result of a target pedestrian corresponding to each of the detection and tracking periods; The tracking result determination submodule is configured to determine a target pedestrian in the video image sequence according to a tracking result of a target pedestrian corresponding to the last detection and tracking period. The device according to claim 15, characterized in that, based on a video image corresponding to each of the detection and tracking periods, a target pedestrian in the video image sequence is detected, tracked, and pedestrian re-identified to determine each pedestrian. When the tracking result of detecting the target pedestrian corresponding to the tracking period is described, the period target pedestrian determination sub-module further includes: A tracking unit is configured to track, for each of the detection and tracking cycles, a predetermined target pedestrian in the video image sequence based on a tracking frame image corresponding to the current detection and tracking cycle, and determine a tracking corresponding to the current detection and tracking cycle. result; A detection unit, configured to perform a pedestrian detection on a detection frame image of a next detection and tracking period relative to the current detection and tracking period for each of the detection and tracking periods to determine a detection result; A pedestrian re-identification unit, configured to perform pedestrian re-identification according to the tracking result and the detection result for each of the detection and tracking cycles, and determine a target pedestrian to be tracked in the next detection and tracking cycle; Wherein, the detection frame image is a first frame video image in a preset number of video image sequences corresponding to each detection tracking cycle, and the tracking frame image is a first frame video image in a video image sequence corresponding to each detection and tracking cycle. Video images other than video images. The device according to claim 16, wherein the predetermined target pedestrian is determined in at least one of the following ways: If the current detection and tracking period is the first detection and tracking period, it is determined according to the detection result of the detection frame image of the detection and tracking period; If the current detection tracking period is not the first detection tracking period, it is determined according to a tracking result corresponding to a previous detection tracking period with respect to the current detection tracking period and a detection result of a detection frame image of the current detection tracking period. The device according to claim 16, wherein the tracking result comprises at least: a target pedestrian to be tracked in a normal tracking state and a tracker of the target pedestrian, and a target pedestrian to be tracked in a tracking loss state; The detection result includes: a detection frame of a currently detected target pedestrian; When pedestrian re-identification is performed according to the tracking result and the detection result, and a target pedestrian to be tracked in the next detection and tracking cycle is determined, the pedestrian re-identification unit is further configured to: Matching the area of the target pedestrian to be tracked in the normal tracking state with the currently detected target pedestrian to determine the target pedestrian pair; For each of the target pedestrian pairs, perform pedestrian re-identification according to the similarity between the target pedestrian to be tracked included in the target pedestrian pair and the currently detected target pedestrian to update the information of the target pedestrian to be tracked; Performing pedestrian re-identification on each of the currently detected target pedestrians that do not constitute a target pedestrian pair and the target pedestrians to be tracked in a tracking loss state to update the information of the target pedestrians to be tracked; For a target pedestrian that needs to be tracked in a normal tracking state that does not constitute a target pedestrian pair, set it to a lost state. The device according to claim 18, wherein the pedestrian re-identification unit is configured to perform a pedestrian in the following manner according to a similarity between a target pedestrian to be tracked included in the target pedestrian pair and a currently detected target pedestrian. Re-identify and update the information of the target pedestrians to be tracked: Determine the similarity between the two target pedestrians according to the pedestrian characteristics of the two target pedestrians included in the target pedestrian pair, wherein the two target pedestrians include: the target pedestrian to be tracked and the currently detected Target pedestrian If the two target pedestrians are determined to be the same person according to the similarity, updating a tracker of the target pedestrian to be tracked through a detection frame of the currently detected target pedestrian; If it is determined that the two target pedestrians are not the same person according to the similarity, setting the target pedestrian to be tracked included in the target pedestrian pair to a lost state, and deleting the tracker of the target pedestrian to be tracked, and Adding the currently detected target pedestrian to the target pedestrian to be tracked; Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology. The device according to claim 18, wherein the pedestrian re-identification unit is configured to identify each of the currently detected target pedestrians that do not constitute a target pedestrian pair and those that need to be tracked in a tracking loss state. The target pedestrian performs pedestrian re-identification and updates the information of the target pedestrian to be tracked: Determining the pairwise similarity between each of the currently detected target pedestrians that do not constitute a target pedestrian pair and each target pedestrian that needs to be tracked in a tracking loss state according to the pedestrian characteristics obtained in advance; If it is determined according to the pairwise similarity that the corresponding currently detected target pedestrian that does not constitute a target pedestrian pair is the same person as the target pedestrian in the tracking loss state, the demand in the tracking loss state is determined. The tracked target pedestrian is restored to a normal tracking state, and a tracker of the tracked target pedestrian to be restored to a normal tracking state is created by using a detection frame of the currently detected target pedestrian; If it is determined according to the pairwise similarity that the currently detected target pedestrian is not a target pedestrian to be tracked in a tracking loss state, adding the currently detected target pedestrian to the target pedestrian to be tracked; Wherein, the pedestrian characteristics are obtained through pedestrian re-identification technology. The device according to claim 18, wherein after adding the currently detected target pedestrian to the target pedestrian to be tracked, if the number of times that the currently detected target pedestrian is detected is less than a preset Quantity, it is removed from the target pedestrians to be tracked. The device according to claim 14, wherein the target pedestrian determination module is configured to perform pedestrian detection and tracking on a video image sequence in a periodical manner. The device according to claim 22, wherein the target pedestrian determination module performs target pedestrian detection by using the first frame of video images in each detection and tracking cycle, and extracts pedestrian characteristics of the target pedestrian; through each detection and tracking cycle The second and subsequent frames of video images are used to track the detected target pedestrian. The device according to claim 23, wherein, for two consecutive detection and tracking cycles, the target pedestrian determination module updates the target pedestrians by detecting the target pedestrians from the first frame of the video image of the latter detection and tracking cycle. Target pedestrian tracking results during a detection tracking cycle. The apparatus according to claim 14, wherein the silhouette image sequence determination module is configured to determine a silhouette image sequence of the target pedestrian in the following manner: Determining the image area of the target pedestrian in the detection frame image according to the detection frame of the target pedestrian, or determining the image area of the target pedestrian in the tracking frame image according to the tracking frame of the target pedestrian; Input the image area of the target pedestrian in the detection frame image or the tracking frame image to a pre-trained network model to obtain the silhouette image of the target pedestrian; The silhouette images corresponding to the same target pedestrian are arranged in the sequence of the video image frames to which the silhouette image belongs, to obtain the silhouette image sequence corresponding to the same target pedestrian. The device according to claim 14, wherein the identity recognition module is configured to perform identity recognition in the following manner: Identify the gait energy map of the target pedestrian through a pre-trained neural network model to determine the gait characteristics of the target pedestrian; Match the gait characteristics of the target pedestrian with the gait characteristics in the preset database, and determine the identity information of the user who matches the gait characteristics of the target pedestrian as the identity information of the target user. An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the computer program, any one of claims 1 to 13 is implemented The gait-based identity recognition method described in item 6. A computer-readable storage medium having stored thereon a computer program, characterized in that when the program is executed by a processor, the steps of the gait-based identification method according to any one of claims 1 to 13 are implemented.

Images