CN116843726A - Pedestrian trajectory tracking method and device, electronic equipment and storage medium - Google Patents

Abstract

The invention discloses a pedestrian trajectory tracking method and device, electronic equipment and storage medium. Wherein, the method includes: obtaining a video collection collected by multiple cameras, wherein the video collection includes multiple videos, each video corresponding to a camera; identifying pedestrian frames in each video in the video collection; based on the The pedestrian frame generates a segmented movement trajectory of the target pedestrian; obtains the spatial characteristics of the multiple cameras, couples the multiple segmented movement trajectories according to the spatial characteristics, and generates a complete movement trajectory of the target pedestrian. Through the present invention, the complete movement trajectory of a target pedestrian can be generated without extracting facial features, realizing full trajectory tracking of pedestrians and solving the technical problem in the related art that a single camera generates incomplete pedestrian trajectories.

Description

Pedestrian trajectory tracking method and device, electronic equipment and storage medium

Technical field

The present invention relates to the field of safety surveillance, and specifically to a pedestrian trajectory tracking method and device, electronic equipment and storage media.

Background technique

Among related technologies, rail transit pedestrian behavior recognition and training methods include image-based recognition methods and human skeleton-based recognition methods. Among them, image-based recognition methods are relatively low-cost and do not require additional data collection equipment. Human skeleton-based recognition methods Depth sensing sensors need to be added, and existing equipment needs to be modified, which is costly.

In related technologies, pedestrian trajectories are mainly tracked through a single camera or face recognition method, and there are problems such as large flow of people and severe occlusion. At the same time, due to the high installation position of surveillance cameras, it is difficult to accurately identify faces. , therefore, it is difficult to implement such methods.

No effective solution has yet been found for the above problems existing in related technologies.

Contents of the invention

The invention provides a pedestrian trajectory tracking method and device, electronic equipment and storage medium.

According to one aspect of the embodiment of the present application, a pedestrian trajectory tracking method is provided. The method includes: obtaining a video collection collected by multiple cameras, wherein the video collection includes multiple videos, each video corresponding to a Camera; identify the pedestrian frame in each video in the video collection; generate the segmented movement trajectory of the target pedestrian based on the pedestrian frame; obtain the spatial characteristics of the multiple cameras, and couple the multiple segments according to the spatial characteristics Movement trajectory to generate the complete movement trajectory of the target pedestrian.

Further, generating the segmented movement trajectory of the target pedestrian based on the pedestrian frame includes: performing bit tracking on the pedestrian frame to generate a preliminary movement trajectory of the target pedestrian; using an adaptive Kalman filter model to update the preliminary movement trajectory, Generate segmented movement trajectories of the target pedestrian.

Further, an adaptive Kalman filter model is used to update the preliminary movement trajectory, and generating the segmented movement trajectory of the target pedestrian includes: starting from a preset initial value, iteratively updating the trajectory vector of the preliminary movement trajectory until the The last trajectory vector of the preliminary movement trajectory:

Input the current trajectory vector into the following prediction stage formula and output the predicted value of the current trajectory vector. :

;

;

Will The following update phase formula outputs the updated value of the current trajectory vector. :

;

;

;

;

;in, is the predicted value of the trajectory vector at the current moment, which includes: position, velocity, and acceleration, is the state transition matrix, used to describe the mathematical relationship between the previous moment in the trajectory vector and the current moment trajectory vector, is the updated value of the trajectory vector at the previous moment, is the system noise coefficient matrix, is the system noise at the current moment, is the predicted value of the covariance matrix at the current moment, is the updated value of the covariance matrix at the previous moment, is the noise covariance matrix, To measure noise, the Kalman gain at the previous moment can be adaptively adjusted, is the identity matrix, is a measurement matrix, used to describe the mathematical relationship between measured quantities and predicted quantities, is the Kalman gain at the current moment, is the updated value of the trajectory vector, is the measured quantity at the current moment, is the updated value of the covariance matrix at the current moment, is the update parameter, b is the forgetting factor, , p is the probability of pedestrian possibility, , is the area boundary of the shooting scene;

After all trajectory vectors of the preliminary movement trajectory are iterated, the updated values of all trajectory points are used to output the segmented movement trajectory of the target pedestrian.

Further, obtaining spatial characteristics of the multiple cameras, coupling multiple segmented movement trajectories according to the spatial characteristics, and generating a complete movement trajectory of the target pedestrian includes: calculating the walking of the target pedestrian based on the segmented movement trajectories. speed and the shortest distance of trajectories between adjacent cameras, where the spatial features include the shortest distance of trajectories; the distribution probability of each segmented movement trajectory in each camera is calculated based on the walking speed and the shortest distance of trajectories. ; Select the target trajectory curve with the highest probability for each camera, and use the target trajectory curve coupling of multiple cameras to generate the complete movement trajectory of the target pedestrian.

Further, calculating the shortest distance between adjacent cameras based on the segmented movement trajectory includes: using the following formula to calculate the Fronmin distance between the first trajectory curve and the second trajectory curve of the target pedestrian in the first camera. Rest distance D: ;in, Represents the identifiable area of the first camera, Represents the identifiable area of the second camera, represents the pedestrian-accessible area, P represents the first trajectory curve of the segmented movement trajectory identified by the first camera, Q represents the second trajectory curve of any segmented movement trajectory identified by the second camera, inf represents the lower bound, and d is the Euclidean distance, t is time, is each possible pair of position description functions that change over time, and is the value of [0,1], is the distance between all points on the two trajectory curves of PQ.

Further, calculating the distribution probability of each segmented movement trajectory in each camera based on the walking speed and the shortest distance of the trajectory includes: for each camera, the following formula is used to calculate the distribution probability p: ;in, is the average walking speed of all pedestrians recognized by the current camera, is the mean of the shortest distance of all segmented movement trajectories identified for the current camera, is the standard deviation of the speed, is the standard deviation of the distance, is the correlation coefficient, v is the walking speed of the target pedestrian in the current segmented movement trajectory, and D is the shortest distance of the current segmented movement trajectory.

Further, before calculating the distribution probability of each segmented movement trajectory in each camera according to the walking speed and the shortest distance of the trajectory, the method also includes: for each camera, determining whether the corresponding maximum distribution probability is less than a predetermined Set a threshold value; if the corresponding maximum distribution probability is less than the preset threshold value, the target trajectory curve of the corresponding camera is eliminated.

According to another aspect of the embodiment of the present application, a pedestrian trajectory tracking device is also provided, including: an acquisition module, configured to acquire a video collection collected by multiple cameras, wherein the video collection includes multiple videos, each The video corresponds to one camera; the identification module is used to identify the pedestrian frame in each video in the video collection; the generation module is used to generate the segmented movement trajectory of the target pedestrian based on the pedestrian frame; the coupling module is used to obtain The spatial characteristics of the multiple cameras are coupled with multiple segmented movement trajectories according to the spatial characteristics to generate a complete movement trajectory of the target pedestrian.

Further, the generation module includes: a first generation unit for bit tracking the pedestrian frame and generating a preliminary movement trajectory of the target pedestrian; a second generation unit for using an adaptive Kalman filter model to generate the preliminary movement trajectory of the target pedestrian. The movement trajectory is updated to generate the segmented movement trajectory of the target pedestrian.

Further, the second generation unit includes: an iterative subunit, used to iteratively update the trajectory vector of the preliminary movement trajectory starting from a preset initial value until the last trajectory vector of the preliminary movement trajectory: convert the current trajectory vector into Enter the following prediction stage formula to output the predicted value of the current trajectory vector. :

;

;

Will The following update phase formula outputs the updated value of the current trajectory vector. :

;

;

;

;

;in, is the predicted value of the trajectory vector at the current moment, which includes: position, velocity, and acceleration, is the state transition matrix, used to describe the mathematical relationship between the previous moment in the trajectory vector and the current moment trajectory vector, is the updated value of the trajectory vector at the previous moment, is the system noise coefficient matrix, is the system noise at the current moment, is the predicted value of the covariance matrix at the current moment, is the updated value of the covariance matrix at the previous moment, is the noise covariance matrix, To measure noise, the Kalman gain at the previous moment can be adaptively adjusted, is the identity matrix, is a measurement matrix, used to describe the mathematical relationship between measured quantities and predicted quantities, is the Kalman gain at the current moment, is the updated value of the trajectory vector, is the measured quantity at the current moment, is the updated value of the covariance matrix at the current moment, is the update parameter, b is the forgetting factor, , p is the probability of pedestrian possibility, , is the regional boundary of the shooting scene; the output subunit is used to output the segmented movement trajectory of the target pedestrian using the updated values of all trajectory points after all trajectory values of the preliminary movement trajectory are iterated.

Further, the coupling module includes: a first calculation unit for calculating the walking speed of the target pedestrian and the shortest distance of the trajectory between adjacent cameras according to the segmented movement trajectory, wherein the spatial feature includes the The shortest distance of the trajectory; the second calculation unit is used to calculate the distribution probability of each segmented movement trajectory in each camera according to the walking speed and the shortest distance of the trajectory; the coupling unit is used to select the target with the highest probability for each camera The trajectory curve uses the target trajectory curve coupling of multiple cameras to generate the complete movement trajectory of the target pedestrian.

Further, the first calculation unit includes: a calculation subunit for calculating the Flamingche distance D between the first trajectory curve and the second trajectory curve of the target pedestrian in the first camera using the following formula: ;in, Represents the identifiable area of the first camera, Represents the identifiable area of the second camera, represents the pedestrian-accessible area, P represents the first trajectory curve of the segmented movement trajectory identified by the first camera, Q represents the second trajectory curve of any segmented movement trajectory identified by the second camera, inf represents the lower bound, and d is the Euclidean distance, t is time, is each possible pair of position description functions that change over time, and is the value of [0,1], is the distance between all points on the two trajectory curves of PQ.

Further, the second calculation unit includes: a calculation subunit, used to calculate the distribution probability p for each camera using the following formula: ; in, is the average walking speed of all pedestrians recognized by the current camera, is the mean of the shortest distance of all segmented movement trajectories identified for the current camera, is the standard deviation of the speed, is the standard deviation of the distance, is the correlation coefficient, v is the walking speed of the target pedestrian in the current segmented movement trajectory, and D is the shortest distance of the current segmented movement trajectory.

Further, the coupling module further includes: a judgment unit, configured to determine the distribution probability of each segmented movement trajectory in each camera according to the walking speed and the shortest distance of the trajectory before the second calculation unit calculates the distribution probability for each segmented movement trajectory in each camera. camera, to determine whether the corresponding maximum distribution probability is less than the preset threshold value; the elimination unit is used to eliminate the target trajectory curve of the corresponding camera if the corresponding maximum distribution probability is less than the preset threshold value.

According to another aspect of the embodiment of the present application, a storage medium is also provided. The storage medium includes a stored program, and the above steps are executed when the program is run.

According to another aspect of the embodiment of the present application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: The memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.

Embodiments of the present application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the above method.

Through the present invention, a video collection collected by multiple cameras is obtained, where the video collection includes multiple videos, each video corresponding to a camera; the pedestrian frame in each video in the video collection is identified, and a classification of the target pedestrian is generated based on the pedestrian frame. segment movement trajectories, obtain the spatial characteristics of multiple cameras, couple multiple segment movement trajectories according to the spatial characteristics, and generate a complete movement trajectory of the target pedestrian. By identifying the segment movement trajectories of the target pedestrian in the video collected by each camera, and Coupling multiple segmented movement trajectories according to the spatial characteristics of the camera can generate the complete movement trajectory of the target pedestrian without extracting facial features, realizing full trajectory tracking of pedestrians and solving the problem of inconsistencies in generating pedestrian trajectories with a single camera in related technologies. Complete technical question.

Description of the drawings

The drawings described here are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached picture:

Figure 1 is a hardware structural block diagram of a computer according to an embodiment of the present invention;

Figure 2 is a flow chart of a pedestrian trajectory tracking method according to an embodiment of the present invention;

Figure 3 is a schematic diagram of adaptive Kalman filtering in an embodiment of the present invention;

Figure 4 is a schematic diagram of the coupling target trajectory curve in the embodiment of the present invention;

Figure 5 is a schematic flow chart of trajectory tracking according to the embodiment of the present invention;

Figure 6 is a structural block diagram of a pedestrian trajectory tracking device according to an embodiment of the present invention.

Detailed ways

In order to enable those in the technical field to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only These are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of this application. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other.

It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, product or apparatus that encompasses a series of steps or units need not be limited to those steps explicitly listed or units, but may include other steps or units not expressly listed or inherent to such processes, methods, products or devices.

Example 1

The method embodiment provided in Embodiment 1 of the present application can be executed in a controller, server, computer, tablet or similar computing device. Taking running on a computer as an example, FIG. 1 is a hardware structure block diagram of a computer according to an embodiment of the present invention. As shown in Figure 1, the computer may include one or more (only one is shown in Figure 1) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and A memory 104 for storing data. Optionally, the above-mentioned computer may also include a transmission device 106 and an input and output device 108 for communication functions. Persons of ordinary skill in the art can understand that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-mentioned computer. For example, the computer may also include more or fewer components than shown in FIG. 1 , or have a different configuration than that shown in FIG. 1 .

The memory 104 can be used to store computer programs, such as software programs and modules of application software, such as a computer program corresponding to a pedestrian trajectory tracking method in an embodiment of the present invention. The processor 102 runs the computer program stored in the memory 104 , thereby executing various functional applications and data processing, that is, implementing the above method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, and these remote memories may be connected to the computer through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

Transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the computer's communication provider. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet wirelessly.

This embodiment provides a pedestrian trajectory tracking method. Figure 2 is a flow chart of a pedestrian trajectory tracking method according to an embodiment of the present invention. As shown in Figure 2, the process includes the following steps:

Step S202: Obtain a video collection collected by multiple cameras, where the video collection includes multiple videos, and each video corresponds to one camera;

The solution of this embodiment can be applied to subway stations, railway stations, bus stations and other places. Multiple cameras are installed in the place, and each camera collects images in the corresponding area. The camera of this embodiment is a visual camera, that is, a two-dimensional camera. .

Step S204, identify pedestrian frames in each video in the video collection;

In this embodiment, the video collected by the camera includes multiple frames of images, and each frame of the image may include several pedestrians. Through pedestrian frame recognition, the pedestrian frame, such as a rectangular frame, can be detected.

Step S206: Generate segmented movement trajectories of the target pedestrian based on the pedestrian frame;

Since each camera collects a video of consecutive multiple frames of images, by continuously identifying the consecutive image frames in the video, the segmented movement trajectory of a pedestrian in the current video segment can be obtained. In this embodiment, each camera corresponds to several segmented movement trajectories, each segmented movement trajectory corresponds to one pedestrian, each target pedestrian corresponds to several segmented movement trajectories, and each segmented movement trajectory corresponds to one camera.

Step S208: Obtain the spatial characteristics of multiple cameras, couple multiple segmented movement trajectories according to the spatial characteristics, and generate a complete movement trajectory of the target pedestrian.

Optionally, the spatial features can be the distance between the segmented movement trajectories recognized by two adjacent cameras, the layout position of the cameras, spatial relationships, etc.

Through the above steps, a video collection collected by multiple cameras is obtained. The video collection includes multiple videos, and each video corresponds to a camera. The pedestrian frame in each video in the video collection is identified, and a classification of the target pedestrian is generated based on the pedestrian frame. segment movement trajectories, obtain the spatial characteristics of multiple cameras, couple multiple segment movement trajectories according to the spatial characteristics, and generate a complete movement trajectory of the target pedestrian. By identifying the segment movement trajectories of the target pedestrian in the video collected by each camera, and Coupling multiple segmented movement trajectories according to the spatial characteristics of the camera can generate the complete movement trajectory of the target pedestrian without extracting facial features, realizing full trajectory tracking of pedestrians and solving the problem of inconsistencies in generating pedestrian trajectories with a single camera in related technologies. Complete technical question.

In one implementation of this embodiment, generating the segmented movement trajectory of the target pedestrian based on the pedestrian frame includes:

S11, perform bit tracking on the pedestrian frame and generate the preliminary movement trajectory of the target pedestrian;

S12, use the adaptive Kalman filter model to update the preliminary movement trajectory and generate the segmented movement trajectory of the target pedestrian.

In one example, the adaptive Kalman filter model is used to update the preliminary movement trajectory. Generating the segmented movement trajectory of the target pedestrian includes: starting from a preset initial value, iteratively updating the trajectory vector of the preliminary movement trajectory until the The last trajectory vector: Input the current trajectory vector into the following prediction stage formula and output the predicted value of the current trajectory vector. :

;

;

Will The following update phase formula outputs the updated value of the current trajectory vector. :

;

;

;

;

;in, is the predicted value of the trajectory vector at the current moment, which includes: position, velocity, and acceleration, is the state transition matrix, used to describe the mathematical relationship between the previous moment in the trajectory vector and the current moment trajectory vector, is the updated value of the trajectory vector at the previous moment, is the system noise coefficient matrix, is the system noise at the current moment, is the predicted value of the covariance matrix at the current moment, is the updated value of the covariance matrix at the previous moment, is the noise covariance matrix, To measure noise, the Kalman gain at the previous moment can be adaptively adjusted, is the identity matrix, is a measurement matrix, used to describe the mathematical relationship between measured quantities and predicted quantities, is the Kalman gain at the current moment, is the updated value of the trajectory vector, is the measured quantity at the current moment, is the updated value of the covariance matrix at the current moment, is the update parameter, b is the forgetting factor, , p is the probability of pedestrian possibility, , is the area boundary of the shooting scene;

After all trajectory vectors of the preliminary movement trajectory are iterated, the updated values of all trajectory points are used to output the segmented movement trajectory of the target pedestrian.

When generating the segmented movement trajectory of each pedestrian, it is difficult to generate trajectories due to the large flow of people in scenes such as subway stations. For this reason, this embodiment first generates a preliminary trajectory through Byte Track, and then uses an improved adaptive Kalman filter (AKF) updates the detected trajectories and generates optimized segmented movement trajectories.

Figure 3 is a schematic diagram of adaptive Kalman filtering in an embodiment of the present invention. Compared with traditional adaptive Kalman filtering, the algorithm of this embodiment adds boundary constraints. Due to the existence of walls in scenes such as subways, there is a problem in prediction and The impact of boundary constraints needs to be considered when updating. At the same time, the update parameters used in R self-adjustment and Q self-adjustment are related to the corresponding pedestrian possibility probability obtained in Byte Track.

In this embodiment, obtaining the spatial characteristics of multiple cameras, coupling multiple segmented movement trajectories according to the spatial characteristics, and generating the complete movement trajectory of the target pedestrian includes:

S21, calculate the walking speed of the target pedestrian and the shortest distance between adjacent cameras based on the segmented movement trajectory, where the spatial features include the shortest distance of the trajectory;

In some examples, calculating the shortest distance between adjacent cameras based on the segmented movement trajectories includes: using the following formula to calculate the flamenche between the first trajectory curve and the second trajectory curve of the target pedestrian in the first camera. Distance D: ;in, Represents the identifiable area of the first camera, Represents the identifiable area of the second camera, represents the pedestrian-accessible area, P represents the first trajectory curve of the segmented movement trajectory identified by the first camera, Q represents the second trajectory curve of any segmented movement trajectory identified by the second camera, inf represents the lower bound, and d is the Euclidean distance, t is time, is each possible pair of position description functions that change over time, and is the value of [0,1], is the distance between all points on the two trajectory curves of PQ.

This embodiment extracts spatial features based on the trajectory generated by a single camera and the camera position. The possibility of the same pedestrian appearing under different cameras is related to the walking speed and the distance of the identifiable trajectory. First, the average of the trajectory curves detectable by different cameras is the input. , calculate its shortest distance via Frechet Distance.

The principle of Frechet Distance application in subway stations is as follows:

;

Among them, D represents the spatial metric distance, that is, the closest distance between the two trajectories, P and Q represent the two trajectory curves, inf represents the lower bound, d is the Euclidean distance, and t is time. is each possible pair of position description functions that change over time, and is the value of [0,1], is the distance between all points on the two trajectory curves of PQ.

Furthermore, the boundary constraint problem in the subway station scenario can be expressed as the following formula:

;

in , Represents the identifiable area of camera 1 and camera 2, Represents a pedestrian-accessible area.

S22, calculate the distribution probability of each segmented movement trajectory in each camera based on the walking speed and the shortest distance of the trajectory;

Optionally, calculating the distribution probability of each segmented movement trajectory in each camera based on the walking speed and the shortest distance of the trajectory includes: for each camera, use the following formula to calculate the distribution probability p: ;in, is the average walking speed of all pedestrians recognized by the current camera, is the mean of the shortest distance of all segmented movement trajectories identified by the current camera, is the standard deviation of the speed, is the standard deviation of the distance, is the correlation coefficient, v is the walking speed of the target pedestrian in the current segmented movement trajectory, and D is the shortest distance of the current segmented movement trajectory.

S23, select the target trajectory curve with the highest probability for each camera, and use the target trajectory curve coupling of multiple cameras to generate the complete movement trajectory of the target pedestrian.

Figure 4 is a schematic diagram of the coupling target trajectory curve in the embodiment of the present invention. , Represents the identifiable areas of camera 1 and camera 2. Trajectory 1 is the target trajectory curve with the highest probability for camera 1. Trajectory 2 is the target trajectory curve with the highest probability for camera 2. The highest probability is the same curve under different cameras, which can be coupled. for output.

In one example, before calculating the distribution probability of each segmented movement trajectory in each camera based on the walking speed and the shortest distance of the trajectory, it also includes: for each camera, determining whether the corresponding maximum distribution probability is less than a preset threshold value ; If the corresponding maximum distribution probability is less than the preset threshold, the target trajectory curve of the corresponding camera is eliminated.

Through elimination, the target trajectory curve of the camera that has not collected the target pedestrian can be filtered out, preventing the movement trajectory from jumping out of order, and improving the accuracy of the complete movement trajectory of the target pedestrian.

Figure 5 is a schematic flow chart of trajectory tracking according to an embodiment of the present invention. A multi-camera coupled rail transit passenger complete trajectory tracking method and device are proposed, which can accurately depict passengers in the station through coupling and association between cameras and spatial characteristics. The walking trajectory includes the following steps:

Step 1: Data collection, transmit the data from each camera to the server in real time for image data processing.

Step 2: Pedestrian recognition, identify pedestrians under a single camera through the yolo framework.

Step 3: Trajectory generation. In the subway station hall scene, where the flow of people is large, trajectory generation is difficult. For this reason, the present invention first generates a preliminary trajectory through ByteTrack, and then uses an improved adaptive Kalman filter (AKF) to detect The trajectory is updated and an optimized trajectory is generated.

Step 4: Spatial feature generation, extract spatial features based on the generated trajectory and camera position, and calculate the trajectory distance through Frechet Distance.

Step 5: Trajectory tracking, obtain the complete trajectory path through the probability distribution curve.

Using the solution of this embodiment, an improved adaptive Kalman filtering method is adopted, camera space feature extraction is based on improved FrechetDistance, and multi-camera trajectory curve coupling is performed. It can solve the problem of dispersed cameras in subway and other scenes and the difficulty of multi-view face fusion detection. It can also solve the problem of high density of people flow, easy occlusion and difficulty in path tracking. For face information collection difficulties, through the recognition of paths and human body features, it can be judged whether the same passenger is viewed by multiple cameras, which can solve the problem of trajectory tracking when facial features cannot be recognized, and can solve the problem of single camera monitoring in crowded situations. It can solve the problem of unstable trajectory in subway stations and solve the problem of full trajectory tracking in subway stations.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is Better implementation. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or that contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk, CD), including several instructions to cause a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

Example 2

This embodiment also provides a pedestrian trajectory tracking device, which is used to implement the above embodiments and preferred implementations. What has already been explained will not be described again. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

Figure 6 is a structural block diagram of a pedestrian trajectory tracking device according to an embodiment of the present invention. As shown in Figure 6, the device includes: an acquisition module 60, an identification module 62, a generation module 64, and a coupling module 66, where,

The acquisition module 60 is used to acquire a video collection collected by multiple cameras, where the video collection includes multiple videos, and each video corresponds to one camera;

Recognition module 62, used to identify pedestrian frames in each video in the video collection;

Generating module 64, configured to generate segmented movement trajectories of the target pedestrian based on the pedestrian frame;

The coupling module 66 is used to obtain the spatial characteristics of the multiple cameras, couple multiple segmented movement trajectories according to the spatial characteristics, and generate a complete movement trajectory of the target pedestrian.

Optionally, the generation module includes: a first generation unit for bit tracking the pedestrian frame and generating a preliminary movement trajectory of the target pedestrian; a second generation unit for using an adaptive Kalman filter model to The preliminary movement trajectory is updated to generate the segmented movement trajectory of the target pedestrian.

Optionally, the second generation unit includes: an iterative subunit, used to iteratively update the trajectory vector of the preliminary movement trajectory starting from a preset initial value until the last trajectory vector of the preliminary movement trajectory: convert the current The trajectory vector inputs the following prediction stage formula and outputs the predicted value of the current trajectory vector. :

;

;

Will The following update phase formula outputs the updated value of the current trajectory vector. :

;

;

;

;

;in, is the predicted value of the trajectory vector at the current moment, which includes: position, velocity, and acceleration, is the state transition matrix, used to describe the mathematical relationship between the previous moment in the trajectory vector and the current moment trajectory vector, is the updated value of the trajectory vector at the previous moment, is the system noise coefficient matrix, is the system noise at the current moment, is the predicted value of the covariance matrix at the current moment, is the updated value of the covariance matrix at the previous moment, is the noise covariance matrix, To measure noise, the Kalman gain at the previous moment can be adaptively adjusted, is the identity matrix, is a measurement matrix, used to describe the mathematical relationship between measured quantities and predicted quantities, is the Kalman gain at the current moment, is the updated value of the trajectory vector, is the measured quantity at the current moment, is the updated value of the covariance matrix at the current moment, is the update parameter, b is the forgetting factor, , p is the probability of pedestrian possibility, , is the regional boundary of the shooting scene; the output subunit is used to output the segmented movement trajectory of the target pedestrian using the updated values of all trajectory points after all trajectory vectors of the preliminary movement trajectory are iterated.

Optionally, the coupling module includes: a first calculation unit, configured to calculate the walking speed of the target pedestrian and the shortest distance between adjacent cameras according to the segmented movement trajectory, wherein the spatial features include The shortest distance of the trajectory; a second calculation unit, used to calculate the distribution probability of each segmented movement trajectory in each camera according to the walking speed and the shortest distance of the trajectory; a coupling unit, used to select the maximum probability of each camera The target trajectory curve of multiple cameras is coupled to generate the complete movement trajectory of the target pedestrian.

Optionally, the first calculation unit includes: a calculation subunit for calculating the Flamingche distance D between the first trajectory curve and the second trajectory curve of the target pedestrian in the first camera using the following formula: : ;in, Represents the identifiable area of the first camera, Represents the identifiable area of the second camera, represents the pedestrian-accessible area, P represents the first trajectory curve of the segmented movement trajectory identified by the first camera, Q represents the second trajectory curve of any segmented movement trajectory identified by the second camera, inf represents the lower bound d is the Euclidean distance , t is time, is each possible pair of position description functions that change over time, and is the value of [0,1], is the distance between all points on the two trajectory curves of PQ.

Optionally, the second calculation unit includes: a calculation subunit, used to calculate the distribution probability p for each camera using the following formula: ; in, is the average walking speed of all pedestrians recognized by the current camera, is the mean of the shortest distance of all segmented movement trajectories identified for the current camera, is the standard deviation of the speed, is the standard deviation of the distance, is the correlation coefficient, v is the walking speed of the target pedestrian in the current segmented movement trajectory, and D is the shortest distance of the current segmented movement trajectory.

Optionally, the coupling module also includes: a judgment unit, configured to calculate the distribution probability of each segmented movement trajectory in each camera according to the walking speed and the shortest distance of the trajectory before the second calculation unit, For each camera, determine whether the corresponding maximum distribution probability is less than the preset threshold value; the elimination unit is used to eliminate the target trajectory curve of the corresponding camera if the corresponding maximum distribution probability is less than the preset threshold value.

It should be noted that each of the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but is not limited to this: the above modules are all located in the same processor; or the above modules can be implemented in any combination. The forms are located in different processors.

Example 3

Embodiments of the present invention also provide a storage medium in which a computer program is stored, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

Optionally, in this embodiment, the above-mentioned storage medium may be configured to store a computer program for performing the following steps:

S1. Obtain a video collection collected by multiple cameras, where the video collection includes multiple videos, and each video corresponds to one camera;

S2, identify the pedestrian frame in each video in the video collection;

S3: Generate segmented movement trajectories of the target pedestrian based on the pedestrian frame;

S4: Obtain the spatial characteristics of the multiple cameras, couple multiple segmented movement trajectories according to the spatial characteristics, and generate a complete movement trajectory of the target pedestrian.

Optionally, in this embodiment, the above storage medium may include but is not limited to: U disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), Various media that can store computer programs, such as removable hard drives, magnetic disks, or optical disks.

An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

Optionally, the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.

Optionally, in this embodiment, the above-mentioned processor may be configured to perform the following steps through a computer program:

S1. Obtain a video collection collected by multiple cameras, where the video collection includes multiple videos, and each video corresponds to one camera;

S2, identify the pedestrian frame in each video in the video collection;

S3: Generate segmented movement trajectories of the target pedestrian based on the pedestrian frame;

S4: Obtain the spatial characteristics of the multiple cameras, couple multiple segmented movement trajectories according to the spatial characteristics, and generate a complete movement trajectory of the target pedestrian.

Optionally, for specific examples in this embodiment, reference can be made to the examples described in the above-mentioned embodiments and optional implementations, and details will not be described again in this embodiment.

The above serial numbers of the embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

In the above-mentioned embodiments of the present application, each embodiment is described with its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or may be Integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the units or modules may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code. .

The above are only the preferred embodiments of the present application. It should be pointed out that for those of ordinary skill in the art, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications can also be made. should be regarded as the scope of protection of this application.

Claims (10)

1. A method of tracking a pedestrian trajectory, the method comprising:

acquiring a video set acquired by a plurality of cameras, wherein the video set comprises a plurality of videos, and each video corresponds to one camera;

identifying a pedestrian box in each video in the set of videos;

generating a segmented movement track of a target pedestrian based on the pedestrian frame;

and acquiring the spatial characteristics of the cameras, coupling a plurality of segmented movement tracks according to the spatial characteristics, and generating a complete movement track of the target pedestrian.

2. The method of claim 1, wherein generating a segmented movement trajectory of a target pedestrian based on the pedestrian frame comprises:

performing bit tracking on the pedestrian frame to generate a preliminary movement track of a target pedestrian;

and updating the preliminary movement track by adopting a self-adaptive Kalman filtering model to generate the segmented movement track of the target pedestrian.

3. The method of claim 2, wherein updating the preliminary movement trajectory using an adaptive kalman filter model, generating a segmented movement trajectory of the target pedestrian comprises:

iteratively updating the track vector of the preliminary movement track from a preset initial value until the last track vector of the preliminary movement track:

inputting the current track vector into the following prediction stage formula, and outputting the predicted value of the current track vector：

；

；

Will beThe following update phase formula outputs the update value of the current track vector +.>：

；

；

；

；

； wherein ,/>A predicted value of a track vector at a current moment, wherein the track vector comprises: position, speed, acceleration, ">Is a state transition matrix for describing mathematical relationship between the track vector of the previous moment and the track vector of the current moment,/for the track vector>For the updated value of the trajectory vector at the previous time, and (2)>Is a system noise coefficient matrix->For the current moment system noise +.>For the predicted value of the covariance matrix at the current moment, < + >>For the updated value of the covariance matrix at the last moment,/->Is a noise covariance matrix>For measuring noise, the adaptive adjustment can be made by means of the Kalman gain at the previous time instant,/ >Is a unitary matrix->For the measurement matrix, a mathematical relationship between the measured quantity and the predicted quantity is described, < >>For the Kalman gain at the current moment, +.>For the updated value of the track vector, +.>For the measurement of the current time, +.>For the updated value of the covariance matrix at the current instant, and (2)>For updating parameters, b is a forgetting factor, < ->P is the probability of pedestrian probability, +.>，/>The method comprises the steps of (1) shooting a region boundary of a scene;

and after all track vectors of the preliminary movement track are iterated, outputting the segmented movement track of the target pedestrian by adopting updated values of all track points.

4. The method of claim 1, wherein obtaining spatial features of the plurality of cameras, coupling a plurality of segmented movement trajectories according to the spatial features, generating a complete movement trajectory of the target pedestrian comprises:

calculating the walking speed of the target pedestrian and the track shortest distance between adjacent cameras according to the segmented moving track, wherein the spatial features comprise the track shortest distance;

calculating the distribution probability of each sectional moving track in each camera according to the walking speed and the track shortest distance;

and selecting a target track curve with the maximum probability of each camera, and generating a complete moving track of the target pedestrian by adopting target track curve coupling of a plurality of cameras.

5. The method of claim 4, wherein calculating a trajectory shortest distance between adjacent cameras from the segmented movement trajectory comprises:

the Fregming distance D between a first track curve and a second track curve of the target pedestrian in the first camera is calculated by adopting the following formula:

；

wherein ,representing an identifiable region of the first camera, < >>Representing an identifiable region of the second camera, < >>Representing a passable area of a pedestrian, P representing a first track curve of a segmented moving track identified by a first camera, Q representing a second track curve of any segmented moving track identified by a second camera, inf representing a lower definition, d being a Euclidean distance, t being time>For each pair of possible location description functions that vary with time, it is [0,1]Value of->Is the distance between all points on the two trajectory curves of PQ.

6. The method of claim 4, wherein calculating a distribution probability for each segmented movement track in each camera based on the travel speed and the track shortest distance comprises:

for each camera, the distribution probability p is calculated using the following formula:

；

wherein ,mean value of walking speeds of all pedestrians identified for the current camera, +. >For the mean value of the track shortest distance of all the segmented moving tracks identified by the current camera, the +.>Is the standard deviation of the velocity +.>Is the standard deviation of distance>V is the walking speed of the target pedestrian in the current sectional moving track, and D is the track shortest distance of the current sectional moving track.

7. The method of claim 4, wherein prior to calculating the probability of distribution of each segmented movement track in each camera from the travel speed and the track shortest distance, the method further comprises:

judging whether the corresponding maximum distribution probability is smaller than a preset threshold value or not according to each camera;

and if the corresponding maximum distribution probability is smaller than a preset threshold value, eliminating the target track curve of the corresponding camera.

8. A tracking device for a pedestrian track, comprising:

the system comprises an acquisition module, a video acquisition module and a display module, wherein the acquisition module is used for acquiring a video set acquired by a plurality of cameras, the video set comprises a plurality of videos, and each video corresponds to one camera;

the identification module is used for identifying pedestrian frames in each video in the video set;

the generation module is used for generating a segmented movement track of the target pedestrian based on the pedestrian frame;

The coupling module is used for acquiring the spatial characteristics of the cameras, coupling the plurality of segmented movement tracks according to the spatial characteristics, and generating the complete movement track of the target pedestrian.

9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; wherein:

a memory for storing a computer program;

a processor for performing the steps of the method of any one of claims 1 to 7 by running a program stored on a memory.

10. A storage medium comprising a stored program, wherein the program when run performs the steps of the method of any of the preceding claims 1 to 7.