CN111401143A - Pedestrian tracking system and method - Google Patents

Abstract

The invention provides a pedestrian tracking system and a pedestrian tracking method, and relates to the technical field of computer vision. Determining that a first frame of a plurality of video frames includes a target frame of a target object; for a subsequent frame except the first frame in the plurality of video frames, determining a current target frame including a target object in the current frame according to the determined target frame; inputting the current target frame into a pre-trained VGG-16 network, and acquiring a candidate feature map of the target frame in the image; inputting the candidate feature map into a pre-trained RPN network to obtain a plurality of target candidate regions; performing pooling operation on the characteristics of the target candidate regions through a plurality of convolution kernels with different sizes to obtain a plurality of interested regions aiming at the target object; performing full-link operation on the characteristics of the multiple interested areas, distinguishing a target from a background, and obtaining multiple tracking affine frames of a target object; and performing non-maximum suppression on the plurality of tracking affine frames to obtain a tracking result of the target object of the current frame.

Description

A pedestrian tracking system and method

technical field

The invention relates to the technical field of computer vision, in particular to a pedestrian tracking system and method.

Background technique

Pedestrian tracking technology uses computer vision technology to identify and track pedestrian targets in videos and images. The pedestrian identification and tracking project has been listed as a key research project by many countries. This project is so valued because its technology is advanced and covers a wide range: in the field of national defense and military, the technology can be used for battlefield detection, target tracking and precision guidance; In the field of urban transportation, the technology can be used in intelligent transportation, violation detection and driverless driving, etc.; in the field of social security, the technology can be used in the monitoring of human flow, etc.

Numerous pedestrian tracking methods and apparatuses are disclosed in the prior art. Although many popular neural network techniques are used in these systems and methods, there is no special solution for accurate localization of deformable targets.

SUMMARY OF THE INVENTION

Aiming at the deficiencies of the prior art, the present invention provides a pedestrian tracking system and method.

The technical scheme adopted by the present invention is:

In one aspect, the present invention provides a pedestrian tracking system including a memory and a processor;

the memory is used to store computer-executable instructions;

The processor is configured to execute the executable instructions to determine that the first frame in the multiple video frames includes the target frame of the target object; for subsequent frames in the multiple video frames except the first frame, according to the determined The target frame determines the current target frame including the target object in the current frame; the current target frame is input into the pre-trained VGG-16 network to obtain the candidate feature map of the target frame in the image; the candidate feature map is input into the pre-trained Multiple target candidate regions are obtained in the RPN network of the The features of the full link operation are performed to distinguish the target and the background, and multiple tracking affine frames of the target object are obtained; and the non-maximum value suppression of the multiple tracking affine frames is performed to obtain the tracking result of the target object of the current frame.

On the other hand, the present invention also provides a pedestrian tracking method, which is implemented by the above-mentioned pedestrian tracking system, and the method includes the following steps:

Step 1: determine that the first frame in the multiple video frames includes the target frame of the target object;

Step 2: For subsequent frames except the first frame, determine the current target frame including the target object in the current frame according to the determined target frame;

Step 3: Adjust the determined target frame to a fixed size and input it into the pre-trained VGG-16 network, obtain the candidate feature map of the target frame in the current frame, and design a loss function.

Step 4: Input the candidate feature map into the pre-trained RPN network to obtain multiple target candidate regions;

The target candidate region is a region where multiple shapes and positions of the target object in the current frame exist simultaneously.

Step 5: Perform a pooling operation on the features of the multiple target candidate regions through multiple convolution kernels of different sizes to obtain multiple regions of interest for the target object;

The multiple convolution kernels of different sizes include three convolution kernels for briefly describing different deformations of the target object.

Step 6: Perform a full-link operation on the features of the multiple regions of interest, distinguish the target and the background, compare the multiple tracking affine frames with the reference target frame, and obtain the affine tracking frame with the largest overlapping area, Thereby obtained multiple tracking affine frames of the target object;

Step 7: performing non-maximum suppression on the multiple tracking affine frames to obtain the tracking result of the target object in the current frame;

Step 8: Determine whether the number of the next frame of the current image is less than the total number of video frames, if not, end immediately, if it is, go back to step 2, and track the next frame of image until all video frames are tracked.

The beneficial effects produced by the above technical solutions are:

The present application uses the affine transformation parameter information of the previous frame of image to crop the current target image, narrow the search range, and improve the efficiency of the algorithm. In addition, during the pooling operation, convolution kernels of different sizes and shapes are applied to initially simulate the deformation of the target, which helps to extract the target position more accurately.

Description of drawings

FIG. 1 is a block diagram of an embodiment of the present invention implemented using a computer architecture.

FIG. 2 is a flowchart of a pedestrian tracking algorithm according to an embodiment of the present invention.

FIG. 3 is a schematic block diagram of a process flow of an embodiment of the present invention.

FIG. 4 is a comparison diagram of the effects of horizontal NMS and affine transformation NMS according to an embodiment of the present invention.

FIG. 5 is a tracking result diagram of an embodiment of the present invention.

FIG. 6 is a network structure of VGG-16 according to an embodiment of the present invention.

Detailed ways

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

In one aspect, the present invention provides a pedestrian tracking system including a memory and a processor;

the memory is used to store computer-executable instructions;

The processor is configured to execute the executable instructions to determine that the first frame in the multiple video frames includes the target frame of the target object; for subsequent frames in the multiple video frames except the first frame, according to the determined The target frame determines the current target frame including the target object in the current frame; the current target frame is input into the pre-trained VGG-16 network to obtain the candidate feature map of the target frame in the image; the candidate feature map is input into the pre-trained Multiple target candidate regions are obtained in the RPN network of the The features of the full link operation are performed to distinguish the target and the background, and multiple tracking affine frames of the target object are obtained; and the non-maximum value suppression of the multiple tracking affine frames is performed to obtain the tracking result of the target object of the current frame.

As shown in FIG. 1 , it shows a schematic structural diagram of an electronic system 600 suitable for implementing embodiments of the present disclosure. The electronic system shown in FIG. 1 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 1 , electronic system 600 may include a processing device (eg, central processing unit, graphics processor, etc.) 601 that may be loaded into random access according to a program stored in read only memory (ROM) 602 or from storage device 608 Various appropriate actions and processes are executed by the programs in the memory (RAM) 603 . In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored. The processing device 601 , the ROM 602 , and the RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to bus 604 .

Typically, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 607 of a computer, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. Communication means 609 may allow electronic system 600 to communicate wirelessly or by wire with other devices to exchange data. While FIG. 6 illustrates electronic system 600 having various devices, it should be understood that not all of the illustrated devices are required to be implemented or available. More or fewer devices may alternatively be implemented or provided. Each block shown in FIG. 6 may represent one device, or may represent multiple devices as required.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication device 609 , or from the storage device 608 , or from the ROM 602 . When the computer program is executed by the processing apparatus 601, the above-described functions defined in the methods of the embodiments of the present disclosure are executed.

It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Rather, in embodiments of the present disclosure, a computer-readable signal medium may include a data signal in baseband or propagated as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

The above-mentioned computer-readable medium may be contained in the above-mentioned electronic system (also referred to herein as "a pedestrian tracking system based on affine multi-task regression"); or may exist alone without being assembled into the electronic system. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic system: 1) determine a target frame in which the previous frame of the plurality of video frames includes the target object 2) Determine the current target frame including the target object in the current frame according to the determined target frame; 3) Input the current target frame into the pre-trained first neural network, obtain the current frame in the 4) Input the candidate feature map into the pre-trained second neural network to obtain multiple target candidate regions; 5) Perform a pooling operation on the features of the multiple target candidate regions , and obtain multiple regions of interest for the target object; 6) perform a full-link operation on the features of the multiple regions of interest to distinguish the target and the background, thereby obtaining multiple tracking affine frames of the target object; and 7) performing non-maximum suppression on the multiple tracking affine frames to obtain the tracking result of the target object in the current frame.

On the other hand, the present invention also provides a pedestrian tracking method based on affine multi-task regression, as shown in FIG. 2 , which is implemented by the above-mentioned pedestrian tracking system based on affine multi-task regression, and the method includes the following steps:

Step 1: determine that the first frame in the multiple video frames includes the target frame of the target object;

Initialize the size of the original image. Let the original image size be m×n (unit: pixel). When t=1, manually mark the position of the target box for this frame. Mark the center position coordinates of the target frame as (cx, cy), where t represents the t-th frame image, t is a positive integer, cx, cy are the horizontal and vertical coordinates of the center position of the target frame, and the target frame includes the object to be tracked, for example Reference numeral 301 is shown in FIG. 3 .

Initialize affine transformation parameters: U ₁ =[ r 1, r 2, r 3, r 4, r 5, r 6] ^T .

Step 2: determine the current target frame including the target object in the current frame according to the determined target frame;

In this embodiment, the current target frame including the target object in the current frame is determined according to the determined target frame. Specifically, the input frame t (t>2) picture is cropped, and the target frame of frame t is determined with the center coordinates (cx, cy) of the target frame tracked or identified by frame t-1 as the center. For example, assuming that the lengths of the two sides of the circumscribed rectangle of the target frame in frame t-1 are denoted as: a, b, then on the image of frame t, the center point of the target in frame t-1 (cx, cy ) as the center, and crop out a picture of size (2a)×(2b), such as the rectangular box labeled 302 in Figure 3. In this application, the purpose of taking the center point of the target in the previous frame as the center is to make the cropped picture contain the target information. This is because the coordinates of the center point of the target in two adjacent frames do not change much, as long as the center point near the center point. position, and crop out a sub-image large enough to contain the target to be tracked.

Step 3: Adjust the determined target frame to a fixed size and input it into the pre-trained VGG-16 network, obtain the candidate feature map of the target frame in the current frame, and design a loss function.

Adjust the cropped target frame to a fixed size and send it to the pre-trained neural network, such as the above-mentioned VGG-16 network, to obtain the feature map of the image after the fifth layer convolution in the network, that is , obtain the candidate feature map of the target frame in the image. For example, the reference numeral 303 in FIG. 3 is shown.

In this embodiment, the accuracy and operation efficiency of the system are comprehensively considered, and the classic VGG-16 network structure is used to implement the various embodiments of the present application. Figure 1 shows an exemplary VGG-16 network structure. As shown in Figure 1, the network structure includes 13 convolutional layers (201) and 3 fully connected layers (203). Specifically, as shown in Figure 1, a convolutional layer is first constructed with 3×3 filters with a stride of 1, assuming that the network input size is m×n×3 (m and n are positive integers), in order to ensure the convolution The first two dimensions of the subsequent feature matrix are the same as the first two dimensions of the input matrix, namely: m×n. Append a circle of 0s to the input matrix. Change the dimension of the input matrix to (m+2)×(n+2), and then 3×3 convolution. In this way, the first two dimensions of the feature matrix after convolution are still: m×n. A max pooling layer 202 is then constructed with a 2×2 filter with a stride of 2. Then do three convolution operations with 256 of the same filters, then pool again, then convolve three more times, then pool again. The activation function used above is the existing relu function. After several rounds of operations in this way, the finally obtained 7×7×512 feature map is fully connected (ie, fully connected layer 203) to obtain 4096 units, and then activated by the softmax function (ie, as shown in the figure). 204), which outputs the results identified from 1000 objects. Although a specific VGG-16 network structure is given here, those skilled in the art should understand that other network structures can also be used without departing from the teachings of the present application.

After re-architecting the above network, train it by using the ImageNet dataset. The ImageNet dataset is divided into training set and test set. This dataset corresponds to, for example, 1000 categories. Each data has a corresponding label vector, and each label vector corresponds to a different category. This application does not care about the specific classification of the input image, but only applies this dataset to train the weights of the VGG-16 network. Specifically, the above ImageNet training set is adjusted to a size of 224×224×3, and then input to the VGG-16 network to train the network to obtain the weight parameter information of each layer or unit of the network. Then, the pre-determined test data set and the label vector of the corresponding category are input into the VGG-16 network structure obtained by training. The size of the test data set may also be, for example, 224×224×3. By inputting the above test data set and the label vector of the corresponding category into the VGG-16 network, the output results of the VGG-16 network can be detected, and the detected results 16 The parameters (weights) of the network are adjusted. Repeat the above steps until the test accuracy reaches a predetermined standard, for example, the accuracy is more than 98%.

Optionally, it is necessary to first calculate the loss and regression, and optimize the affine transformation parameters. The loss function design for the entire network of the above VGG-16 can be expressed as:

The loss function of the VGG-16 network is expressed as:

（1）

(1)

where α ₁ and α ₂ are learning rates. p is the logarithmic loss of category tc , and the formula is shown in (2).

L _c ( p , tc ) = - logp _tc (2)

i represents the sequence number of the regression box where the loss is being calculated;

tc means category label, for example: tc =1 means target, tc =0 means background;

x , y , w , h and other variables are used in combination to represent abscissa/ordinate/width/height respectively.

是预测到的目标框元组，包括中心点横坐标、纵坐标、宽和高； The parameter v _i = ( v _{x ,} v _{y ,} v _{w ,} v _h ) is a true rectangular bounding box tuple, including the center point abscissa, ordinate, width and height;

is the predicted target frame tuple, including the center point abscissa, ordinate, width and height;

u _i = ( r1 , r2 , r3 , r4 , r5 , r6 ) is the affine parameter tuple of the real target area;

为预测到目标区域的仿射参数元组；

is the affine parameter tuple that predicts the target area;

r1，r2，r3，r4，r5，r6）为真实目标区域的仿射变换固定结构的六个分量的值；

r 1, r 2, r 3, r 4, r 5, r 6) are the values of the six components of the affine transformation fixed structure of the real target area;

r1^*，r2^*，r3^*，r4^*，r5^*，r6^*）为预测到目标区域的仿射变换固定结构的六个分量的值；

r 1 ^* , r 2 ^* , r 3 ^* , r 4 ^* , r 5 ^* , r 6 ^* ) are the six components of the fixed structure of the affine transformation predicted to the target region;

表示仿射边界框参数损失函数；

represents the affine bounding box parameter loss function;

表示矩形边界框参数损失函数；

represents the parameter loss function of the rectangular bounding box;

或者

,

定义为: Let ( w , w* ) denote

or

,

defined as:

(3)

(3)

(4)

(4)

where x is a real number.

Step 4: Input the candidate feature map into the pre-trained RPN network to obtain multiple target candidate regions;

The target candidate region is a region where multiple shapes and positions of the target object in the current frame exist simultaneously.

The above feature map obtained from the neural network is input into the RPN (Region Proposal Network) network, and candidate regions for obtaining multiple targets, such as 2000 candidate regions, are extracted. For example, the reference numeral 304 in FIG. 3 is shown. RPN is a network that generates multiple candidate regions of different sizes, which is different from the VGG-16 network. A candidate region is a region with multiple shapes and positions that may exist in the current frame target. The present application pre-estimates multiple regions that may exist in the algorithm, and then performs optimization regression on these regions to screen out more accurate tracking regions.

Step 5: Perform a pooling operation on the features of the multiple target candidate regions through multiple convolution kernels of different sizes to obtain multiple regions of interest for the target object;

The multiple convolution kernels of different sizes include three convolution kernels for briefly describing different deformations of the target object.

The features of these candidate regions of different sizes are pooled to obtain multiple regions of interest (ROI) for the target object. Here, considering the deformation of the target, multiple convolution kernels of different sizes are designed in the pooling layer, for example, three convolution kernels are designed: 7×7, 5×9 and 9×5. For example, the reference numeral 305 in FIG. 3 is shown. Multiple different pooling kernels can roughly describe the deformation of the target. For example: 7×7, 5×9 can describe people standing under different cameras, 9×5 can describe people’s actions such as bending over. Of course, pooling kernels of different sizes can also be designed according to different application scenarios.

Step 6: Perform a full-link operation on the features of the multiple regions of interest, distinguish the target and the background, compare the multiple tracking affine frames with the reference target frame, and obtain the affine tracking frame with the largest overlapping area, Thereby obtained multiple tracking affine frames of the target object;

The results of the above pooling, that is, the features of multiple regions of interest (ROI), are fully connected. Here, the full link operation is to concatenate multiple ROI features in sequence. For example, the reference numeral 306 in FIG. 3 is shown. Then, use the softmax function to score and compare the concatenated features to obtain the score of the target/background result of the compared target area. For example, if the score result is greater than a certain threshold, it is determined as the target area, otherwise, it is the background area.

Step 7: performing non-maximum suppression on the multiple tracking affine frames to obtain the tracking result of the target object in the current frame;

Non-maximum suppression is performed on the obtained affine region that is determined to be the target region (for example, the reference number 308 in Figure 3), and the tracking result of the t-th frame image is obtained, that is, the corresponding affine parameters and frame. For example, the reference numeral 309 in FIG. 3 is shown. In one embodiment, the multiple tracking affine frames can be compared with the reference target frame (that is, the target frame tracked in the previous frame) to obtain the affine tracking frame with the largest overlapping area as the final tracking result .

, 具有李群结构，ga（2）是对应于仿射李群GA（2）的李代数，矩阵G _j （

）是GA（2）的 生成元以及矩阵ga（2）的基。对于矩阵GA（2）的生成元为： In this paper, affine transformation is used to represent the geometric deformation of the target. The affine transformation parameter of the tracking result of the target area of the t -th frame is denoted as U _t , and its structure is: U _t = [ r 1, r 2, r 3, r 4, r 5, r 6] ^T . Corresponding affine transformation matrix

, has a Lie group structure, ga (2) is a Lie algebra corresponding to the affine Lie group GA (2), the matrix G _j (

) is the generator of GA (2) and the basis of the matrix ga (2). The generator for the matrix GA (2) is:

(5)

(5)

For Lie group matrices, the Riemann distance is defined as a matrix logarithm operation:

(6)

(6)

的内均值定义： where X and Y are elements of a Lie group matrix, giving the symmetric positive definite matrix of N

The inner mean definition of :

(7)

(7)

，q为常数； in

, q is a constant;

Perform non-maximum suppression on the above multiple tracking affine frames to obtain the tracking results of t frames of images. Multiple different target regions may be obtained through regression. In order to correctly obtain a detection algorithm with the highest accuracy, the present application adopts an affine transformation non-maximum suppression method to screen out the final tracking result. In addition, the design of the above loss function takes the affine deformation of the target into account, which improves the accuracy of predicting the target position.

The current object detection method, Non-Maximum Suppression (NMS), is widely used to post-process detection candidates. While estimating the axis-aligned bounding box and the skewed bounding box, normal NMS can be performed on the axis-aligned bounding box, or skewed NMS can be performed on the affine transformed bounding box, this application is called affine transform non-maximum suppression . In affine transformation non-maximum suppression, the traditional intersection point (IoU) computation is modified as the IoU between two affine bounding boxes. The effect of the algorithm is shown in Figure 4. In Figure 4, each frame numbered 401 is the candidate tracking frame before non-maximum suppression, the frame numbered 402 is the tracking frame obtained after normal NMS suppression, and the frame numbered 403 is a simulation of this application. The tracking box obtained by the non-maximum suppression of the injection transform. It can be seen that the tracking frame obtained in this application is more accurate.

Step 8: Determine whether the number of t+1 is less than the total number of video frames, if it is, go back to step 2, and perform the tracking of the t+1th frame image. The algorithm ends when all frames of the video are tracked. Part of the tracking result frame is shown as the black frame indicated by the arrows 501, 502, 503, 504 in Figure 5.

In this application, the affine transformation parameter information of the previous frame of image is used to crop the current target image, narrow the search range, and improve the efficiency of the algorithm. In addition, the cropped image is first input to the VGG-16 network to calculate the features, and then input to the RPN network, which avoids the repeated calculation of feature extraction and improves the efficiency of the algorithm. In addition, during the pooling operation, convolution kernels of different sizes and shapes are applied to initially simulate the deformation of the target, which helps to extract the target position more accurately. In this application, the features output by the highest layer of the above network are used as the semantic model, and the affine transformation result is used as the spatial model, and the two complement each other's advantages, because the features of the highest layer contain more semantic information and less space information. Furthermore, the above-mentioned multi-task loss function including affine transformation parameter regression optimizes the network performance.

In the above-mentioned pedestrian tracking system, the candidate regions obtained from the RPN network are regions of multiple shapes and positions where the target object in the current frame exists. In addition, in the step 5, the features of the multiple target candidate regions are pooled through multiple convolution kernels of different sizes to obtain multiple regions of interest for the target object. For example, multiple convolution kernels of different sizes include three convolution kernels to briefly describe different deformations of the target. In particular, as mentioned above, considering the deformation of the target, multiple convolution kernels of different sizes are designed in the pooling layer, for example, three convolution kernels are designed: 7×7, 5×9 and 9×5 . Multiple different pooling kernels can roughly describe the deformation of the target. For example: 7×7, 5×9 can describe people standing under different cameras, 9×5 can describe people’s actions such as bending over. Of course, pooling kernels of different sizes can also be designed according to different application scenarios.

The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, and should also cover, without departing from the above-mentioned inventive concept, the above-mentioned Other technical solutions formed by any combination of technical features or their equivalent features. For example, a technical solution is formed by replacing the above features with the technical features disclosed in the embodiments of the present disclosure (but not limited to) with similar functions.

Claims (4)

1. a pedestrian tracking system, is characterized in that: comprise memory and processor; the memory is used to store computer-executable instructions; The processor is configured to execute the executable instructions to determine that the first frame in the multiple video frames includes the target frame of the target object; for subsequent frames in the multiple video frames except the first frame, according to the determined The target frame determines the current target frame including the target object in the current frame; the current target frame is input into the pre-trained VGG-16 network to obtain the candidate feature map of the target frame in the image; the candidate feature map is input into the pre-trained Multiple target candidate regions are obtained in the RPN network of the The features of the full link operation are performed to distinguish the target and the background, and multiple tracking affine frames of the target object are obtained; and the non-maximum value suppression of the multiple tracking affine frames is performed to obtain the tracking result of the target object of the current frame. 2. A pedestrian tracking method, realized by a pedestrian tracking system described in claim 1, characterized in that, comprising the following steps: Step 1: determine that the first frame in the multiple video frames includes the target frame of the target object; Step 2: For subsequent frames except the first frame, determine the current target frame including the target object in the current frame according to the determined target frame; Step 3: Adjust the determined target frame to a fixed size and input it into the pre-trained VGG-16 network, obtain the candidate feature map of the target frame in the current frame, and design a loss function; Step 4: Input the candidate feature map into the pre-trained RPN network to obtain multiple target candidate regions; Step 5: Perform a pooling operation on the features of the multiple target candidate regions through multiple convolution kernels of different sizes to obtain multiple regions of interest for the target object; Step 6: Perform a full-link operation on the features of the multiple regions of interest, distinguish the target and the background, compare the multiple tracking affine frames with the reference target frame, and obtain the affine tracking frame with the largest overlapping area, Thereby obtained multiple tracking affine frames of the target object; Step 7: performing non-maximum suppression on the multiple tracking affine frames to obtain the tracking result of the target object in the current frame; Step 8: Determine whether the number of the next frame of the current image is less than the total number of video frames, if not, end immediately, if it is, go back to step 2, and track the next frame of image until all video frames are tracked. 3 . The pedestrian tracking method according to claim 2 , wherein the target candidate region in step 4 is a region where multiple shapes and positions of the target object in the current frame exist simultaneously. 4 . 4 . The pedestrian tracking method according to claim 2 , wherein in step 5, the multiple convolution kernels of different sizes include three convolution kernels for roughly describing different deformations of the target object. 5 .

Images