WO2021043168A1 - Person re-identification network training method and person re-identification method and apparatus - Google Patents

Abstract

Provided by the present application are a person re-identification network training method and a person re-identification method and apparatus. The present application relates to the field of artificial intelligence, and relates in particular to the field of computer vision. The method comprises: obtaining M training images and annotation data of the M training images; performing initialization processing on a network parameter of a person re-identification network so as to obtain an initial value of the network parameter of the person re-identification network; inputting a batch of training images among the M training images into the person re-identification network to perform feature extraction so as to obtain a feature vector of each training image in the batch of training images; then determining a loss function according to the feature vectors of the batch of training images; and obtaining, according to a function value of the loss function, a person re-identification network that meets a preset requirement. In the present application, a person re-identification network having good performance may be trained when data is labelled using a single image photographing device.

Description

Pedestrian re-identification network training method, pedestrian re-identification method and device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 5, 2019, the application number is 201910839017.9, and the application title is "Pedestrian Re-identification Network Training Method, Pedestrian Re-identification Method and Device", and its entire contents Incorporated in this application by reference.

Technical field

This application relates to the field of computer vision, and more specifically, to a method for training a pedestrian re-recognition network, a method and device for pedestrian re-recognition.

Background technique

Computer vision is an inseparable part of various intelligent/autonomous systems in various application fields, such as manufacturing, inspection, document analysis, medical diagnosis, and military. It is about how to use cameras/image capturing equipment and computers. To get the knowledge we need, the data and information of the subject. To put it vividly, it is to install eyes (cameras/image capture equipment) and brains (algorithms) on the computer to replace the human eyes to identify, track and measure targets, so that the computer can perceive the environment. Because perception can be seen as extracting information from sensory signals, computer vision can also be seen as a science that studies how to make artificial systems "perceive" from images or multi-dimensional data. In general, computer vision is to use various imaging systems to replace the visual organs to obtain input information, and then the computer replaces the brain to complete the processing and interpretation of the input information. The ultimate research goal of computer vision is to enable computers to observe and understand the world through vision like humans, and have the ability to adapt to the environment autonomously.

The surveillance field often involves the problem of pedestrian re-identification. Pedestrian re-identification (ReID) can also be called pedestrian re-identification. Pedestrian re-identification is a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video sequence.

The traditional scheme is generally training data and annotation data across image capturing equipment to train the pedestrian re-recognition network so that the pedestrian re-recognition network can distinguish images of different pedestrians, and then perform pedestrian recognition. However, the training data in the traditional scheme includes images of the same pedestrian captured by different image capturing devices. For such images captured by different image capturing devices, manual annotation is required to associate the same pedestrian images captured by different image capturing devices. (That is, associate pedestrians across image capture devices). However, in many scenarios, it is very difficult to associate pedestrians across image capturing devices, especially when the number of people increases and the number of image capturing devices increases, the difficulty of performing cross image capturing devices is also greatly increased. The economic cost of data labeling is high and time consuming.

Summary of the invention

This application provides a pedestrian re-recognition network training method, pedestrian re-recognition method and device, so as to train a pedestrian re-recognition network with better performance in the case of single-image shooting equipment labeling data.

In the first aspect, a training method for a pedestrian re-identification network is provided, the method includes:

Step 1: Obtain training data;

Wherein, the training data in step 1 includes M training images and labeled data of M training images, and M is an integer greater than 1;

Step 2: Initialize the network parameters of the pedestrian re-identification network to obtain the initial values of the network parameters of the pedestrian re-identification network;

Repeat the following steps 3 to 5 until the pedestrian re-identification network meets the preset requirements;

Step 3: Input a batch of training images from the M training images to the pedestrian recognition network for feature extraction, and obtain the feature vector of each training image in the batch of training images;

Step 4: Determine the function value of the loss function according to the feature vector of a batch of training images;

Step 5: Update the network parameters of the pedestrian re-identification network according to the function value of the loss function.

In the above step 1, in the M training images of the training data, each training image includes a pedestrian, and the annotation data of each training image includes the bounding box where the pedestrian in each training image is located and the pedestrian identification information. Different pedestrians Corresponding to different pedestrian identification information, among the M training images, the training images with the same pedestrian identification information come from the same image capturing device. The M training images can be all the training images used in the training of the pedestrian re-recognition network. In the specific training process, a batch of training images of the M training images can be selected and input into the pedestrian re-recognition network each time. deal with.

The aforementioned image capturing device may specifically be a device capable of acquiring images of pedestrians, such as a video camera and a camera.

The pedestrian identification information in step 1 above can also be called pedestrian identification information, which is a type of information used to identify the identity of a pedestrian. Each pedestrian can correspond to unique pedestrian identification information. There are many ways to express the pedestrian identification information. , As long as the identity information of the pedestrian can be indicated, for example, the pedestrian identification information may specifically be a pedestrian identity (identity, ID), that is, a unique ID can be assigned to each pedestrian.

In step 2 above, the network parameters of the pedestrian re-identification network can be randomly set to obtain the initial values of the network parameters of the pedestrian re-identification network.

In the above step 3, the above batch of training images may include N anchor point images, where the N anchor point images are any N training images in the above batch of training images, and each of the N anchor point images Each anchor point image corresponds to a most difficult positive sample image, a first most difficult negative sample image and a second most difficult negative sample image.

The following describes the most difficult positive sample image corresponding to each anchor point image, the first most difficult negative sample image, and the second most difficult negative sample image.

The most difficult positive sample image corresponding to each anchor point image: the training image that has the same pedestrian identification information as each anchor point image and the farthest distance from the feature vector of each anchor point image in the above batch of training images ；

The first most difficult negative sample image corresponding to each anchor point image: the above batch of training images and each anchor point image come from the same image capture device, and are different from the pedestrian identification information of each anchor point image and are different from each anchor point image. The training image with the closest distance between the feature vectors of the anchor image;

The second most difficult negative sample image corresponding to each anchor point image: the above batch of training images and each anchor point image come from different image capture equipment, and are different from the pedestrian identification information of each anchor point image and are different from each anchor point image. The training image with the closest distance between the feature vectors of the point image.

In the above step 4, the function value of the loss function is obtained by averaging the function values of the N first loss functions. Wherein, the function value of each first loss function in the above N first loss functions is calculated according to the first difference and the second difference corresponding to each of the N anchor point images.

The above N is a positive integer, and the above N is less than M. When N=1, there is only one function value of the first loss function. At this time, the function value of the first loss function can be directly used as the function value of the loss function in step 4.

Optionally, the function value of each of the foregoing first loss functions is the sum of the first difference and the second difference corresponding to each anchor point image.

Optionally, the function value of each of the foregoing first loss functions is the sum of the first difference, the second difference, and other constant items corresponding to each anchor point image.

The meaning of the first difference value and the second difference value and the respective distances forming the first difference value and the second difference value will be described below.

The first difference corresponding to each anchor image: the difference between the distance of the most difficult positive sample corresponding to each anchor image and the distance of the second most difficult negative sample corresponding to each anchor image;

The second difference value corresponding to each anchor point image: the difference between the distance of the second most difficult negative sample corresponding to each anchor point image and the distance of the first most difficult negative sample corresponding to each anchor point image;

The distance of the most difficult positive sample corresponding to each anchor point image: the distance between the feature vector of the most difficult positive sample image corresponding to each anchor point image and the feature vector of each anchor point image;

The second most difficult negative sample distance corresponding to each anchor point image: the distance between the feature vector of the second most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image;

The distance of the first most difficult negative sample corresponding to each anchor point image: the distance between the feature vector of the first most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image.

In addition, in this application, that several training images are from the same image capturing device means that these several training images are captured by the same image capturing device.

In this application, the most difficult negative sample images from different image capturing devices and the same image capturing device are considered in the process of constructing the loss function, and the first difference and the second difference are reduced as much as possible during the training process. Small, which can eliminate as much as possible the interference of the image capturing device's own information on the image information, so that the trained pedestrian re-recognition network can more accurately extract features from the image.

Specifically, in the training process of the pedestrian re-recognition network, the network parameters of the pedestrian re-recognition network are optimized to make the first difference and the second difference as small as possible, so that the distance between the most difficult positive sample and the second hardest The difference between the distance of the negative sample and the distance between the second most difficult negative sample and the distance of the first most difficult negative sample are as small as possible, so that the pedestrian re-recognition network can distinguish the most difficult image from the second most difficult negative as much as possible. The characteristics of the sample image, and the characteristics of the second most difficult negative sample image and the first most difficult negative sample image, so that the trained pedestrian re-recognition network can perform feature extraction on the image better and more accurately.

With reference to the first aspect, in some implementations of the first aspect, the pedestrian re-identification network meets preset requirements, including: when at least one of the following conditions (1) to (3) is met, the pedestrian re-identification network Meet the preset requirements:

(1) The number of training times of the pedestrian re-identification network is greater than or equal to the preset number;

(2) The function value of the loss function is less than or equal to the preset threshold;

(3) The recognition performance of the pedestrian re-identification network meets the preset requirements.

The above preset threshold can be flexibly set by experience. When the preset threshold is set too large, the pedestrian recognition effect of the trained pedestrian re-recognition network may not be good enough, and when the preset threshold is set too small, the loss of the function during training The function value may be difficult to converge.

Optionally, the value range of the foregoing preset threshold is [0, 0.01].

Specifically, the value of the foregoing preset threshold may be 0.01.

With reference to the first aspect, in some implementations of the first aspect, the function value of the loss function is less than or equal to a preset threshold, including: the first difference is less than the first preset threshold, and the second difference is less than the second preset Set the threshold.

The above-mentioned first preset threshold and second preset threshold can also be determined based on experience. When the first preset threshold and the second preset threshold are set too large, the pedestrian recognition effect of the trained pedestrian re-recognition network may not be good enough , And when the first preset threshold and the second preset threshold are set too small, the function value of the loss function may be difficult to converge during training.

Optionally, the value range of the foregoing first preset threshold is [0, 0.4].

Optionally, the value range of the foregoing first preset threshold is [0, 0.4].

Specifically, both the first preset threshold and the second threshold may be 0.1.

With reference to the first aspect, in some implementations of the first aspect, the above-mentioned M training images are training images from a plurality of image capturing devices, wherein the labeled data of the training images from different image capturing devices are separately labeled .

That is to say, the images of each image capturing device can be marked separately, regardless of whether the same pedestrian will appear between different image capturing devices, specifically, if the multiple images captured by the image capturing device A include pedestrians. X, then, after marking the M training images captured by image capture device A, there is no need to look for images of pedestrian X from the images captured by other image capture devices, thus avoiding the need to use different image capture devices The process of finding the same pedestrian in the captured image can save a lot of marking time and reduce the complexity of marking.

In a second aspect, a pedestrian re-recognition method is provided. The method includes: acquiring an image to be recognized; using a pedestrian re-recognition network to process the image to be recognized to obtain a feature vector of the image to be recognized, wherein the pedestrian re-recognition network is based on the above The training method of the first aspect is obtained by training; according to the feature vector of the image to be recognized and the feature vector of the existing pedestrian image, the recognition result of the image to be recognized is obtained.

In this application, because the pedestrian re-recognition network trained by the training method of the first aspect can better perform feature extraction, the pedestrian re-recognition network trained by the training method of the first aspect can be used for pedestrian recognition. Good pedestrian recognition results.

In combination with the second aspect, in some implementations of the second aspect, the above-mentioned comparison is performed based on the feature vector of the image to be recognized with the feature vector of the existing pedestrian image to obtain the recognition result of the image to be recognized, including: outputting the target pedestrian Image and attribute information of the target pedestrian image.

The above-mentioned target pedestrian image may be a pedestrian image whose feature vector is most similar to the feature vector of the image to be recognized in the existing pedestrian image, and the attribute information of the target pedestrian image includes the shooting time and shooting location of the target pedestrian image. In addition, the attribute information of the target pedestrian image may also include the identity information of the pedestrian and the like.

In a third aspect, a training device for a pedestrian re-identification network is provided. The training device for the pedestrian re-identification network includes various modules for executing the method in the above-mentioned first aspect.

In a fourth aspect, a pedestrian re-identification device is provided. The device includes modules for executing the method in the second aspect.

In a fifth aspect, a training device for a pedestrian re-identification network is provided. The device includes: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed , The processor is configured to execute the method in the above-mentioned first aspect.

In a sixth aspect, a pedestrian re-identification device is provided. The device includes: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed, the The processor is used to execute the method in the second aspect described above.

In a seventh aspect, a computer device is provided, and the computer device includes the training device for the pedestrian re-identification network in the third aspect.

In the above seventh aspect, the computer device may specifically be a server or a cloud device or the like.

In an eighth aspect, an electronic device is provided, and the electronic device includes the pedestrian re-identification device of the fourth aspect.

In the above eighth aspect, the electronic device may specifically be a mobile terminal (for example, a smart phone), a tablet computer, a notebook computer, an augmented reality/virtual reality device, a vehicle-mounted terminal device, and so on.

In a ninth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores program code, and the program code includes instructions for executing steps in any one of the first aspect or the second aspect.

In a tenth aspect, a computer program product containing instructions is provided. When the computer program product runs on a computer, the computer executes any one of the methods in the first aspect or the second aspect.

In an eleventh aspect, a chip is provided. The chip includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface, and executes any one of the first aspect or the second aspect. Kind of method.

Optionally, as an implementation manner, the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory. When the instructions are executed, the The processor is configured to execute any one of the methods in the first aspect or the second aspect described above.

The above-mentioned chip may specifically be a field programmable gate array FPGA or an application-specific integrated circuit ASIC.

It should be understood that in this application, the method in the first aspect may specifically refer to the method in the first aspect and any one of the various implementation manners in the first aspect, and the method in the second aspect may specifically refer to the second aspect. Aspect and the method in any one of the various implementation manners in the second aspect.

Description of the drawings

FIG. 1 is a schematic structural diagram of a system architecture provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of re-identifying pedestrians using the convolutional neural network model provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a chip hardware structure provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a system architecture provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a possible application scenario of an embodiment of the present application;

FIG. 6 is a schematic diagram of the overall flow of the training method of the pedestrian re-identification network according to an embodiment of the present application;

FIG. 7 is a schematic flowchart of a method for training a pedestrian re-identification network according to an embodiment of the present application;

FIG. 8 is a schematic diagram of the process of determining the function value of the loss function;

FIG. 9 is a schematic flowchart of a pedestrian re-identification method according to an embodiment of the present application;

FIG. 10 is a schematic block diagram of a training device for a pedestrian re-identification network according to an embodiment of the present application;

FIG. 11 is a schematic block diagram of a training device for a pedestrian re-identification network according to an embodiment of the present application;

Fig. 12 is a schematic block diagram of a pedestrian re-identification device according to an embodiment of the present application;

Fig. 13 is a schematic block diagram of a pedestrian re-identification device according to an embodiment of the present application.

detailed description

The following describes the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The solution of this application can be applied in the fields of city monitoring, safe city and so on.

Specifically, the present application can be applied to the scene of tracing people in the intelligent monitoring system, and the application in this scene will be introduced below.

Intelligent monitoring system to find people:

Take an intelligent monitoring system deployed in a park as an example. The intelligent monitoring system can collect the images of pedestrians captured by various image capturing devices to form an image library. Next, the pedestrian re-recognition network (also called a pedestrian re-recognition model) can be trained using the images in the image library to obtain a trained pedestrian re-recognition network.

Next, the trained pedestrian re-recognition network can be used to extract the feature vector of the collected pedestrian image. When a person's whereabouts are suspicious, or there are other situations where the pedestrian needs to be tracked across the camera, the feature vector of the pedestrian image collected by the pedestrian re-recognition network can be compared with the feature vector of the image in the image library, and the feature vector is returned the most similar Pedestrian images, and provide basic information such as the shooting time and location of these images. After the follow-up check and screening, the tracing process can be completed.

In the solution of this application, the pedestrian re-identification network may be a neural network (model). In order to better understand the solution of this application, the following first introduces the related terms and concepts of the neural network.

(1) Neural network

A neural network can be composed of neural units. A neural unit can refer to _{an arithmetic unit that takes x s} and intercept 1 as inputs. The output of the arithmetic unit can be as shown in formula (1):

Among them, s=1, 2,...n, n is a natural number greater than 1, W _s is the weight of x _s , and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to perform non-linear transformation of the features in the neural network, thereby converting the input signal in the neural unit into an output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected with the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be a region composed of several neural units.

(2) Deep neural network

Deep neural network (DNN), also known as multi-layer neural network, can be understood as a neural network with multiple hidden layers. The DNN is divided according to the positions of different layers. The neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the number of layers in the middle are all hidden layers. The layers are fully connected, that is to say, any neuron in the i-th layer must be connected to any neuron in the i+1th layer.

其中，

是输入向量，

是输出向量，

是偏移向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量

经过如此简单的操作得到输出向量

由于DNN层数多，系数W和偏移向量

的数量也比较多。这些参数在DNN中的定义如下所述：以系数W为例，假设在一个三层的DNN中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为

上标3代表系数W所 在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。 Although DNN looks complicated, it is not complicated as far as the work of each layer is concerned. Simply put, it is the following linear relationship expression:

among them,

Is the input vector,

Is the output vector,

Is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just the input vector

After such a simple operation, the output vector is obtained

Due to the large number of DNN layers, the coefficient W and the offset vector

The number is also relatively large. The definition of these parameters in DNN is as follows: Take the coefficient W as an example, suppose that in a three-layer DNN, the linear coefficients from the fourth neuron in the second layer to the second neuron in the third layer are defined as

The superscript 3 represents the number of layers where the coefficient W is located, and the subscript corresponds to the output third-level index 2 and the input second-level index 4.

In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as

It should be noted that there is no W parameter in the input layer. In deep neural networks, more hidden layers make the network more capable of portraying complex situations in the real world. In theory, a model with more parameters is more complex and has a greater "capacity", which means it can complete more complex learning tasks. Training the deep neural network is also the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by the vector W of many layers).

(3) Convolutional neural network

Convolutional neural network (convolutional neuron network, CNN) is a deep neural network with a convolutional structure. The convolutional neural network contains a feature extractor composed of a convolutional layer and a sub-sampling layer. The feature extractor can be regarded as a filter. The convolutional layer refers to the neuron layer that performs convolution processing on the input signal in the convolutional neural network. In the convolutional layer of a convolutional neural network, a neuron can only be connected to a part of the neighboring neurons. A convolutional layer usually contains several feature planes, and each feature plane can be composed of some rectangularly arranged neural units. Neural units in the same feature plane share weights, and the shared weights here are the convolution kernels. Sharing weight can be understood as the way of extracting image information has nothing to do with location. The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, and at the same time reduce the risk of overfitting.

(4) Residual network

The residual network is a deep convolutional network proposed in 2015. Compared with the traditional convolutional neural network, the residual network is easier to optimize and can increase the accuracy by adding a considerable depth. The core of the residual network is to solve the side effect (degradation problem) caused by increasing the depth, so that the network performance can be improved by simply increasing the network depth. The residual network generally contains many sub-modules with the same structure. A residual network (ResNet) is usually used to connect a number to indicate the number of times the sub-module is repeated. For example, ResNet50 means that there are 50 sub-modules in the residual network.

(6) Classifier

Many neural network structures have a classifier at the end to classify objects in the image. The classifier is generally composed of a fully connected layer and a softmax function (which can be called a normalized exponential function), and can output different types of probabilities according to the input.

(7) Loss function

In the process of training a deep neural network, because it is hoped that the output of the deep neural network is as close as possible to the value that you really want to predict, you can compare the predicted value of the current network with the target value you really want, and then based on the difference between the two To update the weight vector of each layer of neural network (of course, there is usually an initialization process before the first update, that is, pre-configured parameters for each layer in the deep neural network), for example, if the predicted value of the network If it is high, adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the really wanted target value or a value very close to the really wanted target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value". This is the loss function or objective function, which is used to measure the difference between the predicted value and the target value. Important equation. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, then the training of the deep neural network becomes a process of reducing this loss as much as possible.

(8) Backpropagation algorithm

The neural network can use the backpropagation (BP) algorithm to modify the parameter values in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, forwarding the input signal until the output will cause error loss, and the parameters in the initial neural network model are updated by backpropagating the error loss information, so that the error loss is converged. The backpropagation algorithm is a backpropagation motion dominated by error loss, and aims to obtain the optimal parameters of the neural network model, such as the weight matrix.

The system architecture of the embodiment of the present application will be described in detail below in conjunction with FIG. 1.

FIG. 1 is a schematic diagram of the system architecture of an embodiment of the present application. As shown in FIG. 1, the system architecture 100 includes an execution device 110, a training device 120, a database 130, a client device 140, a data storage system 150, and a data collection system 160.

In addition, the execution device 110 includes a calculation module 111, an I/O interface 112, a preprocessing module 113, and a preprocessing module 114. Among them, the calculation module 111 may include the target model/rule 101, and the preprocessing module 113 and the preprocessing module 114 are optional.

The data collection device 160 is used to collect training data. For the training method of the pedestrian re-recognition network in the embodiment of the present application, the training data may include M training images and the annotation data of the M training images. After the training data is collected, the data collection device 160 stores the training data in the database 130, and the training device 120 trains to obtain the target model/rule 101 based on the training data maintained in the database 130.

The following describes the target model/rule 101 obtained by the training device 120 based on the training data. The training device 120 performs feature extraction on the input training image to obtain the feature vector of the training image, and repeats feature extraction on the input training image until the loss function The function value of satisfies the preset requirement (less than or equal to the preset threshold), thereby completing the training of the target model/rule 101.

It should be understood that the training of the aforementioned target model/rule 101 may be an unsupervised training.

The above-mentioned target model/rule 101 can be used to implement the pedestrian re-identification method of the embodiment of the present application, that is, the pedestrian image (the pedestrian image may be an image that requires pedestrian recognition) is input into the target model/rule 101 to obtain the pedestrian re-identification method. The image extracts feature vectors, and performs pedestrian recognition based on the extracted feature vectors to determine the pedestrian recognition results. The target model/rule 101 in the embodiment of the present application may specifically be a neural network. It should be noted that, in actual applications, the training data maintained in the database 130 may not all come from the collection of the data collection device 160, and may also be received from other devices. In addition, it should be noted that the training device 120 does not necessarily perform the training of the target model/rule 101 completely based on the training data maintained by the database 130. It may also obtain training data from the cloud or other places for model training. The above description should not be used as a reference to this application. Limitations of the embodiment.

The target model/rule 101 trained according to the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in FIG. 1, which can be a terminal, such as a mobile phone terminal, a tablet computer, Notebook computers, augmented reality (AR)/virtual reality (VR), in-vehicle terminals, etc., can also be servers or clouds. In FIG. 1, the execution device 110 is configured with an input/output (input/output, I/O) interface 112 for data interaction with external devices. The user can input data to the I/O interface 112 through the client device 140. The input data in the embodiment of the present application may include: a pedestrian image input by the client device. The client device 140 here may specifically be a monitoring device.

The preprocessing module 113 and the preprocessing module 114 are used to perform preprocessing according to the input data (such as pedestrian images) received by the I/O interface 112. In the embodiment of the present application, the preprocessing module 113 and the preprocessing module 114 may be omitted or There is only one preprocessing module. When the preprocessing module 113 and the preprocessing module 114 do not exist, the calculation module 111 can be directly used to process the input data.

When the execution device 110 preprocesses input data, or when the calculation module 111 of the execution device 110 performs calculations and other related processing, the execution device 110 may call data, codes, etc. in the data storage system 150 for corresponding processing , The data, instructions, etc. obtained by corresponding processing may also be stored in the data storage system 150.

Finally, the I/O interface 112 presents the processing results (specifically, high-quality images obtained by re-recognition of pedestrians), such as the recognition results of the image to be recognized obtained by the target model/rule 101 on pedestrian re-recognition processing, to the customer The device 140 is thus provided to the user.

Specifically, the high-quality image obtained by re-identification of pedestrians through the target model/rule 101 in the calculation module 111 can be processed by the preprocessing module 113 (the processing of the preprocessing module 114 can also be added) (for example, image rendering is performed). After processing), the processing result is sent to the I/O interface, and then the processing result is sent to the client device 140 through the I/O interface for display.

It should be understood that when the preprocessing module 113 and the preprocessing module 114 do not exist in the above system architecture 100, the calculation module 111 can also transmit the high-quality image obtained through the pedestrian re-identification process to the I/O interface, and then the I/O interface. The O interface sends the processing result to the client device 140 for display.

It is worth noting that the training device 120 can target different targets or tasks (for example, the training device can train for real high-quality images and approximate low-quality images in different scenarios), and generate corresponding targets based on different training data. Model/rule 101, the corresponding target model/rule 101 can be used to achieve the above goals or complete the above tasks, so as to provide users with desired results.

It is worth noting that FIG. 1 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, devices, modules, etc. shown in the figure does not constitute any limitation. For example, in FIG. 1, the data The storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110.

As shown in FIG. 1, the target model/rule 101 obtained by training according to the training device 120 may be a neural network (model). Specifically, the neural network (model) may be CNN, deep convolutional neural networks (deep convolutional neural networks, DCNN), and so on.

Since CNN is a very common neural network, the structure of CNN will be introduced in detail below in conjunction with Figure 2. As mentioned in the introduction to the basic concepts above, a convolutional neural network is a deep neural network with a convolutional structure. It is a deep learning architecture. The deep learning architecture refers to the algorithm of machine learning. Multi-level learning is carried out on the abstract level of the system. As a deep learning architecture, CNN is a feed-forward artificial neural network. Each neuron in the feed-forward artificial neural network can respond to the input image.

As shown in FIG. 2, a convolutional neural network (CNN) 200 may include an input layer 210, a convolutional layer/pooling layer 220 (the pooling layer is optional), and a fully connected layer 230. The following is a detailed introduction to the relevant content of these layers.

Convolutional layer/pooling layer 220:

Convolutional layer:

The convolutional layer/pooling layer 220 shown in FIG. 2 may include layers 221-226 as shown in the examples. For example, in one implementation, layer 221 is a convolutional layer, layer 222 is a pooling layer, and layer 223 is a convolutional layer. Layers, 224 is the pooling layer, 225 is the convolutional layer, and 226 is the pooling layer; in another implementation, 221 and 222 are convolutional layers, 223 is the pooling layer, and 224 and 225 are convolutional layers. Layer, 226 is the pooling layer. That is, the output of the convolutional layer can be used as the input of the subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.

The following will take the convolutional layer 221 as an example to introduce the internal working principle of a convolutional layer.

The convolution layer 221 can include many convolution operators. The convolution operator is also called a kernel. Its function in image processing is equivalent to a filter that extracts specific information from the input image matrix. The convolution operator is essentially It can be a weight matrix. This weight matrix is usually pre-defined. In the process of convolution on the image, the weight matrix is usually one pixel after one pixel (or two pixels after two pixels) along the horizontal direction on the input image. ...It depends on the value of stride) to complete the work of extracting specific features from the image. The size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix and the depth dimension of the input image are the same. During the convolution operation, the weight matrix will extend to Enter the entire depth of the image. Therefore, convolution with a single weight matrix will produce a single depth dimension convolution output, but in most cases, a single weight matrix is not used, but multiple weight matrices of the same size (row×column) are applied. That is, multiple homogeneous matrices. The output of each weight matrix is stacked to form the depth dimension of the convolutional image, where the dimension can be understood as determined by the "multiple" mentioned above. Different weight matrices can be used to extract different features in the image. For example, one weight matrix is used to extract edge information of the image, another weight matrix is used to extract specific colors of the image, and another weight matrix is used to eliminate unwanted noise in the image. Perform fuzzification, etc. The multiple weight matrices have the same size (row×column), the size of the convolution feature maps extracted by the multiple weight matrices of the same size are also the same, and then the multiple extracted convolution feature maps of the same size are merged to form The output of the convolution operation.

The weight values in these weight matrices need to be obtained through a lot of training in practical applications. Each weight matrix formed by the weight values obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions. .

When the convolutional neural network 200 has multiple convolutional layers, the initial convolutional layer (such as 221) often extracts more general features, which can also be called low-level features; with the convolutional neural network With the deepening of the network 200, the features extracted by the subsequent convolutional layers (for example, 226) become more and more complex, such as features such as high-level semantics, and features with higher semantics are more suitable for the problem to be solved.

Pooling layer:

Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolutional layer. In the 221-226 layers as illustrated by 220 in Figure 2, it can be a convolutional layer followed by a layer. The pooling layer can also be a multi-layer convolutional layer followed by one or more pooling layers. In the image processing process, the sole purpose of the pooling layer is to reduce the size of the image space. The pooling layer may include an average pooling operator and/or a maximum pooling operator for sampling the input image to obtain an image with a smaller size. The average pooling operator can calculate the pixel values in the image within a specific range to generate an average value as the result of the average pooling. The maximum pooling operator can take the pixel with the largest value within a specific range as the result of the maximum pooling. In addition, just as the size of the weight matrix used in the convolutional layer should be related to the image size, the operators in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer can be smaller than the size of the image of the input pooling layer, and each pixel in the image output by the pooling layer represents the average value or the maximum value of the corresponding sub-region of the image input to the pooling layer.

Fully connected layer 230:

After processing by the convolutional layer/pooling layer 220, the convolutional neural network 200 is not enough to output the required output information. Because as mentioned above, the convolutional layer/pooling layer 220 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolutional neural network 200 needs to use the fully connected layer 230 to generate one or a group of required classes of output. Therefore, the fully connected layer 230 may include multiple hidden layers (231, 232 to 23n as shown in FIG. 2) and an output layer 240. The parameters contained in the multiple hidden layers can be based on specific task types. The relevant training data of the, for example, the task type can include image recognition, image classification, image super-resolution reconstruction and so on.

After the multiple hidden layers in the fully connected layer 230, that is, the final layer of the entire convolutional neural network 200 is the output layer 240. The output layer 240 has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network 200 (as shown in Figure 2 from 210 to 240 is the forward propagation) is completed, the back propagation (as shown in Figure 2 is the propagation from 240 to 210 is the back propagation). Start to update the weight values and deviations of the aforementioned layers to reduce the loss of the convolutional neural network 200 and the error between the output result of the convolutional neural network 200 through the output layer and the ideal result.

It should be noted that the convolutional neural network 200 shown in FIG. 2 is only used as an example of a convolutional neural network. In specific applications, the convolutional neural network may also exist in the form of other network models.

It should be understood that the convolutional neural network (CNN) 200 shown in FIG. 2 may be used to perform the pedestrian re-identification method of the embodiment of the present application. As shown in FIG. 2, the pedestrian image passes through the input layer 210 and the convolutional layer/pooling layer 220. After processing with the fully connected layer 230, the image characteristics of the image to be recognized can be obtained, and then the recognition result of the image to be recognized can be obtained according to the image characteristics of the image to be recognized.

FIG. 3 is a hardware structure of a chip provided by an embodiment of the application, and the chip includes a neural network processor 50. The chip can be set in the execution device 110 as shown in FIG. 1 to complete the calculation work of the calculation module 111. The chip can also be set in the training device 120 as shown in FIG. 1 to complete the training work of the training device 120 and output the target model/rule 101. The algorithms of each layer in the convolutional neural network as shown in Fig. 2 can be implemented in the chip as shown in Fig. 3.

A neural network processor (neural-network processing unit, NPU) 50 is mounted on a main central processing unit (central processing unit, CPU) (host CPU) as a coprocessor, and the main CPU allocates tasks. The core part of the NPU is the arithmetic circuit 503. The controller 504 controls the arithmetic circuit 503 to extract data from the memory (weight memory or input memory) and perform calculations.

In some implementations, the arithmetic circuit 503 includes multiple processing units (process engines, PE). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit 503 fetches the data corresponding to matrix B from the weight memory 502 and caches it on each PE in the arithmetic circuit 503. The arithmetic circuit 503 takes the matrix A data and the matrix B from the input memory 501 to perform matrix operations, and the partial result or final result of the obtained matrix is stored in an accumulator 508.

The vector calculation unit 507 can perform further processing on the output of the arithmetic circuit 503, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, and so on. For example, the vector calculation unit 507 can be used for network calculations in the non-convolutional/non-FC layer of the neural network, such as pooling, batch normalization, local response normalization, etc. .

In some implementations, the vector calculation unit 507 can store the processed output vector to the unified buffer 506. For example, the vector calculation unit 507 may apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate the activation value. In some implementations, the vector calculation unit 507 generates a normalized value, a combined value, or both. In some implementations, the processed output vector can be used as an activation input to the arithmetic circuit 503, for example for use in a subsequent layer in a neural network.

The unified memory 506 is used to store input data and output data.

The weight data directly transfers the input data in the external memory to the input memory 501 and/or the unified memory 506 through the storage unit access controller 505 (direct memory access controller, DMAC), and stores the weight data in the external memory into the weight memory 502, And the data in the unified memory 506 is stored in the external memory.

The bus interface unit (BIU) 510 is used to implement interaction between the main CPU, the DMAC, and the instruction fetch memory 509 through the bus.

An instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504;

The controller 504 is used to call the instructions cached in the memory 509 to control the working process of the computing accelerator.

Generally, the unified memory 506, the input memory 501, the weight memory 502, and the instruction fetch memory 509 are all on-chip memories. The external memory is a memory external to the NPU. The external memory can be a double data rate synchronous dynamic random access memory. Memory (double data rate synchronous dynamic random access memory, referred to as DDR SDRAM), high bandwidth memory (HBM) or other readable and writable memory.

In addition, in this application, the operations of each layer in the convolutional neural network shown in FIG. 2 may be executed by the arithmetic circuit 503 or the vector calculation unit 507.

As shown in FIG. 4, an embodiment of the present application provides a system architecture 300. The system architecture includes a local device 301, a local device 302, an execution device 210 and a data storage system 250, where the local device 301 and the local device 302 are connected to the execution device 210 through a communication network.

The execution device 210 may be implemented by one or more servers. Optionally, the execution device 210 can be used in conjunction with other computing devices, such as data storage, routers, load balancers, and other devices. The execution device 210 may be arranged on one physical site or distributed on multiple physical sites. The execution device 210 may use the data in the data storage system 250 or call the program code in the data storage system 250 to implement the pedestrian re-identification method in the embodiment of the present application.

The user can operate respective user devices (for example, the local device 301 and the local device 302) to interact with the execution device 210. Each local device can represent any computing device, such as personal computers, computer workstations, smart phones, tablets, smart cameras, smart cars or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc.

The local device of each user can interact with the execution device 210 through a communication network of any communication mechanism/communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

In an implementation manner, the local device 301 and the local device 302 obtain the relevant parameters of the target neural network from the execution device 210, deploy the target neural network on the local device 301 and the local device 302, and use the target neural network to perform pedestrian reconstruction. Recognition.

In another implementation, the target neural network can be directly deployed on the execution device 210, and the execution device 210 obtains pedestrian images from the local device 301 and the local device 302 (the local device 301 and the local device 302 can upload the image of the pedestrian to the execution device 210), and perform pedestrian re-identification on the pedestrian image according to the target neural network, and send the high-quality image obtained by the pedestrian re-identification to the local device 301 and the local device 302.

The above-mentioned execution device 210 may also be referred to as a cloud device. At this time, the execution device 210 is generally deployed in the cloud.

Fig. 5 is a schematic diagram of a possible application scenario of an embodiment of the present application.

As shown in Figure 5, in this application, the pedestrian re-recognition network can be trained through the single-image shooting device annotation data, and a trained pedestrian re-recognition network can be obtained. The trained pedestrian re-recognition network can process pedestrian images. Obtain the feature vector of the pedestrian image. Next, by comparing the feature vector of the pedestrian image with the feature vector in the image library, the person you are looking for can be obtained. Specifically, through feature comparison, the target pedestrian image that is most similar to the feature vector of the pedestrian image can be found, and basic information such as the shooting time and location of the target pedestrian image can be output.

It should be understood that the feature vector of each pedestrian image and related information of the pedestrian corresponding to the pedestrian image are stored in the image library.

It should be understood that the annotation data of the single-image shooting device herein may include multiple training images and annotation data of multiple training images. The single-image capturing device labeling data refers to the individual labeling of the training images obtained by each image capturing device, without the need to search for the same pedestrians between different image capturing devices. This labeling method does not need to pay attention to different images. The relationship between the training images captured by the shooting device can save a lot of labeling time and reduce the complexity of labeling. The multiple training images and the labeled data of the multiple training images may also be collectively referred to as training data.

FIG. 6 is a schematic diagram of the overall flow of the training method of the pedestrian re-identification network according to an embodiment of the present application.

As shown in FIG. 6, by individually data annotations on the video images acquired by each image capture device, single image capture device annotation data can be obtained. The biggest advantage of the label data of a single image capturing device is that it is easy to label and collect. In this application, the label data of a single image capturing device does not require the same pedestrian to appear under multiple image capturing devices.

In the single-image capture device annotation data, it is assumed that each pedestrian only appears in one image capture device (or one image capture device group). In this way, after pedestrian detection and tracking are used to obtain the pedestrian image in the video, only a small amount is required. Human power can associate several pictures of the same person in similar frames to form annotations. Moreover, the labels of each image capturing device are relatively independent, and there will be no overlap in the number of pedestrians of different image capturing devices. By setting different collection time periods for different image capturing devices, the number of people recurring in the video captured by each image capturing device can be reduced, thereby achieving the requirement of labeling data for a single image capturing device.

In some relatively small scenes (for example, an office park), most people have a small range of activities, and a considerable number of people only appear in a certain image capturing equipment group. Such data can naturally meet this requirement. Since cameras in an image capturing device group have similar or overlapping fields of view and similar lighting conditions, these cameras can basically be equivalent to one camera.

After the single-image shooting device annotation data is obtained, the single-image shooting device annotation data can be used to train the pedestrian re-recognition network (model), and the trained pedestrian re-recognition network can be used for testing and deployment. Specifically, the pedestrian re-identification network obtained by training can be used to implement the pedestrian re-identification method of the embodiment of the present application.

FIG. 7 is a schematic flowchart of a training method for a pedestrian re-identification network according to an embodiment of the present application. The method shown in FIG. 7 may be executed by the training device of the pedestrian re-recognition network of the embodiment of the present application (for example, it may be executed by the device shown in FIG. 10 and FIG. 11), and the method shown in FIG. 7 includes steps 1001 to 1008. , These steps are described in detail below.

1001. Start.

Step 1001 represents the start of the training process of the pedestrian re-recognition network.

1002. Obtain training data.

The training data in the above step 1002 includes M (M is an integer greater than 1) training images and the annotation data of M training images. Among the M training images, each training image includes pedestrians, and each training image The annotation data includes the bounding box and pedestrian identification information of the pedestrian in each training image. Different pedestrians correspond to different pedestrian identification information. Among the M training images, the training images with the same pedestrian identification information are taken from the same image. equipment.

The aforementioned image capturing device may specifically be a device capable of acquiring images of pedestrians, such as a video camera and a camera.

The pedestrian identification information in the above step 1002 can also be referred to as pedestrian identification information, which is a kind of information used to indicate the identity of a pedestrian. Each pedestrian can correspond to unique pedestrian identification information. There are many ways to express the pedestrian identification information. , As long as the identity information of the pedestrian can be indicated, for example, the pedestrian identification information may specifically be a pedestrian identity (identity, ID), that is, a unique ID can be assigned to each pedestrian.

1003. Perform initialization processing on the network parameters of the pedestrian re-identification network to obtain initial values of the network parameters of the pedestrian re-identification network.

In the above step 1003, the network parameters of the pedestrian re-identification network can be randomly set, and the initial values of the network parameters of the pedestrian re-identification network are obtained.

1004. Input a batch of training images among the M training images to the pedestrian recognition network to perform feature extraction, and obtain a feature vector of each training image in the batch of training images.

The above batch of training images is part of the training images in M training images. When M training images are used to train the pedestrian re-recognition network, the M training images can be divided into different batches to train the pedestrian re-recognition network. The number of training images in each batch can be the same or different.

For example, there are 5000 training images in total, and 100 training images can be input in each batch to train the pedestrian re-recognition network.

The foregoing batch of training images may include N anchor point images, where the N anchor point images are any N training images in the foregoing batch of training images, and each anchor point image in the N anchor point images corresponds to one The most difficult positive sample image, a first most difficult negative sample image and a second most difficult negative sample image, N is a positive integer, and N is less than M.

The following describes the most difficult positive sample image corresponding to each anchor point image, the first most difficult negative sample image, and the second most difficult negative sample image.

The most difficult positive sample image corresponding to each anchor point image: the training image that has the same pedestrian identification information as each anchor point image and the farthest distance from the feature vector of each anchor point image in the above batch of training images ；

The first most difficult negative sample image corresponding to each anchor point image: the above batch of training images and each anchor point image come from the same image capture device, and are different from the pedestrian identification information of each anchor point image and are different from each anchor point image. The training image with the closest distance between the feature vectors of the anchor image;

The second most difficult negative sample image corresponding to each anchor point image: the above batch of training images and each anchor point image come from different image capture equipment, and are different from the pedestrian identification information of each anchor point image and are different from each anchor point image. The training image with the closest distance between the feature vectors of the point image.

1005. Determine the function value of the loss function according to the feature vector of the above batch of training images.

The function value of the loss function in the above step 1005 is obtained by averaging the function values of the N first loss functions.

Wherein, the function value of each first loss function in the above N first loss functions is calculated according to the first difference and the second difference corresponding to each of the N anchor point images.

Optionally, the function value of each of the foregoing first loss functions is the sum of the first difference and the second difference corresponding to each anchor point image.

The above N is a positive integer, and the above N is less than M. When N=1, there is only one function value of the first loss function. At this time, the function value of the first loss function can be directly used as the function value of the loss function in step 1005.

For example, the function value of the first loss function may be as shown in formula (2).

L1=D1+D2 (2)

Wherein, L1 represents the function value of the first loss function, D1 represents the above-mentioned first difference value, and D2 represents the above-mentioned second difference value.

Optionally, the function value of each of the foregoing first loss functions is the sum of the absolute value of the first difference and the absolute value of the second difference corresponding to each anchor point image.

For example, the function value of the first loss function may be as shown in formula (3).

L1=|D1|+|D2| (3)

Wherein, L1 represents the function value of the first loss function, |D1| represents the absolute value of the above-mentioned first difference, and |D2| represents the absolute value of the above-mentioned second difference.

Optionally, the function value of each of the foregoing first loss functions is the sum of the first difference, the second difference, and other constant items corresponding to each anchor point image.

For example, the function value of the first loss function may be as shown in formula (4).

L1=D1+D2+m (4)

Wherein, L1 represents the function value of the first loss function, D1 represents the above-mentioned first difference value, D2 represents the above-mentioned second difference value, m represents a constant, and the size of m can be set to an appropriate value based on experience.

For another example, the function value of the first loss function may be as shown in formula (5).

L1=|m ₁ +D1|+|m ₂ +D2| (5)

Among them, L1 represents the function value of the first loss function, |D1| represents the above-mentioned first difference, |D2| represents the above-mentioned second difference, m ₁ and m ₂ represent constants, _{and the magnitudes of m 1} and m ₂ can be based on experience To set the appropriate value.

It should be understood that calculating the absolute value of D1 and D2 when calculating the function value of the first loss function is only an optional implementation method. In fact, when determining the function value of the first loss function, D1 and D2 can also be calculated. Other operations, for example, [X] ₊ operation can be performed on D1 and D2 (this operation can be referred to as the operation of taking the positive part of the function value).

Among them, when X is greater than 0, [X] ₊ =X, and when X is less than 0, [X] ₊ = 0. (For details, please refer to https://en.wikipedia.org/wiki/Positive_and_negative_parts)

The meaning of the first difference and the second difference will be described below.

The first difference corresponding to each anchor image: the difference between the distance of the most difficult positive sample corresponding to each anchor image and the distance of the second most difficult negative sample corresponding to each anchor image;

The second difference value corresponding to each anchor point image: the difference between the distance of the second most difficult negative sample corresponding to each anchor point image and the distance of the first most difficult negative sample corresponding to each anchor point image.

The above-mentioned first difference and second difference are obtained by performing difference on different distances. The meaning of these distances is explained below.

The distance of the most difficult positive sample corresponding to each anchor point image: the distance between the feature vector of the most difficult positive sample image corresponding to each anchor point image and the feature vector of each anchor point image;

The second most difficult negative sample distance corresponding to each anchor point image: the distance between the feature vector of the second most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image;

The distance of the first most difficult negative sample corresponding to each anchor point image: the distance between the feature vector of the first most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image.

记

为网络模型输出的特征(向量)，记||f ₁-f ₂||为两个特征f ₁和f ₂的欧式距离，那么，上述最难正样本距离可以如公式(6)所示。 Specifically, assuming that the number of image capturing devices corresponding to a batch of images in the training process is C, the number of pedestrians under each image capturing device is P, and the number of images of each pedestrian is K, then the number of images in a batch It is C×P×K. Remember that the anchor image in the batch of images is

Remember

Is the feature (vector) output by the network model, denote ||f ₁ -f ₂ || as the Euclidean distance of the two features f ₁ and f ₂ , then the above-mentioned most difficult positive sample distance can be as shown in formula (6).

The above-mentioned second most difficult negative sample distance can be as shown in formula (7).

The above-mentioned first most difficult negative sample distance can be as shown in formula (8).

The above-mentioned first difference value may be the difference between formula (6) and formula (7), and the above-mentioned second difference value may be the difference between formula (7) and formula (8).

The loss function composed of the above-mentioned first difference and the second difference can be as shown in formula (9). Among them, L represents the loss function, m ₁ and m ₂ are two constants, and the specific value can be set based on experience. For example, m ₁ =0.1 and m ₂ =0.1.

表示对

进行[X] ₊操作，当

的取值大于或者等于0时，

的取值就是

当

的取值小于0时，

的取值就是0。 In the above formula (9),

Means right

Perform [X] ₊ operation, when

When the value of is greater than or equal to 0,

The value of is

when

When the value of is less than 0,

The value of is 0.

表示对

进行[X] ₊操作，当

的取值大于或者等于0时，

的取值就是

而当

小于0时，

的取值就是0。 In the above formula (9),

Means right

Perform [X] ₊ operation, when

When the value of is greater than or equal to 0,

The value of is

And when

When less than 0,

The value of is 0.

1006. Update the network parameters of the pedestrian re-identification network according to the function value of the loss function.

Specifically, the network parameters of the pedestrian re-identification network can be updated according to the function value of the loss function shown in the above formula (9). And in the process of updating, the function value of the loss function shown in formula (9) is getting smaller and smaller.

1007. Determine whether the pedestrian re-identification network meets the preset requirements.

Optionally, the pedestrian re-identification network meets preset requirements, including: the pedestrian re-identification network meets at least one of the following conditions:

(1) The pedestrian recognition performance of the pedestrian re-identification network meets the preset performance requirements;

(2) The update times of the network parameters of the pedestrian re-identification network are greater than or equal to the preset times;

(3) The function value of the loss function is less than or equal to the preset threshold.

In step 1007, when the pedestrian re-identification network meets at least one of the above conditions (1) to (3), it can be determined that the pedestrian re-identification network meets the preset requirements, and step 1008 is executed, and the training process of the pedestrian re-identification network ends; When the pedestrian re-recognition network does not meet any of the above conditions (1) to (3), it means that the pedestrian re-recognition network has not yet met the preset requirements, and the pedestrian re-recognition network needs to continue to be trained, that is, step 1004 is re-executed To 1007, the network will not be recognized until pedestrians meeting the preset requirements are obtained.

The above preset threshold can be flexibly set by experience. When the preset threshold is set too large, the pedestrian recognition effect of the trained pedestrian re-recognition network may not be good enough, and when the preset threshold is set too small, the loss of the function during training The function value may be difficult to converge.

Optionally, the value range of the foregoing preset threshold is [0, 0.01].

Specifically, the value of the foregoing preset threshold may be 0.01.

The function value of the aforementioned loss function is less than or equal to the preset threshold, which specifically includes: the first difference is less than the first preset threshold, and the second difference is less than the second preset threshold.

The above-mentioned first preset threshold and second preset threshold can also be determined based on experience. When the first preset threshold and the second preset threshold are set too large, the pedestrian recognition effect of the trained pedestrian re-recognition network may not be good enough , And when the first preset threshold and the second preset threshold are set too small, the function value of the loss function may be difficult to converge during training.

Optionally, the value range of the foregoing first preset threshold is [0, 0.4].

Optionally, the value range of the foregoing first preset threshold is [0, 0.4].

Specifically, both the first preset threshold and the second threshold may be 0.1.

1008. The training is over.

In addition, in this application, that several training images are from the same image capturing device means that these several training images are captured by the same image capturing device.

In this application, the most difficult negative sample images from different image capturing devices and the same image capturing device are considered in the process of constructing the loss function, and the first difference and the second difference are reduced as much as possible during the training process. Small, which can eliminate as much as possible the interference of the image capturing device's own information on the image information, so that the trained pedestrian re-recognition network can more accurately extract features from the image.

Specifically, in the training process of the pedestrian re-recognition network, the network parameters of the pedestrian re-recognition network are optimized to make the first difference and the second difference as small as possible, so that the distance between the most difficult positive sample and the second hardest The difference between the distance of the negative sample and the distance between the second most difficult negative sample and the distance of the first most difficult negative sample are as small as possible, so that the pedestrian re-recognition network can distinguish the most difficult image from the second most difficult negative as much as possible. The characteristics of the sample image, and the characteristics of the second most difficult negative sample image and the first most difficult negative sample image, so that the trained pedestrian re-recognition network can perform feature extraction on the image better and more accurately.

The process of determining the function value of the loss function according to a batch of training images in the above step 1004 and step 1005 will be described in detail below with reference to FIG. 8.

As shown in Figure 8, after inputting a batch of training images into the pedestrian recognition network, the feature vectors of this batch of training images can be obtained. Next, you can select multiple anchor point images from this batch of training images, and determine the corresponding most difficult positive sample image and the first most difficult negative sample image for each of the multiple anchor point images. And the second most difficult negative sample image.

In this way, a lot of four training images (an anchor image, the most difficult positive sample image corresponding to the anchor image, the first most difficult negative sample image corresponding to the anchor image, and the second most difficult image corresponding to the anchor image) can be obtained. Then, a first loss function can be determined according to the distance relationship between the feature vectors of the training images in each training image group.

As shown in Figure 8, there are a total of N training image groups. According to the N training image groups, a total of N first loss functions can be determined. Next, the function values of these N first loss functions are averaged, and then The function value of the loss function in the above step 1005 can be obtained.

It should be understood that the foregoing N training image groups include a total of N anchor point images, and the N anchor point images are different from each other, that is, each training image group corresponds to a unique anchor point image. However, other training images (images other than the anchor point image) contained in different training image groups may be the same. For example, the hardest positive sample image in the first training image group is the same as the hardest positive sample image in the second training group.

As another example, suppose the number of the above batch of training images is 100, then 10 can be selected from the 100 training images (it can also be other numbers, here only 10 is taken as an example) anchor image , And then select the corresponding most difficult positive sample image, the first most difficult negative sample image and the second most difficult negative sample image for each anchor image from the 100 training images. Thus, 10 training image groups are obtained. According to the 10 training image groups, the function values of 10 first loss functions can be obtained. Next, by averaging the function values of the 10 first loss functions, the above can be obtained. The function value of the loss function in step 1005.

The following is a detailed introduction to the design and training process of the pedestrian re-identification network.

The pedestrian re-identification network in this application can use the existing residual network (for example, ResNet50) as the main body of the network, remove the last fully connected layer, and add the global mean after the last layer of residual block (ResBlock) Pooling (global average pooling) layer, and obtain the feature vector of 2048 dimensions (or other values) as the output of the network model.

In each batch of training images, each camera can collect 4 people, and each person collects 8 images. If there are fewer than 8 images of one person, repeat the collection to fill up 8 images.

When training the trained pedestrian re-recognition network, the above formula (9) can be used as the loss function. When testing, the number of cameras in different data sets can be different. For example, for the DukeMTMC-reID data set, the cameras have 8, at this time, C=8 in formula (9); for the Market-1501 data set, there are 6 cameras, at this time, C=6 in formula (9).

The two parameters in the loss function shown in the above formula (9) may be m ₁ =0.1 and m ₂ =0.1, respectively. The input training image can be scaled to a size of 256x128 pixels, the adaptive moment estimation (Adam) optimizer can be used to train the network parameters during training, and the learning rate can be set to 2×10 ^-4 . After 100 rounds of training, the learning rate decays exponentially, until after 200 rounds of learning, the learning rate can be set to 2×10 ^-7 , then training can be stopped.

The pedestrian re-identification network trained according to the pedestrian re-identification network training method of the embodiment of the present application can be used to implement the pedestrian re-identification method of the embodiment of the present application. The pedestrian re-identification method of the embodiment of the present application will be described below with reference to the accompanying drawings.

FIG. 9 is a schematic flowchart of a pedestrian re-identification method according to an embodiment of the present application. The pedestrian re-identification method shown in FIG. 9 may be executed by the pedestrian re-identification device of the embodiment of the present application (for example, it may be executed by the device shown in FIG. 12 and FIG. 13), and the pedestrian re-identification method shown in FIG. 9 includes steps 2001 to In 2003, the steps 2001 to 2003 will be described in detail below.

2001. Obtain the image to be recognized.

2002. Use the pedestrian re-recognition network to process the image to be recognized, and obtain the feature vector of the image to be recognized.

Wherein, the pedestrian re-identification network used in step 2002 may be obtained by training according to the training method of the pedestrian re-identification network of the embodiment of the present application. Specifically, the pedestrian re-identification network in step 2002 may be obtained by the method shown in FIG. 7 Get it through training.

2003. Compare the feature vector of the image to be recognized with the feature vector of the existing pedestrian image to obtain the recognition result of the image to be recognized.

In this application, the pedestrian re-recognition network trained by the training method of the pedestrian re-recognition network in the embodiments of this application can better perform feature extraction. Therefore, using the pedestrian re-recognition network to process the image to be recognized can obtain more information. Good pedestrian recognition results.

Optionally, the above step 2003 specifically includes: comparing the feature vector of the image to be recognized with the feature vector of the existing pedestrian image to determine the output target pedestrian image; outputting the target pedestrian image and the attribute information of the target pedestrian image.

The above-mentioned target pedestrian image may be a pedestrian image whose feature vector is most similar to the feature vector of the image to be recognized in the existing pedestrian image, and the attribute information of the target pedestrian image includes the shooting time and shooting location of the target pedestrian image. In addition, the attribute information of the target pedestrian image may also include the identity information of the pedestrian and the like.

The following describes the pedestrian recognition effect of the pedestrian re-identification network in the embodiment of the present application in combination with specific test results.

Table 1

Table 1 shows the test results of different schemes on different data sets. Among them, the test results include Rank-1 and mean average precision (mAP), where Rank-1 represents the feature vector in the existing image The probability that the image with the closest distance to the feature vector of the image to be recognized belongs to the same pedestrian as the image to be recognized.

In Table 1 above, data set 1 is Duke-SCT, and data set 2 is Market-SCT.

Among them, Duke-SCT is a subset of the DukeMTMC-reID data set, and Market-SCT is a subset of the Market-1501 data set. When acquiring Duke-SCT and Market-SCT, we processed the training data from the original data set (DukeMTMC-reID and Market-1501) as follows: each pedestrian randomly selects the image under a certain camera for preservation (Different pedestrians may choose different cameras), resulting in the formation of new data sets Duke-SCT and Market-SCT. At the same time, the test set remains unchanged.

Existing solution 1: a discriminative feature learning approach for deep face, which was published at the European Conference on Computer Vision (ECCV) in 2016;

Existing solution 2: Deep hypersphere embedding for face recognition (deep hypersphere embedding for face recognition), which was published at the International Conference on Computer Vision and Pattern Recognition (CVPR) in 2017;

Existing solution 3: Additional angular margin loss for deep face recognition for deep face recognition, which was published in CVPR in 2019;

Existing plan 4: person retrieval with refined part pooling, which was published in ECCV in 2018;

Existing scheme 5: Part-aligned bilinear representations for person re-identification, which was published in ECCV in 2018;

Existing solution 6: Learning discriminative features with multiple granularities for person re-Identification multimedia, ACMMM) published.

It can be seen from Table 1 that the Rank-1 and mAP of the scheme of the present application are superior to the existing schemes in both dataset 1 and dataset 2, and have a better recognition effect.

Fig. 10 is a schematic block diagram of a training device for a pedestrian re-identification network according to an embodiment of the present application. The training device 8000 of the pedestrian re-identification network shown in FIG. 10 includes an acquisition unit 8001 and a training unit 8002.

The acquisition unit 8001 and the training unit 8002 may be used to execute the pedestrian re-identification network training method in the embodiment of the present application.

Specifically, the acquiring unit 8001 may perform the above steps 1001 and 1002, and the training unit 8002 may perform the above steps 1003 to 1008.

The acquisition unit 8001 in the device 8000 shown in FIG. 10 may be equivalent to the communication interface 9003 in the device 9000 shown in FIG. 11, through which the corresponding training images can be obtained, or the acquisition unit 8001 may also provide It is equivalent to the processor 9002. At this time, the training image can be obtained from the memory 9001 through the processor 9002, or the training image can be obtained from the outside through the communication interface 9003. In addition, the training unit 8002 in the device 8000 may be equivalent to the processor 9002 in the device 9000.

FIG. 11 is a schematic diagram of the hardware structure of a training device for a pedestrian re-identification network according to an embodiment of the present application. The training device 9000 of the pedestrian re-identification network shown in FIG. 11 (the device 9000 may specifically be a computer device) includes a memory 9001, a processor 9002, a communication interface 9003, and a bus 9004. Among them, the memory 9001, the processor 9002, and the communication interface 9003 implement communication connections between each other through the bus 9004.

The memory 9001 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 9001 may store a program. When the program stored in the memory 9001 is executed by the processor 9002, the processor 9002 is configured to execute each step of the pedestrian re-identification network training method in the embodiment of the present application.

The processor 9002 may adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more The integrated circuit is used to execute related programs to implement the pedestrian re-identification network training method in the embodiment of the present application.

The processor 9002 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the training method of the pedestrian re-recognition network of the present application can be completed by the integrated logic circuit of the hardware in the processor 9002 or the instructions in the form of software.

The aforementioned processor 9002 may also be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), a ready-made programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gates or transistor logic devices, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers. The storage medium is located in the memory 9001, and the processor 9002 reads the information in the memory 9001, and combines its hardware to complete the functions required by the units included in the training device of the pedestrian re-identification network, or execute the pedestrian re-identification in the embodiment of the application The training method of the network.

The communication interface 9003 uses a transceiver device such as but not limited to a transceiver to implement communication between the device 9000 and other devices or a communication network. For example, the image to be recognized can be acquired through the communication interface 9003.

The bus 9004 may include a path for transferring information between various components of the device 9000 (for example, the memory 9001, the processor 9002, and the communication interface 9003).

Fig. 12 is a schematic block diagram of a pedestrian re-identification device according to an embodiment of the present application. The pedestrian re-identification device 10000 shown in FIG. 12 includes an acquisition unit 10001 and an identification unit 10002.

The acquiring unit 10001 and the identifying unit 10002 may be used to execute the pedestrian re-identification method in the embodiment of the present application.

Specifically, the acquiring unit 10001 may perform the foregoing step 6001, and the identifying unit 10002 may perform the foregoing step 6002.

The acquisition unit 10001 in the device 10000 shown in FIG. 12 may be equivalent to the communication interface 11003 in the device 11000 shown in FIG. 13, through which the image to be recognized can be obtained, or the acquisition unit 10001 may also be equivalent. In the processor 11002, the image to be recognized can be obtained from the memory 11001 through the processor 11002 at this time, or the image to be recognized can be obtained from the outside through the communication interface 11003.

In addition, the recognition unit 10002 in the device 10000 shown in FIG. 12 is equivalent to the processor 11002 in the device 11000 shown in FIG. 13.

FIG. 13 is a schematic diagram of the hardware structure of a pedestrian re-identification device according to an embodiment of the present application. Similar to the above device 10000, the pedestrian re-identification device 11000 shown in FIG. 13 includes a memory 11001, a processor 11002, a communication interface 11003, and a bus 11004. Among them, the memory 11101, the processor 11102, and the communication interface 11003 implement communication connections between each other through the bus 11004.

The memory 11001 may be ROM, static storage device and RAM. The memory 11001 may store a program. When the program stored in the memory 11001 is executed by the processor 11002, the processor 11002 and the communication interface 11003 are used to execute the steps of the pedestrian re-identification method in the embodiment of the present application.

The processor 11002 may adopt a general-purpose CPU, a microprocessor, an ASIC, a GPU or one or more integrated circuits to execute related programs to realize the functions required by the units in the pedestrian re-identification device of the embodiment of the present application. , Or implement the pedestrian re-identification method in the embodiment of this application.

The processor 11002 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the pedestrian re-identification method in the embodiment of the present application can be completed by the integrated logic circuit of the hardware in the processor 11002 or the instructions in the form of software.

The aforementioned processor 11002 may also be a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers. The storage medium is located in the memory 11001, and the processor 11002 reads the information in the memory 11001, and combines its hardware to complete the functions required by the units included in the pedestrian re-identification device of the embodiment of the present application, or execute the pedestrian re- recognition methods.

The communication interface 11003 uses a transceiver device such as but not limited to a transceiver to implement communication between the device 11000 and other devices or a communication network. For example, the image to be recognized can be acquired through the communication interface 11003.

The bus 11004 may include a path for transferring information between various components of the device 11000 (for example, the memory 11001, the processor 11002, and the communication interface 11003).

It should be noted that although the foregoing device 9000 and device 11000 only show the memory, processor, and communication interface, in the specific implementation process, those skilled in the art should understand that the device 9000 and device 11000 may also include those necessary for normal operation. Other devices. At the same time, according to specific needs, those skilled in the art should understand that the device 9000 and the device 11000 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the device 9000 and the device 11000 may also only include the components necessary to implement the embodiments of the present application, and not necessarily include all the components shown in FIG. 11 and FIG. 13.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (14)

A training method for a pedestrian re-recognition network, which is characterized in that it includes: Step 1: Obtain M training images and annotation data of the M training images, each of the M training images includes a pedestrian, and the annotation data of each training image includes each of the training images The bounding frame and pedestrian identification information in which the pedestrian is located in, where different pedestrians correspond to different pedestrian identification information. Among the M training images, the training images with the same pedestrian identification information come from the same image capturing device, M Is an integer greater than 1; Step 2: Initialize the network parameters of the pedestrian re-identification network to obtain the initial values of the network parameters of the pedestrian re-identification network; Step 3: Input a batch of training images of the M training images to the pedestrian re-recognition network for feature extraction, and obtain a feature vector of each training image in the batch of training images; Wherein, the batch of training images includes N anchor point images, the N anchor point images are any N training images in the batch of training images, and each anchor point in the N anchor point images The image corresponds to a hardest positive sample image, a first hardest negative sample image and a second hardest negative sample image, N is a positive integer; The most difficult positive sample image corresponding to each anchor point image is the same as the pedestrian identification information of each anchor point image in the batch of training images, and is different from the feature vector of each anchor point image. The training image with the farthest distance, the first most difficult negative sample image corresponding to each anchor point image is from the same image capturing device as each anchor point image in the batch of training images, and the The pedestrian identification information of each anchor image is different and the distance between the feature vector of each anchor image is the closest to the training image, and the second most difficult negative sample image corresponding to each anchor image is the training image. The distance between the batch of training images and each of the anchor point images is from a different image capturing device, and is different from the pedestrian identification information of each anchor point image and from the feature vector of each anchor point image The most recent training image; Step 4: Determine the function value of the loss function according to the feature vectors of the batch of training images, and the function value of the loss function is obtained by averaging the function values of the N first loss functions; Wherein, the function value of each first loss function in the N first loss functions is calculated according to the first difference and the second difference corresponding to each of the N anchor point images , The first difference value corresponding to each anchor point image is the difference between the most difficult positive sample distance corresponding to each anchor point image and the second most difficult negative sample distance corresponding to each anchor point image, The second difference value corresponding to each anchor point image is the difference between the second most difficult negative sample distance corresponding to each anchor point image and the first most difficult negative sample distance corresponding to each anchor point image, The most difficult positive sample distance corresponding to each anchor point image is the distance between the feature vector of the most difficult positive sample image corresponding to each anchor point image and the feature vector of each anchor point image. The second most difficult negative sample distance corresponding to the anchor point image is the distance between the feature vector of the second most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image. The first most difficult negative sample distance corresponding to the anchor point image is the distance between the feature vector of the first most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image; Step 5: Update the network parameters of the pedestrian re-identification network according to the function value of the loss function; Repeat the above steps 3 to 5 until the pedestrian re-identification network meets the preset requirements. The training method of claim 1, wherein the pedestrian re-identification network meets a preset requirement, comprising: When at least one of the following conditions is met, the pedestrian re-identification network meets the preset requirements: The number of times of training of the pedestrian re-recognition network is greater than or equal to a preset number of times; The function value of the loss function is less than or equal to a preset threshold; The recognition performance of the pedestrian re-identification network meets the preset requirements. The training method according to claim 2, wherein the function value of the loss function is less than or equal to a preset threshold value, comprising: The first difference is less than a first preset threshold, and the second difference is less than a second preset threshold. The training method according to any one of claims 1 to 3, wherein the M training images are training images from multiple image capturing devices, wherein the labeled data of the training images from different image capturing devices It is separately marked. A pedestrian re-identification method, which is characterized in that it includes: Obtain the image to be recognized; The pedestrian re-recognition network is used to process the image to be recognized to obtain the feature vector of the image to be recognized, wherein the pedestrian re-recognition network is obtained by training according to the training method of any one of claims 1-4 of; The recognition result of the image to be recognized is obtained by comparing the feature vector of the image to be recognized with the feature vector of the existing pedestrian image. A training device for pedestrian re-identification network, which is characterized in that it comprises: The acquisition unit is used to perform step 1; Step 1: Obtain M training images and annotation data of the M training images, each of the M training images includes a pedestrian, and the annotation data of each training image includes each of the training images The bounding frame and pedestrian identification information in which the pedestrian is located in, where different pedestrians correspond to different pedestrian identification information. Among the M training images, the training images with the same pedestrian identification information come from the same image capturing device, M Is an integer greater than 1; Training unit, used to perform step 2; Step 2: Initialize the network parameters of the pedestrian re-identification network to obtain the initial values of the network parameters of the pedestrian re-identification network; The training unit is also used to repeat steps 3 to 5 until the pedestrian re-identification network meets the preset requirements; Step 3: Input a batch of training images of the M training images to the pedestrian re-recognition network for feature extraction, and obtain a feature vector of each training image in the batch of training images; Wherein, the batch of training images includes N anchor point images, the N anchor point images are any N training images in the batch of training images, and each anchor point in the N anchor point images The image corresponds to a hardest positive sample image, a first hardest negative sample image and a second hardest negative sample image, N is a positive integer; The most difficult positive sample image corresponding to each anchor point image is the same as the pedestrian identification information of each anchor point image in the batch of training images, and is different from the feature vector of each anchor point image. The training image with the farthest distance, the first most difficult negative sample image corresponding to each anchor point image is from the same image capturing device as each anchor point image in the batch of training images, and the The pedestrian identification information of each anchor point image is different and the training image with the closest distance to the feature vector of each anchor point image, the second most difficult negative sample image corresponding to each anchor point image is the The distance between the batch of training images and each anchor point image is from a different image capturing device, and is different from the pedestrian identification information of each anchor point image, and the distance from the feature vector of each anchor point image The most recent training image; Step 4: Determine the function value of the loss function according to the feature vectors of the batch of training images, and the function value of the loss function is obtained by averaging the function values of the N first loss functions; Wherein, the function value of each first loss function in the N first loss functions is calculated according to the first difference and the second difference corresponding to each of the N anchor point images , The first difference value corresponding to each anchor point image is the difference between the most difficult positive sample distance corresponding to each anchor point image and the second most difficult negative sample distance corresponding to each anchor point image, The second difference value corresponding to each anchor point image is the difference between the second most difficult negative sample distance corresponding to each anchor point image and the first most difficult negative sample distance corresponding to each anchor point image, The most difficult positive sample distance corresponding to each anchor point image is the distance between the feature vector of the most difficult positive sample image corresponding to each anchor point image and the feature vector of each anchor point image. The second most difficult negative sample distance corresponding to the anchor point image is the distance between the feature vector of the second most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image. The first most difficult negative sample distance corresponding to the anchor point image is the distance between the feature vector of the first most difficult negative sample image corresponding to each anchor point image and the feature vector of each anchor point image; Step 5: Update the network parameters of the pedestrian re-identification network according to the function value of the loss function. The training device according to claim 6, wherein the pedestrian re-identification network meets preset requirements, comprising: When at least one of the following conditions is met, the pedestrian re-identification network meets the preset requirements: The number of times of training of the pedestrian re-recognition network is greater than or equal to a preset number of times; The function value of the loss function is less than or equal to a preset threshold; The recognition performance of the pedestrian re-identification network meets the preset requirements. 8. The training device according to claim 7, wherein the function value of the loss function is less than or equal to a preset threshold, comprising: The first difference is less than a first preset threshold, and the second difference is less than a second preset threshold. The training device according to any one of claims 6-8, wherein the M training images are training images from a plurality of image capturing devices, wherein the labeled data of the training images from different image capturing devices It is separately marked. A pedestrian re-identification device, which is characterized in that it comprises: The acquiring unit is used to acquire the image to be recognized; The recognition unit is configured to process the image to be recognized by using the pedestrian re-recognition network to obtain the feature vector of the image to be recognized, wherein the pedestrian re-recognition network is according to any one of claims 1-4 Training method of training; The recognition unit is further configured to compare the feature vector of the image to be recognized with the feature vector of the existing pedestrian image to obtain the recognition result of the image to be recognized. A computer-readable storage medium, wherein the computer-readable medium stores program code for device execution, and the program code includes a program code for executing the training method according to any one of claims 1-4. A computer-readable storage medium, wherein the computer-readable medium stores a program code for device execution, and the program code includes a program code for executing the pedestrian re-identification method according to claim 5. A chip, characterized in that the chip comprises a processor and a data interface, and the processor reads the instructions stored on the memory through the data interface to execute the method according to any one of claims 1-4 Training method. A chip, characterized in that the chip includes a processor and a data interface, and the processor reads instructions stored on a memory through the data interface to execute the pedestrian re-identification method according to claim 5.

Images