CN111126249A - A pedestrian re-identification method and device combining big data and Bayesian - Google Patents

Abstract

The invention discloses a pedestrian re-identification method and device combining big data and Bayesian. The method includes: using a pedestrian image database to perform distributed training on a pedestrian re-identification system model; Multiple candidate objects are re-identified and re-ranked based on Bayesian query expansion; the re-ranked query objects and candidate objects are processed by PTGAN; the query objects and candidate objects processed by PTGAN are input into the trained Bayesian model , calculate the true matching probability of each candidate object through the image distance in the training data, and reorder the candidate objects; adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model; A good pedestrian re-identification system model searches for the pedestrian image with the highest similarity. The invention solves the problems that the pedestrian re-identification method in the prior art is difficult to retrieve across cameras and has a low re-identification accuracy rate.

Description

A pedestrian re-identification method and device combining big data and Bayesian

technical field

The invention relates to the technical fields of computer vision and smart cities, and in particular to a pedestrian re-identification method, device, terminal device and computer-readable medium combining big data and Bayesian.

Background technique

With the continuous development of artificial intelligence, computer vision and hardware technology, video image processing technology has been widely used in smart city systems.

Pedestrian Re-identification (Person Re-identification) is also called Pedestrian Re-Identification, or Re-ID for short. It is a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video sequence. Widely regarded as a sub-problem of image retrieval. Given a surveillance pedestrian image, retrieve the pedestrian image across devices. Due to the differences between different camera devices, pedestrians are both rigid and flexible, and their appearance is easily affected by clothing, scale, occlusion, posture, and perspective, making pedestrian re-identification a field of computer vision. Challenging topic.

At present, although the detection ability of pedestrian re-identification has been significantly improved, many challenging problems have not been fully solved in practical situations: for example, in complex scenes, light differences, changes in perspective and posture, a large number of pedestrians In a surveillance camera network medium situation. In these cases, cross-camera retrieval is usually very difficult. At the same time, the labeling work during the training of video image samples in the early stage is expensive and requires a lot of manpower, and the existing algorithms often fail to achieve the expected results. lower.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a pedestrian re-identification method, device, terminal device and computer-readable medium combining big data and Bayesian, which can improve the accuracy rate of pedestrian re-identification with different cameras, and solve the problem. The pedestrian re-identification method in the prior art is difficult to retrieve across cameras, and the re-identification accuracy is low.

A first aspect of the embodiments of the present invention provides a pedestrian re-identification method combining big data and Bayesian, including:

Distributed training is performed on the pedestrian re-identification system model by using a pedestrian image database, and the trained pedestrian re-identification system model is obtained, wherein the pedestrian image database includes a plurality of matching image groups, and the matching image group includes at least two match image;

Input the query object into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects;

Re-identification and re-ranking based on Bayesian query expansion is performed on the query object and the plurality of candidate objects in the ranking list;

The reordered query objects and candidate objects are processed by PTGAN to realize the migration of the background difference area under the premise of the pedestrian foreground unchanged;

Input the query object and candidate object processed by PTGAN into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and reorder the candidate objects;

Multi-dimensional feature extraction is performed on the query object after PTGAN processing and the candidate object after reordering, and the inference clue model is determined;

using an inference algorithm to adjust the inference cue model and determine a final inference cue model;

Adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model;

By inputting the image to be recognized into the trained pedestrian re-identification system model, the pedestrian image with the highest similarity is searched.

Further, using the pedestrian image database to perform distributed training on the pedestrian re-identification system model, to obtain the pedestrian re-identification system model after the training, including:

iteratively train the person re-identification system model by increasing the batch size using multiple processors;

Iteratively train the pedestrian re-identification system model according to the linear scaling and warm-up strategy algorithm;

Applying adaptation rate scaling (LARS) uses different learning rates for each layer of the network in the person re-id system model.

Further, re-identification and re-ranking based on Bayesian query expansion is performed on the query object and multiple candidate objects in the ranking list, including:

Use the pedestrian image database to train the Bayesian model to obtain the trained Bayesian model;

According to the distance between the query object and a plurality of candidate object images, the true matching probability of each candidate object is predicted by the trained Bayesian model;

Query expansion is performed according to the true matching probability of each candidate object, and a new ranking list is generated through the query expansion.

Further, the multi-dimensional feature extraction is performed on the query object after PTGAN processing and the reordered candidate object and the inference clue model is determined, including:

Extract the appearance features of pedestrians;

Extract the facial features of pedestrians;

The positioning branch Markov chain is constructed according to the time and positioning features of pedestrians in different video heads, and the inference cue model is trained according to the positioning branch Markov chain.

A second aspect of the embodiments of the present invention provides a pedestrian re-identification device combining big data and Bayesian, characterized in that it includes:

The distributed training module is used to perform distributed training on the pedestrian re-identification system model by using a pedestrian image database to obtain the pedestrian re-identification system model after training, wherein the pedestrian image database includes a plurality of matching image groups, and the The matched image group includes at least two matched images;

a ranking list acquisition module, used for inputting the query object into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects;

A re-identification module for performing re-identification and re-ordering based on Bayesian query expansion on the query object and multiple candidate objects in the ranking list;

The PTGAN processing module is used to perform PTGAN processing on the reordered query objects and candidate objects, so as to realize the migration of the background difference area under the premise that the pedestrian foreground remains unchanged;

The training module is used to input the query object and candidate object processed by PTGAN into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and re-engineer the candidate object. sort;

The inference clue module is used to extract multi-dimensional features for the query object processed by PTGAN and the candidate object after reordering and determine the inference clue model;

an inference clue adjustment module, used for using an inference algorithm to adjust the inference clue model and to determine the final inference clue model;

a model adjustment module, configured to adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model;

The recognition module is used to search for the pedestrian image with the highest similarity by inputting the image to be recognized into the trained pedestrian re-identification system model.

Further, the distributed training module includes:

a processor augmentation module for iteratively training the person re-identification system model by using a plurality of processors to increase the batch size;

The batch algorithm module is used for iteratively training the pedestrian re-identification system model according to the linear scaling and warm-up strategy algorithm;

A learning rate adjustment module for applying adaptation rate scaling (LARS) to use a different learning rate for each layer of the network in the person re-identification system model.

Further, the re-identification module includes:

The Bayesian training module is used to train the Bayesian model using the pedestrian image database to obtain the trained Bayesian model;

a prediction module, configured to predict the true matching probability of each candidate object through the trained Bayesian model according to the distance between the query object and a plurality of candidate object images;

A query expansion module, configured to perform query expansion according to the true matching probability of each candidate object, and generate a new ranking list through the query expansion.

Further, the inference clue module includes:

Appearance extraction module, used to extract the appearance features of pedestrians;

The face extraction module is used to extract the facial features of pedestrians;

The positioning branch module is used to construct the positioning branch Markov chain according to the time and positioning features of pedestrians in different video heads, and train the inference clue model according to the positioning branch Markov chain.

A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program Steps to implement the above method of person re-identification combining big data and Bayesian.

A fourth aspect of the embodiments of the present invention provides a computer-readable medium, where the computer-readable medium stores a computer program, and when the computer program is processed and executed, realizes person re-identification combining the above-mentioned big data and Bayesian steps of the method.

In the embodiment of the present invention, by performing distributed training on the pedestrian re-identification system model, the speed of model training is greatly improved, and at the same time, through the re-identification and re-ordering based on Bayesian query expansion and PTGAN processing, the downlink of complex conditions is improved. The accuracy of person re-identification improves the robustness of the system. The problem of the pedestrian re-identification method in the prior art that the retrieval across cameras is difficult and the re-identification accuracy rate is low is solved.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is a flow chart of a pedestrian re-identification method combining big data and Bayesian provided by an embodiment of the present invention;

2 is a comparison diagram of real-time conversion effects of different pedestrian re-identification methods provided by an embodiment of the present invention;

3 is a schematic structural diagram of a pedestrian re-identification device combining big data and Bayesian provided by an embodiment of the present invention;

4 is a detailed structural diagram of a distributed training module provided by an embodiment of the present invention;

5 is a detailed structural diagram of a re-identification module provided by an embodiment of the present invention;

6 is a detailed structural diagram of an inference clue module provided by an embodiment of the present invention;

FIG. 7 is a schematic diagram of a terminal device provided by an embodiment of the present invention.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

In order to illustrate the technical solutions of the present invention, the following specific embodiments are used for description.

Please refer to FIG. 1. FIG. 1 is a flowchart of a method for pedestrian re-identification combining big data and Bayesian provided by an embodiment of the present invention. As shown in FIG. 1 , the pedestrian re-identification method combining big data and Bayesian in this embodiment includes the following steps:

Step S102, using the pedestrian image database to perform distributed training on the pedestrian re-identification system model to obtain a trained pedestrian re-identification system model, wherein the pedestrian image database includes multiple matching image groups, and the matching image group includes at least two matching images.

Further, using the pedestrian image database to perform distributed training on the pedestrian re-identification system model, to obtain the pedestrian re-identification system model after the training, including:

Step 1, iteratively train the person re-ID system model by using multiple processors to increase the batch size.

Reduce total training time by iterating the algorithm by scaling the algorithm to use more processors and loading more pedestrian image data at each iteration;

Generally speaking, larger batch sizes will make a single GPU faster, within certain limits. The reason is that a low-level matrix computation library will be more efficient. For training a Res-Net 50 model with ImageNet, the optimal batch size per GPU is 512. If you want to use many GPUs and make each GPU efficient, you will need a larger batch size. For example, if there are 16 GPUs, then the batch size should be set to 16×512=8192. Ideally, if the total number of accesses is fixed, and the batch size increases linearly with the number of processors, then the number of iterations of the improved SGD (stochastic gradient descent) will decrease linearly, and the time cost of each iteration remains constant. so the total time will also decrease linearly with the number of processors.

The specific improved stochastic gradient descent (SGD) iterative algorithm is as follows: let w represent the weight of the DNN, X represent the training data, n be the number of samples in X, and Y represent the labeling of the training data X. We let x _i be a sample of X, and ( _xi , w) be the loss computed by _xi and its label y _i (i∈{1,2,...,n)). The present invention uses a loss function such as a cross-entropy function. The goal of DNN training is to minimize the loss function in Equation (1). The formula is as follows:

In the t-th iteration, the algorithm of the present invention uses forward and backward propagation to obtain the gradient of the loss function to the weight. Then, using this gradient to update the weights, the equation (2) for updating the weights according to the gradient is as follows:

where η is the learning rate. The algorithm of the present invention sets the batch size of the t-th iteration as B _t and the size of B _t as b. The weights can then be updated based on the following equation (3):

更新权重w_t为w_t+1。This method is called mini-batch stochastic gradient descent. To simplify the expression, we can say that the update rule in equation (4) represents the gradient of our use weights

The update weight _wt is wt ₊₁ .

Using this method, iterating and using as many processors as possible at the same time can greatly reduce the training time linearly.

Step 2: Iteratively train the pedestrian re-identification system model according to the linear scaling and warm-up strategy algorithm.

When training large batches, you need to ensure that you achieve about the same test accuracy as small batches while running the same number of epochs. Here we fixed the number of epochs because: Statistically, an epoch means that the algorithm touches the entire dataset once; computationally, a fixed number of epochs means that Fixed number of floating point operations. Methods for training large batches include two techniques:

(1) Linear scaling: increase the batch from B to kB, then the learning rate should also be increased from η to kη.

(2) Warm-up strategy: If you use a large learning rate (η), you should start with a small value of η and then increase it to a large η in the first few epochs (epochs).

With linear scaling and warm-up strategies, relatively large batches of data images can be used within a certain range.

Step 3. Apply adaptation rate scaling (LARS) to use different learning rates for each layer of the network in the person re-id system model.

The final fast-training model is obtained by applying adaptation rate scaling (LARS) to the corresponding training of the large batch training layers.

来更新权重。使用数据并行方法，可以用相同的方式处理多机器版本。In order to improve the accuracy of large batch training, the method of the present invention uses a new update learning rate (LR) rule. The single machine situation must be considered here, using

to update the weights. Using a data-parallel approach, multi-machine versions can be processed in the same way.

标准SGD算法对所有层使用相同的LR(η)。然而，从日常实验中，可以观察到不同的层可能需要不同的LR。原因是||w||2和

之间的比率不同的层有很大的不同。Each layer has its own weight w and gradient

The standard SGD algorithm uses the same LR(n) for all layers. However, from daily experiments, it can be observed that different layers may require different LRs. The reason is that ||w||2 and

The ratios between the different layers are very different.

The present invention uses the LARS algorithm to solve this problem. The basic LR rule is defined in equation (1). l is the scaling factor. In this algorithm, l is set to 0.001 in AlexNet and ResNet training. γ is the tuning parameter of the user. Usually a good γ, the value is between [1, 50]. In this equation, different layers can have different LRs. Add momentum (denoted by μ) and weight decay (denoted by β) to SGD, and use the following sequence for LARS:

get the local LR for each learnable parameter,

Get the real LR of each layer, which is η=γ×α;

更新梯度；pass

update gradient;

更新加速项a；pass

Update acceleration item a;

The weights are updated with w=w-a.

Using this approach to warmup, SGD with large batch sizes can achieve the same accuracy as the baseline. To scale to larger batch sizes (e.g. 32k), local response normalization (LRN) needs to be changed to batch normalization (BN). The inventive method adds BN after each convolutional layer. LARS can help ResNet-50 maintain high test accuracy. Current methods (linear scaling and warmup) are much less accurate for batch sizes of 16k and 32k.

Step S104, the query object is input into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects.

Among them, the person re-identification system model can be any existing person re-identification network model such as MA-CNN or RNN.

Step S106, performing re-identification and re-ranking based on Bayesian query expansion on the query object and the multiple candidate objects in the ranking list.

Further, re-identification and re-ranking based on Bayesian query expansion is performed on the query object and multiple candidate objects in the ranking list, including:

Step 1, using the pedestrian image database to train the Bayesian model to obtain the trained Bayesian model;

Step 2, according to the distance between the query object and a plurality of candidate object images, predict the true matching probability of each candidate object through the trained Bayesian model;

Step 3: Perform query expansion according to the true matching probability of each candidate object, and generate a new ranking list through the query expansion.

为训练集，

为图库集。然后，将训练图像

查询图像q以及图库图像

的标识分别表示为

l^q以及

设

为查询图像q和图库图像

之间的距离。然后，初始秩列表表示为

其中

由此，基于离线训练的贝叶斯模型可以对初始秩列表进行重新排序。BQE uses information from the initial gallery rank list to generate new queries for retrieving gallery images again. Specifically, the dataset is divided into three parts: query, gallery and training data. In the offline process, Bayesian posterior estimation training is first performed on the training data. Given a distance metric, a Bayesian model can predict the true matching probability of a candidate object. During online retrieval, an initial rank list can be obtained by calculating the similarity between the query and gallery images. According to the ranking table, the true matching probability of each candidate object is calculated using the Bayesian model. Then, image features from the initial rank list with high probability are merged with the original query, thereby starting a new query to perform another round of retrieval. After a new round of retrieval, the query expansion process can be used again, resulting in an iterative algorithm. Formally, each image is represented by a d-dimensional feature vector, denoted as x ∈ R ^d . make

is the training set,

for the gallery set. Then, the training image

Query image q as well as gallery images

are identified as

l ^q and

Assume

For query images q and gallery images

the distance between. Then, the initial rank list is expressed as

in

Thus, the initial rank list can be reordered based on the offline-trained Bayesian model.

In step S108, PTGAN processing is performed on the reordered query objects and candidate objects, so as to realize the migration of the background difference area under the premise that the pedestrian foreground remains unchanged.

PTGAN (Person Transfer GAN) is a generative adversarial network for re-identification Re-ID problem. In the present invention, the biggest feature of PTGAN is to realize the migration of background area differences on the premise that the pedestrian foreground is kept unchanged as much as possible. First of all, the loss function of the PTGAN network consists of two parts:

Among them, L _Style represents the generated style loss, or the regional difference domain loss, which is whether the generated image resembles the new dataset style. L _ID stands for the identity loss of the generated image, which is to verify whether the generated image is the same person as the original image. _λ1 here is the weight that balances the two losses. The two losses are defined as follows:

First, the loss function (Loss) of PTGAN mentioned in the present invention is divided into two parts; the first part is L _Style , and its specific formula is as follows:

代表标准对抗性损失，L_Cyc代表周期一致性损失,A、B为两帧做GAN处理的图像，令G为图像A到B风格映射功能函数，

为B到A的风格映射功能函数，λ2为分割损失和身份损失的权重。in,

Represents the standard adversarial loss, L _Cyc represents the cycle consistency loss, A and B are the images processed by GAN for two frames, let G be the image A to B style mapping function,

is the style mapping function from B to A, and λ2 is the weight of segmentation loss and identity loss.

The above parts are the losses of normal PTGAN, the purpose is to ensure that the difference area (domain) of the generated image and the expected dataset is the same.

Secondly, in order to ensure that the foreground remains unchanged during the image transfer process, PSPNet is used to perform a foreground segmentation on the video image to obtain a mask (mask layer) area. Generally speaking, traditional generative adversarial networks such as CycleGAN are not used for Re-ID tasks, so there is no need to ensure that the identity information of foreground objects remains unchanged. The result is that the quality of the foreground may be blurred, and the quality is even worse. The phenomenon is that the appearance of pedestrians may change. In order to solve this problem, the present invention proposes L _ID loss, the foreground extracted by PSPNet, this foreground is a mask layer mask, and the final loss of identity information is:

Among them, M(a) and M(b) are the two segmented foreground mask layers, and the identity information loss function (Loss) will constrain the pedestrian foreground to remain as unchanged as possible during the migration process.

是是图像b中转移的行人图像，

为A的数据分布，

为B的数据分布，M(a)和M(b)是两个分割出来的面具层区域。where G(a) is the transferred pedestrian image in image a,

is the transferred pedestrian image in image b,

is the data distribution of A,

For the data distribution of B, M(a) and M(b) are the two segmented mask layer regions.

Figure 2 shows a comparison chart of real-time conversion effects of different pedestrian re-identification methods, in which the first row of pictures is the picture to be converted, and the fourth row shows the result of PTGAN conversion. Compared with the three-line pictures, the images generated by PTGAN are of higher quality. For example, the appearance of the person remains the same and the style is effectively transferred to another camera. Shadows, road markings and backgrounds are automatically generated, similar to those captured by another camera. Meanwhile, PTGAN can handle the noisy segmentation results produced by PSPNet well. It can be seen that the algorithm of the present invention can better guarantee the identity information of pedestrians intuitively compared with the traditional ring generative adversarial network (CycleGAN).

Step S110 , input the query object and candidate object after PTGAN processing into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and reorder the candidate objects.

Essentially, a Bayesian model represents the distribution of match scores for true matches and false matches. The model is created on the training set and used during testing to estimate the probability that the top image truly matches the query.

For each image x in the ranking list returned from the person re-id system, there is a distance computed by the learned metric. Since images with a smaller distance from the query will be listed at the top, you need to know if you can use the top image to reorder the image list to improve performance. The selection of candidate images is critical, as a false match will have an opposite impact on performance. When the distance P(x|d(x,q)) between the query and the image is given, since images with the same or different features usually have significantly different distance ranges, the present invention uses the distance to distinguish candidate objects. The present invention uses a Bayesian model to estimate the probability of image correlation in the ranking table.

Specifically, for a query q and a gallery image x, based on the distance between the two images, the probability that the two images belong to the same feature is usually calculated, namely

According to Bayes' theorem, the probability can be rewritten as follows:

可以通过以下公式计算，in

It can be calculated by the following formula,

和

来计算

和

的近似值。为了计算

和

需要计算训练数据中每个图像之间的距离，并使用距离范围来替代距离确切值。本发明将距离范围

值)划分成M个区间，然后计算每个间隔内的候选数。假设

在[0.2,0.3]这个区间内，则

可由候选数除以该区间中红柱的频率来计算。

的计算方法与之类似。间隔M的数量是根据数据集的大小来选择的。实际操作中，如果测试阶段的距离大于训练阶段的上限(或小于下限)，则使用上限(或下限)的结果。The present invention utilizes training data to estimate probabilities. can be used directly

and

to calculate

and

approximate value. in order to calculate

and

The distance between each image in the training data needs to be calculated and the distance range is used instead of the exact distance value. The present invention will range the distance

value) is divided into M intervals, and then the number of candidates in each interval is calculated. Assumption

In the interval [0.2, 0.3], then

It can be calculated by dividing the number of candidates by the frequency of the red bars in the interval.

The calculation method is similar. The number of intervals M is chosen according to the size of the dataset. In practice, if the distance in the test phase is greater than the upper limit (or smaller than the lower limit) in the training phase, the upper (or lower) result is used.

For query expansion, a new query is invoked to reorder the candidates. Only k high-probability candidates are merged into a new query, where K≤number of true matches and k<<n. The value of K is adjusted according to the actual situation. Feature pooling strategies are diverse.

There are two simple strategies for query expansion: Average Query Expansion (AQE) and Maximum Query Expansion (MQE). For both methods, average pooling and max pooling are used to fuse the features of the query image and top candidates, respectively. For AQE, the extended query is calculated as:

The disadvantage of these strategies is that their effectiveness depends heavily on the quality of the initial sorting table and the value of the parameter k. When the initial sorted table is not satisfied or the value of k is large, false matches will be used to construct a new query, which will affect the precision of the query.

To overcome this deficiency, the present invention assigns different weights to each candidate object during feature pooling. Then, an extended probe _qnew for the initial query q is computed by unifying the top K images and the query q with the probability. Here, the present invention simply uses weighted average pooling, where the weights are the probabilities. The formula is as follows:

Finally use this new query to calculate the distance and rearrange the initial ranking list. Then iterate over the result for more iterations. Often the expanded query will produce a better ranked list, which can result in a better query. The present invention can repeatedly perform the process of generating ranked lists, feature pools, and query expansion. By repeating the BQE, the effect will be enhanced. Denote T as the number of iterations.

离线计算。对于查询扩展过程，需要计算概率并构造新的查询。生成新查询的时间复杂度是

其中K是池化图像的数量。由于参数K小于真匹配数，并且K≤N，所以可以将复杂度限制为

然后用复杂度

计算信任图像的成对距离。得出结果，对于一个查询，计算复杂度为

Suppose the training set and gallery set are of size M and N, respectively. Bayesian Models with Complexity

Offline computing. For the query expansion process, probabilities need to be calculated and new queries constructed. The time complexity of generating a new query is

where K is the number of pooled images. Since the parameter K is less than the number of true matches, and K≤N, the complexity can be limited to

Then use the complexity

Calculate the pairwise distance of the trust images. As a result, for a query, the computational complexity is

Step S112 , perform multi-dimensional feature extraction on the query object after PTGAN processing and the reordered candidate object, and determine an inference clue model.

The present invention uses appearance, face and possible destination cues, and features for each timestamp are extracted individually for all detections across cameras.

其中

W和b是输入数据Xi的隐含观测量，分别有各自的权重与偏差，而f是激活隐藏层的校正线性单元。基于上述步骤，对每个时间戳的连续帧视频图像中的行人进行外观特征提取。Appearance-based attributes are first extracted from person detection, which capture individual traits and characteristics in the form of appearance. The common denominator of image representations is Convolutional Neural Networks (CNN). The present invention uses the AlexNet model pre-trained on ImageNet as the appearance feature extractor. This is done by removing the top output layer and using the activations of the last fully connected layer as features (length 4096). The AlexNet architecture consists of five convolutional layers, three fully connected layers, and three max-pooling layers followed by the first, second, and fifth convolutional layers. The first convolutional layer has 96 filters of size 11×11, the second layer has 256 filters of size 5×5, the third, fourth and fifth layers are connected to each other without any interference pooling, and have 384/384 and 256 filters of size 3×3, respectively. Fully connected layer L learns nonlinear functions

in

W and b are the implicit observations of the input data Xi, with their own weights and biases, respectively, and f is the rectified linear unit that activates the hidden layer. Based on the above steps, appearance feature extraction is performed on pedestrians in consecutive frames of video images at each timestamp.

Second, facial features are extracted, and facial biometrics is an established biometric technology used for identification and verification. Face morphology can be used for re-identification because it is essentially a non-contact biometric and can be extracted remotely. The present invention extracts facial features from facial bounding boxes using the VGG-16 model pre-trained on ImageNet. This is done by removing the top output layer and using the activations of the last fully connected layer as facial features (length 4096). VGG-16 is a convolutional neural network whose structure consists of 13 convolutional layers and 3 fully connected layers with a filter size of 3 × 3. Pooling will be applied between convolutional layers with a 2×2 pixel window with a stride of 2. The mean subtraction of the training set is used as a preprocessing step.

At the same time, the present invention describes position constraints, which are linear in nature and predict the most probable paths within and between cameras. For re-identification and tracking across multiple cameras, knowledge about possible destinations is taken as a priori that a person is present in another camera's field of view. Typically, transition probability distributions are modeled by learning recurring patterns that occur in the camera network. An individual exiting a camera view from a particular grid space is likely to enter another camera view from another particular grid space. The present invention models the state transition probability distribution as a Markov chain, and each camera view is divided into n states. Assuming that there are k cameras, the total number of states is N=n×k. A Markov chain is described as an n×n transition probability matrix p with each entry in the interval [0,1] and the sum of the entries in each row adding up to 1.

Therefore, using the Markov property, the probability distribution of transitions between states S _i and S _j is estimated as:

After the above-mentioned multi-scale feature extraction, an inference cue model is trained.

Step S114, use an inference algorithm to adjust the inference clue model and determine the final inference clue model.

At each time step, the problem of re-identification can be represented by an association matrix, where each row represents a previously seen entity and the column contains the currently active entity. The task of optimally correlating each row and column according to the characteristics or properties of the related entities can be formulated as a linear programming problem as follows:

st W∈[0, 1], W1=1, 1 ^T W=1

where p is the affinity matrix or probability matrix to store the matching probabilities of the associated entities, and w is the weight matrix to be optimized. Figure 3 depicts how the proposed inference algorithm works on the association matrix P. The matching probability in the association matrix is the cosine distance of each mid-level attribute and face feature computed using pretrained Alexnet and VGG-16 models, respectively, or the location score, a transition probability model of possible movement patterns between entities.

The effect of the constraint w1=1 is to normalize the matching probabilities from column to column and force them to sum to 1 for each previous entity. From the expression of this constraint, it is clear that there is only one maximum value for the set of associated probabilities for each previous entity. This means that each previous entity can only be associated with at most one current entity. Therefore, choosing the value of the weight matrix w essentially reduces the assignment of a value of 1 to the best association, so computing the best possible association is equivalent to the greedy method of choosing the largest matching probability in order. Finally, combined with the constraints of each feature extraction, the final inference clue model is determined.

The overall objective function can be expressed as:

where Θ denotes the parameters in the inference model. _L1 , _L2 and L3 _denote the classification losses in the face, appearance, localization branches, respectively. λ ₁ , λ ₂ , λ ₃ represent the weights of the corresponding losses.

Step S116, adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model.

Step S118 , by inputting the image to be recognized into the trained pedestrian re-identification system model, search out the pedestrian image with the highest similarity.

In the embodiment of the present invention, by performing distributed training on the pedestrian re-identification system model, the speed of model training is greatly improved, and at the same time, through the re-identification and re-ordering based on Bayesian query expansion and PTGAN processing, the downlink of complex conditions is improved. The accuracy of person re-identification improves the robustness of the system. The problem of the pedestrian re-identification method in the prior art that the retrieval across cameras is difficult and the re-identification accuracy rate is low is solved.

Please refer to FIG. 3. FIG. 3 is a structural block diagram of a person re-identification device combining big data and Bayesian provided by an embodiment of the present invention. As shown in FIG. 3 , the person re-identification 20 combining big data and Bayesian in this embodiment includes a ranking distributed training module 202 , a ranking list obtaining module 204 , a re-identification module 206 , a PTGAN processing module 208 , and a training module 210 , an inference clue module 212 , an inference clue adjustment module 214 , a model adjustment module 216 and a recognition module 218 . The distributed training module 202, the ranking list acquisition module 204, the re-identification module 206, the PTGAN processing module 208, the training module 210, the inference clue module 212, the inference clue adjustment module 214, the model adjustment module 216, and the identification module 218 are respectively used to execute the graph For the specific methods in S102, S104, S106, S108, S110, S112, S114, S116, and S118 in 1, please refer to the relevant introduction in Figure 1 for details, which are only briefly described here:

The distributed training module 202 is used to perform distributed training on the pedestrian re-identification system model by using the pedestrian image database to obtain the pedestrian re-identification system model after training, wherein the pedestrian image database includes a plurality of matching image groups, and the matching image group includes at least two matching images;

The ranking list obtaining module 204 is used to input the query object into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects;

A re-identification module 206, configured to perform re-identification and re-ranking based on Bayesian query expansion on the query object and a plurality of candidate objects in the ranking list;

The PTGAN processing module 208 is used to perform PTGAN processing on the reordered query objects and candidate objects, so as to realize the migration of the background difference area under the premise that the pedestrian foreground remains unchanged;

The training module 210 is used to input the query object and candidate object after PTGAN processing into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and reorder the candidate objects ;

The inference clue module 212 is used to perform multi-dimensional feature extraction on the query object after PTGAN processing and the reordered candidate object and determine the inference clue model;

an inference clue adjustment module 214, configured to use an inference algorithm to adjust the inference clue model and determine the final inference clue model;

A model adjustment module 216, configured to adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model;

The identification module 218 is configured to search for a pedestrian image with the highest similarity by inputting the image to be identified into the trained pedestrian re-identification system model.

Further, referring to FIG. 4 , the distributed training module 202 includes:

a processor increasing module 2021, configured to iteratively train the pedestrian re-identification system model by increasing the batch size by using multiple processors;

A batch algorithm module 2022, configured to iteratively train the pedestrian re-identification system model according to the linear scaling and warm-up strategy algorithm;

The learning rate adjustment module 2023 is used for applying adaptation rate scaling (LARS) to use different learning rates for each layer of the network in the person re-identification system model.

Further, referring to FIG. 5 , the re-identification module 206 includes:

The Bayesian training module 2061 is used to train the Bayesian model by using the pedestrian image database to obtain the trained Bayesian model;

A prediction module 2062, configured to predict the true matching probability of each candidate object through the trained Bayesian model according to the distance between the query object and multiple candidate object images;

The query expansion module 2063 is configured to perform query expansion according to the true matching probability of each candidate object, and generate a new ranking list through the query expansion.

Further, referring to FIG. 6 , the reasoning clue module 212 includes:

Appearance extraction module 2121, used to extract the appearance features of pedestrians;

The face extraction module 2122 is used to extract the facial features of pedestrians;

The positioning branch module 2123 is used to construct a positioning branch Markov chain according to the time and positioning features of pedestrians in different video heads, and train an inference clue model according to the positioning branch Markov chain.

In the embodiment of the present invention, by performing distributed training on the pedestrian re-identification system model, the speed of model training is greatly improved, and at the same time, through the re-identification and re-ordering based on Bayesian query expansion and PTGAN processing, the downlink of complex conditions is improved. The accuracy of person re-identification improves the robustness of the system. The problem of the pedestrian re-identification method in the prior art that the retrieval across cameras is difficult and the re-identification accuracy rate is low is solved.

FIG. 7 is a schematic diagram of a terminal device provided by an embodiment of the present invention. As shown in FIG. 7 , the terminal device 10 in this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and running on the processor 100, for example, to perform big data and A procedure for pedestrian re-identification combined with Yeas. When the processor 100 executes the computer program 102 , the steps in the above method embodiments are implemented, for example, the steps of S102 , S104 , S106 , S108 , S110 , S112 , S114 , S116 , and S118 shown in FIG. 1 . Alternatively, when the processor 100 executes the computer program 102, the functions of the modules/units in the above device embodiments are implemented, such as the distributed training module 202, the ranking list obtaining module 204, and the re-identification module 206 shown in FIG. 7 . , PTGAN processing module 208, training module 210, inference clue module 212, inference clue adjustment module 214, model adjustment module 216 and recognition module 218 functions.

Exemplarily, the computer program 102 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 101 and executed by the processor 100 to complete the this invention. The one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 102 in the terminal device 10 . For example, ranking distributed training module 202 , ranking list acquisition module 204 , re-identification module 206 , PTGAN processing module 208 , training module 210 , inference cue module 212 , inference cue adjustment module 214 , model adjustment module 216 , and recognition module 218 . (Module in the virtual device), the specific functions of each module are as follows:

The distributed training module 202 is used to perform distributed training on the pedestrian re-identification system model by using the pedestrian image database to obtain the pedestrian re-identification system model after training, wherein the pedestrian image database includes a plurality of matching image groups, and the matching image group includes at least two matching images;

The ranking list obtaining module 204 is used to input the query object into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects;

A re-identification module 206, configured to perform re-identification and re-ranking based on Bayesian query expansion on the query object and a plurality of candidate objects in the ranking list;

The PTGAN processing module 208 is used to perform PTGAN processing on the reordered query objects and candidate objects, so as to realize the migration of the background difference area under the premise that the pedestrian foreground remains unchanged;

The training module 210 is used to input the query object and candidate object after PTGAN processing into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and reorder the candidate objects ;

The inference clue module 212 is used to perform multi-dimensional feature extraction on the query object after PTGAN processing and the reordered candidate object and determine the inference clue model;

an inference clue adjustment module 214, configured to use an inference algorithm to adjust the inference clue model and determine the final inference clue model;

A model adjustment module 216, configured to adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model;

The identification module 218 is configured to search for a pedestrian image with the highest similarity by inputting the image to be identified into the trained pedestrian re-identification system model.

The terminal device 10 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device 10 may include, but is not limited to, a processor 100 and a memory 101 . Those skilled in the art can understand that FIG. 7 is only an example of the terminal device 10, and does not constitute a limitation on the terminal device 10, and may include more or less components than the one shown, or combine some components, or different components For example, the terminal device may further include an input and output device, a network access device, a bus, and the like.

The processor 100 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 101 may be an internal storage unit of the terminal device 10 , such as a hard disk or a memory of the terminal device 10 . The memory 101 may also be an external storage device of the terminal device 10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card equipped on the terminal device 10, Flash card (Flash Card) and so on. Further, the memory 101 may also include both an internal storage unit of the terminal device 10 and an external storage device. The memory 101 is used to store the computer program and other programs and data required by the terminal device 10 . The memory 101 may also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Electric carrier signals and telecommunication signals are not included.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the foregoing implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the within the protection scope of the present invention.

Claims (10)

1. a pedestrian re-identification method combining big data and Bayesian, is characterized in that, comprises: Distributed training is performed on the pedestrian re-identification system model by using a pedestrian image database, and the trained pedestrian re-identification system model is obtained, wherein the pedestrian image database includes a plurality of matching image groups, and the matching image group includes at least two match image; Input the query object into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects; Re-identification and re-ranking based on Bayesian query expansion is performed on the query object and the plurality of candidate objects in the ranking list; The reordered query objects and candidate objects are processed by PTGAN to realize the migration of the background difference area under the premise of the pedestrian foreground unchanged; Input the query object and candidate object processed by PTGAN into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and reorder the candidate objects; Multi-dimensional feature extraction is performed on the query object after PTGAN processing and the candidate object after reordering, and the inference clue model is determined; using an inference algorithm to adjust the inference cue model and determine a final inference cue model; Adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model; By inputting the image to be recognized into the trained pedestrian re-identification system model, the pedestrian image with the highest similarity is searched. 2. the pedestrian re-identification method combining big data and Bayesian according to claim 1, is characterized in that, utilizes pedestrian image database to carry out distributed training to pedestrian re-identification system model, obtains described pedestrian re-identification after training. Identify system models, including: iteratively train the person re-identification system model by increasing the batch size using multiple processors; Iteratively train the pedestrian re-identification system model according to the linear scaling and warm-up strategy algorithm; Applying adaptation rate scaling (LARS) uses different learning rates for each layer of the network in the person re-id system model. 3. The pedestrian re-identification method combining big data and Bayesian according to claim 1, is characterized in that, carrying out the re-identification based on Bayesian query expansion on the query object and a plurality of candidate objects in the ranking list. Identify reordering, including: Use the pedestrian image database to train the Bayesian model to obtain the trained Bayesian model; According to the distance between the query object and a plurality of candidate object images, the true matching probability of each candidate object is predicted by the trained Bayesian model; Query expansion is performed according to the true matching probability of each candidate object, and a new ranking list is generated through the query expansion. 4. the pedestrian re-identification method combining big data and Bayesian according to claim 3, it is characterized in that, the described query object after carrying out PTGAN processing and the candidate object after carrying out reordering are carried out multi-dimensional feature extraction And identify inference cue models, including: Extract the appearance features of pedestrians; Extract the facial features of pedestrians; The positioning branch Markov chain is constructed according to the time and positioning features of pedestrians in different video heads, and the inference cue model is trained according to the positioning branch Markov chain. 5. A pedestrian re-identification device combining big data and Bayesian features, comprising: The distributed training module is used to perform distributed training on the pedestrian re-identification system model by using a pedestrian image database to obtain the pedestrian re-identification system model after training, wherein the pedestrian image database includes a plurality of matching image groups, and the The matched image group includes at least two matched images; a ranking list acquisition module, used for inputting the query object into the pedestrian re-identification system model to obtain a ranking list of multiple candidate objects; A re-identification module for performing re-identification and re-ordering based on Bayesian query expansion on the query object and multiple candidate objects in the ranking list; The PTGAN processing module is used to perform PTGAN processing on the reordered query objects and candidate objects, so as to realize the migration of the background difference area under the premise that the pedestrian foreground remains unchanged; The training module is used to input the query object and candidate object processed by PTGAN into the trained Bayesian model, calculate the true matching probability of each candidate object through the image distance in the training data, and re-engineer the candidate object. sort; The inference clue module is used to extract multi-dimensional features for the query object processed by PTGAN and the candidate object after reordering and determine the inference clue model; an inference clue adjustment module, used for using an inference algorithm to adjust the inference clue model and to determine the final inference clue model; a model adjustment module, configured to adjust the parameter value of the target parameter of the pedestrian re-identification system model according to the inference clue model; The recognition module is used to search for the pedestrian image with the highest similarity by inputting the image to be recognized into the trained pedestrian re-identification system model. 6. The pedestrian re-identification device combined with big data and Bayesian according to claim 5, is characterized in that, described distributed training module comprises: a processor augmentation module for iteratively training the person re-identification system model by using a plurality of processors to increase the batch size; The batch algorithm module is used for iteratively training the pedestrian re-identification system model according to the linear scaling and warm-up strategy algorithm; A learning rate adjustment module for applying adaptation rate scaling (LARS) to use a different learning rate for each layer of the network in the person re-identification system model. 7. The pedestrian re-identification device combining big data and Bayesian according to claim 5, wherein the re-identification module comprises: The Bayesian training module is used to train the Bayesian model using the pedestrian image database to obtain the trained Bayesian model; a prediction module, configured to predict the true matching probability of each candidate object through the trained Bayesian model according to the distance between the query object and a plurality of candidate object images; A query expansion module, configured to perform query expansion according to the true matching probability of each candidate object, and generate a new ranking list through the query expansion. 8. The pedestrian re-identification device combining big data and Bayesian according to claim 5, wherein the inference clue module comprises: Appearance extraction module, used to extract the appearance features of pedestrians; The face extraction module is used to extract the facial features of pedestrians; The positioning branch module is used to construct the positioning branch Markov chain according to the time and positioning features of pedestrians in different video heads, and train the inference clue model according to the positioning branch Markov chain. 9. A terminal device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the computer program as claimed in the claims when executing the computer program The steps of any one of 1-4. 10. A computer-readable medium storing a computer program, characterized in that, when the computer program is processed and executed, the steps of the method according to any one of claims 1-4 are implemented.

Images