CN115239770A - A Kernel Correlation Filtering Target Tracking Method and System for Severely Occluded Scenes - Google Patents

Abstract

The invention relates to a nuclear correlation filtering target tracking method and a nuclear correlation filtering target tracking system for a severely occluded scene; the method comprises the following steps: step 1, initializing a KCF algorithm of a KCF filter and a Kalman filtering algorithm of the Kalman filter, and taking a target position of a first frame as an initial position of the KCF algorithm and the Kalman filtering algorithm; step 2, judging whether the target is seriously shielded in the current image by using a shielding detection mechanism; if not, adopting a tracking result of a KCF algorithm as the position of the target in the current image; if so, adopting a tracking result of a Kalman filtering algorithm as the position of the target in the current image; step 3, respectively carrying out self-adaptive updating on the Kalman filter and the KCF filter according to respective tracking results; step 4, reading the next video frame, and returning to execute the step 2; until the tracking process is finished. The invention can still realize the tracking of the target when the target is seriously shielded.

Description

A Kernel Correlation Filtering Target Tracking Method and System for Severely Occluded Scenes

technical field

The invention belongs to the technical field of target tracking, and in particular relates to a kernel correlation filtering target tracking method and system for severely occluded scenes.

Background technique

Target tracking can be mainly divided into generative algorithms and discriminative algorithms. The main difference between the two algorithms lies in the way of modeling the target model.

Generative algorithm: This kind of method first realizes the construction of the target tracking model by extracting the target features in the first frame image, and then extracts the candidate samples of the target in the subsequent video images, and performs similarity detection with the target model to obtain the target model. The most similar sample (that is, the tracked target position), and the position of this sample is used as the target position, and finally the target information is updated to the target model. This kind of method has obvious defects such as low utilization of image background information, randomness and diversity of the appearance of the target itself. The specific performance is that in the case of illumination changes, motion blur, low resolution, target rotation deformation, etc., the establishment of the model will be greatly affected, thereby affecting the accuracy of tracking; there is no effective prediction mechanism for the establishment of the model. The situation cannot be solved well.

Discriminant algorithm: This method first uses the target positive sample information and background negative sample information to train the target tracking model, then extracts candidate regions in the subsequent video images, and selects the optimal candidate region as the target position through the discriminative model. The discriminative model is updated accordingly. Compared with the generative method, the discriminative method uses the background information to train the classifier, which makes the classifier have stronger discrimination ability and can better distinguish the foreground and background. Therefore, the discriminative method is generally better than the generative model method, and the tracking performance is better. Robustness is stronger, and it gradually occupies the mainstream position in the field of target tracking.

Kernel correlation filter tracking algorithm is a classic discriminative target tracking algorithm, which can be well applied to small-angle rotation of the target, slight occlusion and some other disturbances. The main idea is to use a given sample to train a discriminative classifier, and obtain a similarity heat map by calculating the cyclic correlation between the features of the search area and the target area. The displacement between two adjacent frames finally determines the movement trajectory of the target. The algorithm uses circulant matrices to sample samples during computations, which enables the use of fast Fourier transforms to speed up computations. The process framework of the traditional kernel correlation filtering algorithm can be summarized as follows:

In the initial image, according to the given target feature information, initialize the relevant target tracking filter;

Read the post-sequence video image, extract the candidate sample feature information according to the position and size of the target in the previous frame, and perform cosine window processing on the sample feature data to reduce the boundary effect, and then perform fast Fourier transform processing to realize the candidate sample information from time to time. domain to frequency domain conversion;

The correlation calculation operation is performed on the feature information of the candidate samples and the target filter in the frequency domain, and the Gaussian response map is obtained by inverse fast Fourier transform processing, and the position with the largest response in the response map is used as the tracking target position;

Through the model online update strategy, the new target appearance information is updated to the target tracking filter and the target feature model, so that the kernel correlation filter model can adapt to the change of the target.

Although the kernel correlation filter tracking algorithm can complete effective tracking in the face of slight occlusion, and has certain anti-interference ability and robustness, it will fail to track in the face of severe occlusion. Therefore, there is an urgent need to optimize the existing kernel correlation filtering algorithms for severe occlusion scenarios.

SUMMARY OF THE INVENTION

For the scene where the target is severely occluded, since the kernel correlation filter tracking algorithm has no corresponding occlusion detection mechanism and update stop mechanism, the algorithm has no ability to distinguish the occluder and the target, and the feature information of a large number of occluders is updated into the target model, which causes The pollution of the target model eventually leads to the failure of the target tracking process. The purpose of the present invention is to provide a kernel correlation filtering target tracking method and system for severely occluded scenes, so as to solve the problem of target tracking drift when the target is severely occluded.

In order to realize the above-mentioned purpose of the invention, the present invention adopts the following technical solutions:

A kernel correlation filter tracking method for severely occluded scenes, comprising the following steps:

Step 1, initialize the KCF algorithm of the KCF filter and the Kalman filter algorithm of the Kalman filter, take the target position of the first frame as the initial position of the KCF algorithm and the Kalman filter algorithm;

Step 2. Use the occlusion detection mechanism to determine whether the target is severely occluded in the current image; if not, use the tracking result of the KCF algorithm as the position of the target object in the current image; if so, use the tracking result of the Kalman filtering algorithm as the target object position in the current image;

Step 3, according to the respective tracking results, adaptively update the Kalman filter and the KCF filter;

Step 4: Read the next video frame, and return to step 2 until the tracking process ends.

As a preferred solution, in the step 2, a joint index occlusion detection mechanism is used to process the image to determine whether the target in the current image is severely occluded.

As a preferred solution, the joint index occlusion detection mechanism is:

(1) Similarity judgment based on fusion features

The feature fusion index f is constructed using HSV and LBP features, including:

Based on the detection area of the KCF algorithm, the HSV and LBP features of the target and background areas are extracted respectively, and the extracted (HSV _target , HSV _background ) and (LBP _target , LBP _background ) are normalized using the following formulas. The processed feature vector is x=(x ₁ , x ₂ ,..., x _n-1 , x _n ) ^T , and the normalized feature vector is x ^* :

After obtaining the normalized eigenvectors, calculate the Euclidean distance D _HSV between the HSV _target and the HSV _background , and the Euclidean distance D _LBP between the LBP _target and the LBP _background :

表示目标的HSV特征向量的第i个分量值，

表示目标的LBP特征向量的第i个分量值，

表示背景区域的HSV特征向量的第i个分量值，

表示背景区域的LBP特征向量的第i个分量值；in,

represents the i-th component value of the HSV feature vector of the target,

represents the ith component value of the LBP feature vector of the target,

represents the i-th component value of the HSV feature vector of the background region,

represents the ith component value of the LBP feature vector of the background region;

Index D is used to reflect the degree of distinction between the target and the background under this feature;

The weights of HSV and LBP in the fusion feature are calculated as follows:

γ _HSV = 1 - γ _LBP

和

By extracting the HSV and LBP features of the target features of two adjacent frames, the feature similarity of the target in the kth frame is calculated.

and

Among them, HSV _k and HSV _k-1 represent the HSV features of the target in the kth frame and the k-1th frame, and LBP _k and LBP _k-1 represent the kth frame. The LBP feature of the target in the k-1th frame;

The feature fusion f is obtained by weighted summation of the weights γ _LBP and γ _HSV :

Use fusion feature f for similarity judgment: when the Nth frame is executed, first calculate the average similarity distance s using the obtained historical similarity distance data, and calculate the corresponding threshold th ₁ , the calculation formula is as follows:

th ₁ =δ ₁ *s (1<δ ₁ <2)

Among them, δ ₁ represents the threshold coefficient of the average similarity distance s;

Then calculate the Euclidean distance f _N of the target fusion features of the N-1th and N frames, and judge whether the Euclidean distance f _N is greater than th ₁ ; if not, no occlusion has occurred; if so, occlusion may occur;

The indicator of whether occlusion occurs is ε ₁ , and ε ₁ is expressed as:

(2) Judgment of tracking effect based on maximum response value

Use the maximum response value F _max to judge the tracking effect, including: when the execution reaches the Nth frame, first, count the historical maximum response value information of the previous N-1 frames, calculate the average maximum response value m and the threshold th ₂ , m and The calculation formula of th ₂ is as follows:

th ₂ =δ ₂ *m (0<δ ₂ <1)

表示第i帧的最大响应值，δ₂表示平均最大响应响应值m的阈值系数；in,

represents the maximum response value of the i-th frame, and δ ₂ represents the threshold coefficient of the average maximum response response value m;

是否大于th₂；若是，则当前并未发生遮挡；若否，则可能发生了遮挡；Then, determine the maximum response value of the Nth frame

Whether it is greater than th ₂ ; if so, no occlusion has occurred; if not, occlusion may have occurred;

The indicator of whether occlusion occurs is ε ₂ , and ε ₂ is expressed as:

(3) Evaluation mechanism based on average correlation peak ratio

Using the average correlation peak ratio APCE to evaluate the tracking reliability, including: on the basis of the response matrix, using the average correlation peak ratio to evaluate the tracking effect of the nuclear correlation filter tracking method; the APCE calculation formula is as follows:

Among them, F _max represents the maximum response value of the current frame, and F _min represents the minimum response value of the current frame;

和阈值th₃，用于评估当前帧，即第N帧的跟踪效果，计算公式如下：Calculate the average of the first N-1 frames

and the threshold th ₃ , which are used to evaluate the tracking effect of the current frame, that is, the Nth frame. The calculation formula is as follows:

的阈值系数；where δ ₃ represents the average correlation peak ratio

The threshold coefficient of ;

是否大于th₃；若是，则未发生遮挡；若否，则可能发生遮挡；judge

Whether it is greater than th ₃ ; if so, no occlusion occurs; if not, occlusion may occur;

The indicator of whether occlusion occurs is ε ₃ , and ε ₃ is expressed as:

The joint index ∈ is constructed as:

∈=ε ₁ +ε ₂ +ε ₃

When ∈=0 or 1, it means that the target is not occluded or slightly occluded, and the result of the KCF algorithm is used as the tracking result;

When ∈=2 or 3, it means that the target is severely occluded, and the result of the Kalman filtering algorithm is used as the tracking result.

As a preferred solution, the step 3 includes:

If the result of the KCF algorithm is used as the tracking result, the KCF filter is updated, and the Kalman filter is updated with the result of the KCF algorithm as the Kalman measurement value;

If the result of the Kalman filter algorithm is used as the tracking result, the Kalman filter is updated, and the KCF filter is stopped.

As a preferred solution, the update of the KCF filter includes:

α _t =θ((1-ρ)α _t-1 +ρα _t )+(1-θ)α _t-1

x _t =θ((1-ρ)x _t-1 +ρx _t )+(1-θ)x _t-1

Among them, ρ represents the update coefficient of the KCF filter parameters; αt _-1 and αt represent the KCF filter coefficients of the t-1th frame and the tth frame, and _xt-1 and _xt represent the t-1th frame and the t- _th frame. The parameters of the target model selected by the KCF filter of the t frame.

The present invention also provides a kernel correlation filtering tracking system for severely occluded scenes, including:

The initialization module is used to initialize the KCF algorithm of the KCF filter and the Kalman filter algorithm of the Kalman filter, and the target position of the first frame is used as the initial position of the KCF algorithm and the Kalman filter algorithm;

The judgment module is used to use the occlusion detection mechanism to judge whether the target is severely occluded in the current image; if not, the tracking result of the KCF algorithm is used as the position of the target object in the current image; if so, the tracking result of the Kalman filtering algorithm is used as the The position of the target object in the current image;

The update module is used to adaptively update the Kalman filter and the KCF filter according to the respective tracking results;

The execution module is used for reading the next video frame, and returns to step 2 until the tracking process ends.

As a preferred solution, the judging module uses a joint index occlusion detection mechanism to process the image, so as to judge whether the target in the current image is severely occluded.

As a preferred solution, the joint index occlusion detection mechanism is:

(1) Similarity judgment based on fusion features

The feature fusion index f is constructed using HSV and LBP features, including:

Based on the detection area of the KCF algorithm, the HSV and LBP features of the target and background areas are extracted respectively, and the extracted (HSV _target , HSV _background ) and (LBP _target , LBP _background ) are normalized using the following formulas. The processed feature vector is x=(x ₁ , x ₂ ,..., x _n-1 , x _n ) ^T , and the normalized feature vector is x ^* :

After obtaining the normalized eigenvectors, calculate the Euclidean distance D _HSV between the HSV _target and the HSV _background , and the Euclidean distance D _LBP between the LBP _target and the LBP _background :

表示目标的HSV特征向量的第i个分量值，

表示目标的LBP特征向量的第i个分量值，

表示背景区域的HSV特征向量的第i个分量值，

表示背景区域的LBP特征向量的第i个分量值；in,

represents the i-th component value of the HSV feature vector of the target,

represents the ith component value of the LBP feature vector of the target,

represents the i-th component value of the HSV feature vector of the background region,

represents the ith component value of the LBP feature vector of the background region;

Index D is used to reflect the degree of distinction between the target and the background under this feature;

The weights of HSV and LBP in the fusion feature are calculated as follows:

γ _HSV = 1 - γ _LBP

和

By extracting the HSV and LBP features of the target features of two adjacent frames, the feature similarity of the target in the kth frame is calculated.

and

Among them, HSV _k and HSV _k-1 represent the HSV features of the target in the kth frame and the k-1th frame, and LBP _k and LBP _k-1 represent the kth frame. The LBP feature of the target in the k-1th frame;

The feature fusion f is obtained by weighted summation of the weights γ _LBP and γ _HSV :

Use fusion feature f for similarity judgment: when the Nth frame is executed, first calculate the average similarity distance s using the obtained historical similarity distance data, and calculate the corresponding threshold th ₁ , the calculation formula is as follows:

th ₁ =δ ₁ *s (1<δ ₁ <2)

Among them, δ ₁ represents the threshold coefficient of the average similarity distance s;

Then calculate the Euclidean distance f _N of the target fusion features of the N-1th and N frames, and judge whether the Euclidean distance f _N is greater than th ₁ ; if not, no occlusion has occurred; if so, occlusion may occur;

The indicator of whether occlusion occurs is ε ₁ , and ε ₁ is expressed as:

(2) Judgment of tracking effect based on maximum response value

Use the maximum response value F _max to judge the tracking effect, including: when the execution reaches the Nth frame, first, count the historical maximum response value information of the previous N-1 frames, calculate the average maximum response value m and the threshold th ₂ , m and The calculation formula of th ₂ is as follows:

th ₂ =δ ₂ *m (0<δ ₂ <1)

表示第i帧的最大响应值，δ₂表示平均最大响应响应值m的阈值系数；in,

represents the maximum response value of the i-th frame, and δ ₂ represents the threshold coefficient of the average maximum response response value m;

是否大于th₂；若是，则当前并未发生遮挡；若否，则可能发生了遮挡；Then, determine the maximum response value of the Nth frame

Whether it is greater than th ₂ ; if so, no occlusion has occurred; if not, occlusion may have occurred;

The indicator of whether occlusion occurs is ε ₂ , and ε ₂ is expressed as:

(3) Evaluation mechanism based on average correlation peak ratio

Using the average correlation peak ratio APCE to evaluate the tracking reliability, including: on the basis of the response matrix, using the average correlation peak ratio to evaluate the tracking effect of the nuclear correlation filter tracking method; the APCE calculation formula is as follows:

Among them, F _max represents the maximum response value of the current frame, and F _min represents the minimum response value of the current frame;

和阈值th₃，用于评估当前帧，即第N帧的跟踪效果，计算公式如下：Calculate the average of the first N-1 frames

and the threshold th ₃ , which are used to evaluate the tracking effect of the current frame, that is, the Nth frame. The calculation formula is as follows:

的阈值系数；where δ ₃ represents the average correlation peak ratio

The threshold coefficient of ;

是否大于th₃；若是，则未发生遮挡；若否，则可能发生遮挡；judge

Whether it is greater than th ₃ ; if so, no occlusion occurs; if not, occlusion may occur;

The indicator of whether occlusion occurs is ε ₃ , and ε ₃ is expressed as:

The joint index ∈ is constructed as:

∈=ε ₁ +ε ₂ +ε ₃

When ∈=0 or 1, it means that the target is not occluded or slightly occluded, and the result of the KCF algorithm is used as the tracking result;

When ∈=2 or 3, it means that the target is severely occluded, and the result of the Kalman filtering algorithm is used as the tracking result.

As a preferred solution, the update process of the update module includes:

If the result of the KCF algorithm is used as the tracking result, the KCF filter is updated, and the Kalman filter is updated with the result of the KCF algorithm as the Kalman measurement value;

If the result of the Kalman filter algorithm is used as the tracking result, the Kalman filter is updated, and the KCF filter is stopped.

As a preferred solution, the update of the KCF filter includes:

α _t =θ((1-ρ)α _t-1 +ρα _t )+(1-θ)α _t-1

x _t =θ((1-ρ)x _t-1 +ρx _t )+(1-θ)x _t-1

Among them, ρ represents the update coefficient of the KCF filter parameters; αt _-1 and αt represent the KCF filter coefficients of the t-1th frame and the tth frame, and _xt-1 and _xt represent the t-1th frame and the t- _th frame. The parameters of the target model selected by the KCF filter of the t frame.

Compared with the prior art, the present invention has the following beneficial effects:

(1) Aiming at the lack of the occlusion detection mechanism of the kernel correlation filtering algorithm, a joint index occlusion detection mechanism is proposed, which makes it possible to distinguish between normal scenes and occlusion scenes;

(2) The adaptive update strategy of the model in the case of occlusion is designed, so that the feature model is updated online without being seriously polluted by the occluder;

(3) The Kalman target tracking processing mechanism under occlusion is added, so that the target can still be tracked even if the target is severely occluded.

Description of drawings

FIG. 1 is a flowchart of a kernel correlation filtering target tracking method for severely occluded scenes according to an embodiment of the present invention;

Fig. 2 is the relationship between similarity distance fN and threshold th ₁ in consecutive frames according to an embodiment of the present invention;

Fig. 3 is the relationship between the maximum response value Fmax and the threshold th ₂ in consecutive frames according to an embodiment of the present invention;

4 is the relationship between the average correlation peak ratio ACPE and the threshold th ₃ in consecutive frames according to an embodiment of the present invention;

Fig. 5 is the change of each parameter of the joint occlusion detection mechanism before and after occlusion in consecutive frames according to an embodiment of the present invention;

FIG. 6 is a structural diagram of a kernel correlation filtering target tracking system for severely occluded scenes according to an embodiment of the present invention.

Detailed ways

In order to describe the embodiments of the present invention more clearly, the following will describe specific embodiments of the present invention with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative efforts, and obtain other implementations.

The kernel correlation filtering target tracking method for severely occluded scenes in the embodiment of the present invention mainly optimizes the existing framework in the following three aspects:

1. In view of the lack of detection mechanism for occlusion, a joint index occlusion detection mechanism is proposed;

2. Design the adaptive update strategy of the model in the case of occlusion;

3. Added the Kalman target tracking processing mechanism in the case of occlusion.

Specifically, as shown in FIG. 1 , the kernel correlation filtering target tracking method for severely occluded scenes according to the embodiment of the present invention includes the following steps:

Step 1: Initialize the KCF target tracking algorithm and the Kalman algorithm, and use the target position of the first frame as the initial position of the KCF and Kalman algorithms.

The Kalman algorithm uses the position of the target frame in the previous frame and the moving speed of the target to predict and track the position of the target in the next frame. In the target tracking task of real-time video, the time interval between every two frames is relatively small. It can be considered that the target moves slowly in adjacent images, and the target motion can be approximated as a uniform linear motion. The position and moving speed of the target in the next frame can be determined by the dynamic Learn the formula:

x _k =x _k-1 +v _k-1 Δt

v _k = v _k-1

Among them, Δt is the time interval of two frames, x _k-1 represents the position of the target in the k-1th frame, and _vk-1 represents the speed of the target in the k-1th frame. Because the system is a linear dynamic model, taking the two-dimensional case as an example, the system transition state equation is as follows:

X(k)=AX(k-1)+G(k)

表示速率变动，在匀速直线运动中可视为运动过程中的高斯白噪声，

和

均服从N(0，1)分布。Among them, A represents the motion transfer matrix of the object, X(k)=(x _k , y _k , v _x , _vy ) ^T , x _k represents the position of the target on the x-axis in the k-1th frame, and y _k represents the target In the position of the y-axis in the k-1th frame, v _x represents the movement speed of the target on the x-axis, v _y represents the movement speed of the target on the y-axis, and Δt represents the period of two detections of the video;

Represents the rate change, which can be regarded as Gaussian white noise in the process of motion in uniform linear motion,

and

All obey the N(0,1) distribution.

The system measurement value calculation formula is:

Y(k)=HX(k)+C(k)

In the formula, H represents the observation matrix of the measurement system, and C(k) represents the noise of the measurement process.

The target position and its corresponding error covariance matrix P can be predicted using the Kalman filter:

where Q is the covariance of motion noise. Combining the predicted value and the measured value, the optimal estimate of the current state k can be obtained as:

And update the error covariance of X(k|k) in k states:

P(k|k)=P(k|k-1)-K _k HP(k|k-1)

Among them, K _k is the Kalman gain, and its calculation formula is as follows:

K _k =P(k|k-1)H ^T /(HP(k|k-1)H ^T +R _k )

Among them, R _k refers to the covariance of measurement noise, and when the value of measurement covariance is lower, it means that there is a greater weight on the current measurement value, that is, the reliability of the measurement value is considered to be higher.

The overall process of the Kalman filtering algorithm is briefly described as follows:

(1) Determine the state transition matrix A and measurement matrix H of the system.

(2) Initialize the initial value of the covariance matrix P(0|0) and the initial value of the state quantity X(0).

和协方差矩阵P(k|k-1)，并更新卡尔曼增益K_k。(3) According to the state transition matrix A, the state recursion equation and the covariance matrix, determine the state quantity at the next moment

and the covariance matrix P(k|k-1), and update the Kalman gain K _k .

修正状态量，得到该更新周期的状态估计修正值

即X(k)。(4) According to the measurement vector Y(k), the Kalman gain K _k and the state quantity

Correct the state quantity to obtain the state estimate correction value of the update period

That is X(k).

(5) According to the state estimation correction value X(k) and the measurement value calculation formula, update the covariance matrix to obtain the covariance matrix correction value P(k|k).

(6) Go back to the above step (3) and repeat the above steps until the tracking process ends.

Step 2: Use the joint index occlusion detection mechanism to process the image to determine whether the target in the current image is severely occluded. If there is no target occlusion, the tracking result of the KCF algorithm is used as the position of the target object in the current frame; if the target is severely occluded, the prediction result of the Kalman algorithm is used as the position of the target in the current frame.

The joint index occlusion detection mechanism according to the embodiment of the present invention is described in detail as follows:

1) Similarity judgment based on fusion features

The feature fusion index f is constructed using HSV and LBP features. The specific method is as follows: Based on the detection region of the KCF algorithm, the HSV and LBP features of the target and background regions are extracted respectively, and the extracted (HSV _target , HSV _background ) and ( LBP _target , LBP _background ) are normalized using the following formula, and the feature vector to be processed is x=(x ₁ , x ₂ ,..., x _n-1 , x _n ) ^T , after normalization A vector of x ^* :

After obtaining the normalized eigenvectors, use the following formulas to calculate the Euclidean distance d _HSV of HSV _target and HSV _background respectively, and the Euclidean distance d _LBP of LBP _target and LBP _background :

表示目标的LBP特征向量的第i个分量值，同样地有

表示背景区域的LBP特征向量的第i个分量值，同样地有

In the formula,

Represents the i-th component value of the LBP feature vector of the target, and also has

Represents the i-th component value of the LBP feature vector of the background area, and also has

The index D is used to reflect the degree of distinction between the target and the background under this feature. The higher the discrimination between the target and the background, the more accurate the target model constructed using this feature, so the constructed fusion feature should contain more information about this feature. The weights of HSV and LBP in the fusion feature are calculated as follows:

γ _HSV = 1 - γ _LBP

和

By extracting the HSV and LBP features of the target features of two adjacent frames, the feature similarity of the target in the kth frame can be calculated by the following formula

and

where HSV _k and HSV _k-1 represent the HSV features of the target in the kth frame and the k-1th frame, and LBP _k and LBP _k-1 represent the LBP features of the target in the kth frame and the k-1th frame. Then through the weighted summation of the weights γ _LBP and γ _HSV obtained above, the two features can be fused to obtain the feature fusion f, and the formula of f is as follows:

Next, the obtained fusion feature f is used for similarity judgment: when the algorithm executes to the Nth frame, the average similarity distance s is first calculated using the obtained historical similarity distance data, and the corresponding threshold th ₁ is calculated. The calculation formula is as follows:

th ₁ =δ ₁ *s (1<δ ₁ <2)

Among them, δ ₁ represents the threshold coefficient of the average similarity distance s, and generally, δ ₁ =1.5 is selected. Then calculate the Euclidean distance f _N of the target fusion features of frames N-1 and N. When f _N is less than th ₁ , it means that no occlusion has occurred in the near time period. On the contrary, it means that occlusion may have occurred. Under this feature, the indicator of whether occlusion occurs is ε ₁ . When the target may be occluded, ε ₁ takes the value of 1; if there is no occlusion, ε ₁ takes the value of 0, and ε 1 can be expressed as:

In consecutive frames, the relationship between the similarity distance fN and the threshold th ₁ is shown in Figure 2.

2) Judgment of tracking effect based on maximum response value

Use the maximum response value F _max to judge the tracking effect: when running to the Nth frame, first count the historical maximum response value information of the previous N-1 frames, calculate the average maximum response value m and the threshold th ₂ , and calculate m and th ₂ The formula is as follows:

th ₂ =δ ₂ *m (0<δ ₂ <1)

表示第i帧的最大响应值，δ₂表示平均最大响应响应值m的阈值系数，一般选取δ₂＝0.6。然后比较第N帧的最大响应值

和th₂的大小关系，如果最大响应值

大于th₂，则说明当前并未发生遮挡。如果最大响应值

小于th2，则说明近段时间内可能发生了遮挡。记在此特征下，是否发生遮挡的指标为ε₂，当目标可能被遮挡时，ε₂取值为1；若未发生遮挡，ε₂取值为0，ε₂可表示为：in,

represents the maximum response value of the ith frame, and δ ₂ represents the threshold coefficient of the average maximum response response value m, generally δ ₂ =0.6. Then compare the maximum response value of the Nth frame

and the size of th ₂ , if the maximum response value

If it is greater than th ₂ , it means that there is no occlusion currently. If the maximum response value

If it is less than th2, it means that occlusion may have occurred in the recent period. Under this feature, the indicator of whether occlusion occurs is ε ₂ . When the target may be occluded, ε ₂ takes the value of 1; if there is no occlusion, ε ₂ takes the value of 0, and ε ₂ can be expressed as:

In consecutive frames, the relationship between the maximum response value F _max and the threshold value th ₂ is shown in FIG. 3 .

3) Evaluation mechanism based on average correlation peak ratio

Use the average correlation peak ratio APCE to evaluate the tracking reliability: On the basis of the response matrix, Wang et al. proposed the average correlation peak ratio (APCE) to evaluate the tracking effect of the nuclear correlation filter tracking algorithm. The APCE calculation formula is as follows, where F _max represents the maximum response value of the current frame, and F _min represents the minimum response value of the current frame.

和阈值th₃，用于评估当前帧(第N帧)的跟踪效果，计算公式如下：Calculate the average of the first N-1 frames

and the threshold th ₃ , which is used to evaluate the tracking effect of the current frame (the Nth frame). The calculation formula is as follows:

的阈值系数，一般选取δ₃＝0.5。记在此特征下，是否发生遮挡的指标为ε₃，当目标可能被遮挡时，当ε₃取值为1；若未发生遮挡，ε₃取值为0，ε₃可表示为：where δ ₃ represents the average correlation peak ratio

The threshold coefficient of , generally choose δ ₃ =0.5. Under this feature, the indicator of whether occlusion occurs is ε ₃ . When the target may be occluded, ε ₃ takes the value of 1; if there is no occlusion, ε ₃ takes the value of 0, and ε ₃ can be expressed as:

In consecutive frames, the relationship between the average correlation peak ratio ACPE and the threshold th ₃ is shown in Fig. 4 .

Finally, the above three indicators are combined to construct a joint indicator ∈ to realize the detection of target occlusion. The formula is as follows:

∈=ε ₁ +ε ₂ +ε ₃

where ε ₁ represents the tracking situation based on the fusion feature f response, where ε ₂ represents the tracking situation based on the maximum response value F _max response, and ε ₃ represents the tracking situation based on the average correlation peak ratio APCE response. When ∈=0 or 1, it means that the target is not occluded or slightly occluded, which will not cause too much pollution to the KCF filter. The filter can be updated normally, and the result of the KCF algorithm is used as the tracking result. When ∈=2 or 3, it means that the target is severely occluded, other operations need to be performed, and the result of the Kalman algorithm is used as the tracking result. In consecutive video frames, before and after target occlusion, the above f, _Fmax and APCE changes are shown in Figure 5.

一致)，对Kalman滤波器进行更新。Step 3: According to the results of the occlusion detection mechanism, use the results of Step 2 to adaptively update the Kalman filter and the KCF filter. If there is no target occlusion, the result of the KCF algorithm is used as the Kalman measurement value (that is, the measurement vector Y(k) is set as the result of KCF), the Kalman filter is updated, and the regular update of the KCF filter is performed at the same time; if there is When the target is blocked, the update of the KCF filter is stopped, and the predicted value of the Kalman filter is used as the Kalman measurement value (that is, the measurement vector Y(k) is set to be the same as the prior state.

Consistent), update the Kalman filter.

For the KCF filter, the adaptive update strategy is based on the original strategy, adding a stopping mechanism. θ is the index to determine whether to update, ∈ is the result of the occlusion detection mechanism, and the determination process is as follows:

When there is no occlusion or slight occlusion (θ=1), the KCF filter update strategy at this time is the same as the original strategy. When severe occlusion occurs (θ=0), the KCF filter should stop updating to avoid contamination. Corresponding to the KCF filter, the update formula of its specific parameters is as follows:

α _t =θ((1-ρ)α _t-1 +ρα _t )+(1-θ)α _t-1

x _t =θ((1-ρ)x _t-1 +ρx _t )+(1-θ)x _t-1

Among them, ρ represents the update coefficient of KCF filter parameters, which is generally 0.8. αt _-1 and αt represent the KCF filter coefficients of the t-1th and _tth frames, and _xt-1 and _xt represent the selected target model in the KCF filter of the t-1th and tth frames. For parameters, reference may be made to the prior art, and details are not described here.

After the adaptive update is complete, the method will read the next video frame and go back to step 2.

The method ends until all video frames have been read.

Based on the above kernel correlation filtering and tracking method according to the embodiment of the present invention, as shown in FIG. 6 , an embodiment of the present invention also provides a kernel correlation filtering and tracking system for severely occluded scenes, including an initialization module, a judgment module, an update module and an execution module.

Specifically, the initialization module is used to initialize the KCF algorithm of the KCF filter and the Kalman filter algorithm of the Kalman filter, with the target position of the first frame as the initial position of the KCF algorithm and the Kalman filter algorithm;

The judgment module is used to use the occlusion detection mechanism to judge whether the target is severely occluded in the current image; if not, the tracking result of the KCF algorithm is used as the position of the target object in the current image; if so, the tracking result of the Kalman filtering algorithm is used as the target. the position of the object in the current image;

Specifically, the judging module uses a joint index occlusion detection mechanism to process the image to judge whether the target in the current image is severely occluded.

The joint index occlusion detection mechanism is:

(1) Similarity judgment based on fusion features

The feature fusion index f is constructed using HSV and LBP features, including:

Based on the detection area of the KCF algorithm, the HSV and LBP features of the target and background areas are extracted respectively, and the extracted (HSV _target , HSV _background ) and (LBP _target , LBP _background ) are normalized using the following formulas. The processed feature vector is x=(x ₁ , x ₂ ,..., x _n-1 , x _n ) ^T , and the normalized feature vector is x ^* :

After obtaining the normalized eigenvectors, calculate the Euclidean distance D _HSV between the HSV _target and the HSV _background , and the Euclidean distance D _LBP between the LBP _target and the LBP _background :

表示目标的HSV特征向量的第i个分量值，

表示目标的LBP特征向量的第i个分量值，

表示背景区域的HSV特征向量的第i个分量值，

表示背景区域的LBP特征向量的第i个分量值；in,

represents the i-th component value of the HSV feature vector of the target,

represents the ith component value of the LBP feature vector of the target,

represents the i-th component value of the HSV feature vector of the background region,

represents the ith component value of the LBP feature vector of the background region;

Index D is used to reflect the degree of distinction between the target and the background under this feature;

The weights of HSV and LBP in the fusion feature are calculated as follows:

γ _HSV = 1 - γ _LBP

和

By extracting the HSV and LBP features of the target features of two adjacent frames, the feature similarity of the target in the kth frame is calculated.

and

Among them, HSV _k and HSV _k-1 represent the HSV features of the target in the kth frame and the k-1th frame, and LBP _k and LBP _k-1 represent the kth frame. The LBP feature of the target in the k-1th frame;

The feature fusion f is obtained by weighted summation of the weights γ _LBP and γ _HSV :

Use fusion feature f for similarity judgment: when the Nth frame is executed, first calculate the average similarity distance s using the obtained historical similarity distance data, and calculate the corresponding threshold th ₁ , the calculation formula is as follows:

th ₁ =δ ₁ *s (1<δ ₁ <2)

Among them, δ ₁ represents the threshold coefficient of the average similarity distance s;

Then calculate the Euclidean distance f _N of the target fusion features of the N-1th and N frames, and judge whether the Euclidean distance f _N is greater than th ₁ ; if not, no occlusion has occurred; if so, occlusion may occur;

The indicator of whether occlusion occurs is ε ₁ , and ε ₁ is expressed as:

(2) Judgment of tracking effect based on maximum response value

Use the maximum response value F _max to judge the tracking effect, including: when the execution reaches the Nth frame, first, count the historical maximum response value information of the previous N-1 frames, calculate the average maximum response value m and the threshold th ₂ , m and The calculation formula of th ₂ is as follows:

th ₂ =δ ₂ *m (0<δ ₂ <1)

表示第i帧的最大响应值，δ₂表示平均最大响应响应值m的阈值系数；in,

represents the maximum response value of the i-th frame, and δ ₂ represents the threshold coefficient of the average maximum response response value m;

是否大于th₂；若是，则当前并未发生遮挡；若否，则可能发生了遮挡；Then, determine the maximum response value of the Nth frame

Whether it is greater than th ₂ ; if so, no occlusion has occurred; if not, occlusion may have occurred;

The indicator of whether occlusion occurs is ε ₂ , and ε ₂ is expressed as:

(3) Evaluation mechanism based on average correlation peak ratio

Using the average correlation peak ratio APCE to evaluate the tracking reliability, including: on the basis of the response matrix, using the average correlation peak ratio to evaluate the tracking effect of the nuclear correlation filter tracking method; the APCE calculation formula is as follows:

Among them, F _max represents the maximum response value of the current frame, and F _min represents the minimum response value of the current frame;

和阈值th₃，用于评估当前帧，即第N帧的跟踪效果，计算公式如下：Calculate the average of the first N-1 frames

and the threshold th ₃ , which are used to evaluate the tracking effect of the current frame, that is, the Nth frame. The calculation formula is as follows:

的阈值系数；where δ3 represents the average correlation peak ratio

The threshold coefficient of ;

是否大于th3；若是，则未发生遮挡；若否，则可能发生遮挡；judge

Is it greater than th3; if so, no occlusion has occurred; if not, occlusion may occur;

The indicator of whether occlusion occurs is ε ₃ , and ε ₃ is expressed as:

The joint index ∈ is constructed as:

∈=ε ₁ +ε ₂ +ε ₃

When ∈=0 or 1, it means that the target is not occluded or slightly occluded, and the result of the KCF algorithm is used as the tracking result;

When ∈=2 or 3, it means that the target is severely occluded, and the result of the Kalman filtering algorithm is used as the tracking result.

The updating module in the embodiment of the present invention is used to adaptively update the Kalman filter and the KCF filter according to the respective tracking results. Specifically, the update process of the update module includes:

If the result of the KCF algorithm is used as the tracking result, the KCF filter is updated, and at the same time, the result of the KCF algorithm is used as the Kalman measurement value to update the Kalman filter; the Kalman filter may refer to the prior art.

If the result of the Kalman filter algorithm is used as the tracking result, the Kalman filter is updated, and the KCF filter is stopped.

Among them, the update of the KCF filter, including:

α _t =θ((1-ρ)α _t-1 +ρα _t )+(1-θ)α _t-1

x _t =θ((1-ρ)x _t-1 +ρx _t )+(1-θ)x _t-1

Among them, ρ represents the update coefficient of the KCF filter parameters; αt _-1 and αt represent the KCF filter coefficients of the t-1th frame and the tth frame, and _xt-1 and _xt represent the t-1th frame and the t- _th frame. The parameters of the target model selected by the KCF filter of the t frame.

The execution module in the embodiment of the present invention is configured to read the next video frame, and return to execute the above process until the tracking process ends.

The above is only a detailed description of the preferred embodiments and principles of the present invention. For those of ordinary skill in the art, according to the ideas provided by the present invention, there will be changes in the specific implementation, and these changes should also be It is regarded as the protection scope of the present invention.

Claims (10)

1. a nuclear correlation filter tracking method for severely occluded scenes, is characterized in that, comprises the following steps: Step 1, initialize the KCF algorithm of the KCF filter and the Kalman filter algorithm of the Kalman filter, take the target position of the first frame as the initial position of the KCF algorithm and the Kalman filter algorithm; Step 2. Use the occlusion detection mechanism to determine whether the target is severely occluded in the current image; if not, use the tracking result of the KCF algorithm as the position of the target object in the current image; if so, use the tracking result of the Kalman filtering algorithm as the target object position in the current image; Step 3, according to the respective tracking results, adaptively update the Kalman filter and the KCF filter; Step 4: Read the next video frame, and return to step 2 until the tracking process ends. 2. A kind of kernel correlation filtering tracking method oriented to severe occlusion scene according to claim 1, it is characterized in that, in described step 2, adopt joint index occlusion detection mechanism to process image, to judge whether the target in the current image is serious or not. occlude. 3. A kind of kernel correlation filter tracking method for severe occlusion scene according to claim 1, is characterized in that, described joint index occlusion detection mechanism is: (1) Similarity judgment based on fusion features The feature fusion index f is constructed using HSV and LBP features, including: Based on the detection area of the KCF algorithm, the HSV and LBP features of the target and background areas are extracted respectively, and the extracted (HSV _target , HSV _background ) and (LBP _target , LBP _background ) are normalized using the following formulas. The processed feature vector is x=(x ₁ , x ₂ ,..., x _n-1 , x _n ) ^T , and the normalized feature vector is x ^* :

After obtaining the normalized eigenvectors, calculate the Euclidean distance D _HSV between the HSV _target and the HSV _background , and the Euclidean distance D _LBP between the LBP _target and the LBP _background :

in,

represents the i-th component value of the HSV feature vector of the target,

represents the ith component value of the LBP feature vector of the target,

represents the i-th component value of the HSV feature vector of the background region,

represents the ith component value of the LBP feature vector of the background region; Index D is used to reflect the degree of distinction between the target and the background under this feature; The weights of HSV and LBP in the fusion feature are calculated as follows:

γ _HSV = 1 - γ _LBP By extracting the HSV and LBP features of the target features of two adjacent frames, the feature similarity of the target in the kth frame is calculated.

and

Among them, HSV _k and HSV _k-1 represent the HSV features of the target in the kth frame and the k-1th frame, and LBP _k and LBP _k-1 represent the kth frame. The LBP feature of the target in the k-1th frame; The feature fusion f is obtained by weighted summation of the weights γ _LBP and γ _HSV :

Use fusion feature f for similarity judgment: when the Nth frame is executed, first calculate the average similarity distance s using the obtained historical similarity distance data, and calculate the corresponding threshold th ₁ , the calculation formula is as follows:

th ₁ =δ ₁ *s (1<δ ₁ <2) Among them, δ ₁ represents the threshold coefficient of the average similarity distance s; Then calculate the Euclidean distance f _N of the target fusion features of the N-1th and N frames, and judge whether the Euclidean distance f _N is greater than th ₁ ; if not, no occlusion has occurred; if so, occlusion may occur; The indicator of whether occlusion occurs is ε ₁ , and ε ₁ is expressed as:

(2) Judgment of tracking effect based on maximum response value Use the maximum response value F _max to judge the tracking effect, including: when the execution reaches the Nth frame, first, count the historical maximum response value information of the previous N-1 frames, calculate the average maximum response value m and the threshold th ₂ , m and The calculation formula of th ₂ is as follows:

th ₂ =δ ₂ *m (0<δ ₂ <1) in,

represents the maximum response value of the i-th frame, and δ ₂ represents the threshold coefficient of the average maximum response response value m; Then, determine the maximum response value of the Nth frame

Whether it is greater than th ₂ ; if so, no occlusion has occurred; if not, occlusion may have occurred; The indicator of whether occlusion occurs is ε ₂ , and ε ₂ is expressed as:

(3) Evaluation mechanism based on average correlation peak ratio Using the average correlation peak ratio APCE to evaluate the tracking reliability, including: on the basis of the response matrix, using the average correlation peak ratio to evaluate the tracking effect of the nuclear correlation filter tracking method; the APCE calculation formula is as follows:

Among them, F _max represents the maximum response value of the current frame, and F _min represents the minimum response value of the current frame; Calculate the average of the first N-1 frames

and the threshold th ₃ , which are used to evaluate the tracking effect of the current frame, that is, the Nth frame. The calculation formula is as follows:

where δ ₃ represents the average correlation peak ratio

The threshold coefficient of ; judge

Whether it is greater than th ₃ ; if so, no occlusion occurs; if not, occlusion may occur; The indicator of whether occlusion occurs is ε ₃ , and ε ₃ is expressed as:

The joint index ∈ is constructed as: ∈=ε ₁ +ε ₂ +ε ₃ When ∈=0 or 1, it means that the target is not occluded or slightly occluded, and the result of the KCF algorithm is used as the tracking result; When ∈=2 or 3, it means that the target is severely occluded, and the result of the Kalman filtering algorithm is used as the tracking result. 4. A kind of kernel correlation filter tracking method for severely occluded scene according to claim 3, is characterized in that, described step 3, comprises: If the result of the KCF algorithm is used as the tracking result, the KCF filter is updated, and the Kalman filter is updated with the result of the KCF algorithm as the Kalman measurement value; If the result of the Kalman filter algorithm is used as the tracking result, the Kalman filter is updated, and the KCF filter is stopped. 5. a kind of kernel correlation filter tracking method for serious occlusion scene according to claim 4, is characterized in that, the update of described KCF filter, comprises: α _t =θ((1-ρ)α _t-1 +ρα _t )+(1-θ)α _t-1 x _t =θ((1-ρ)x _t-1 +ρx _t )+(1-θ)x _t-1

Among them, ρ represents the update coefficient of the KCF filter parameters; αt _-1 and αt represent the KCF filter coefficients of the t-1th frame and the tth frame, and _xt-1 and _xt represent the t-1th frame and the t- _th frame. The parameters of the target model selected by the KCF filter of the t frame. 6. A kernel correlation filter tracking system for severely occluded scenes, characterized in that it comprises: The initialization module is used to initialize the KCF algorithm of the KCF filter and the Kalman filter algorithm of the Kalman filter, and the target position of the first frame is used as the initial position of the KCF algorithm and the Kalman filter algorithm; The judgment module is used to use the occlusion detection mechanism to judge whether the target is severely occluded in the current image; if not, the tracking result of the KCF algorithm is used as the position of the target object in the current image; if so, the tracking result of the Kalman filtering algorithm is used as the The position of the target object in the current image; The update module is used to adaptively update the Kalman filter and the KCF filter according to the respective tracking results; The execution module is used for reading the next video frame, and returns to step 2 until the tracking process ends. 7 . The kernel correlation filtering tracking system for severely occluded scenes according to claim 6 , wherein the judging module uses a joint index occlusion detection mechanism to process the image to judge whether the target in the current image is severely occluded. 8 . 8. A kind of kernel correlation filter tracking method for severe occlusion scene according to claim 7, it is characterized in that, described joint index occlusion detection mechanism is: (1) Similarity judgment based on fusion features The feature fusion index f is constructed using HSV and LBP features, including: Based on the detection area of the KCF algorithm, the HSV and LBP features of the target and background areas are extracted respectively, and the extracted (HSV _target , HSV _background ) and (LBP _target , LBP _background ) are normalized using the following formulas. The processed feature vector is x=(x ₁ , x ₂ ,..., x _n-1 , x _n ) ^T , and the normalized feature vector is x ^* :

After obtaining the normalized eigenvectors, calculate the Euclidean distance D _HSV between the HSV _target and the HSV _background , and the Euclidean distance D _LBP between the LBP _target and the LBP _background :

in,

represents the i-th component value of the HSV feature vector of the target,

represents the ith component value of the LBP feature vector of the target,

represents the i-th component value of the HSV feature vector of the background region,

represents the ith component value of the LBP feature vector of the background region; Index D is used to reflect the degree of distinction between the target and the background under this feature; The weights of HSV and LBP in the fusion feature are calculated as follows:

γ _HSV = 1 - γ _LBP By extracting the HSV and LBP features of the target features of two adjacent frames, the feature similarity of the target in the kth frame is calculated.

and

Among them, HSV _k and HSV _k-1 represent the HSV features of the target in the kth frame and the k-1th frame, and LBP _k and LBP _k-1 represent the kth frame. The LBP feature of the target in the k-1th frame; The feature fusion f is obtained by weighted summation of the weights γ _LBP and γ _HSV :

Use fusion feature f for similarity judgment: when the Nth frame is executed, first calculate the average similarity distance s using the obtained historical similarity distance data, and calculate the corresponding threshold th ₁ , the calculation formula is as follows:

th ₁ =δ ₁ *s (1<δ ₁ <2) Among them, δ ₁ represents the threshold coefficient of the average similarity distance s; Then calculate the Euclidean distance f _N of the target fusion features of the N-1th and N frames, and judge whether the Euclidean distance f _N is greater than th ₁ ; if not, no occlusion has occurred; if so, occlusion may occur; The indicator of whether occlusion occurs is ε ₁ , and ε ₁ is expressed as:

(2) Judgment of tracking effect based on maximum response value Use the maximum response value F _max to judge the tracking effect, including: when the execution reaches the Nth frame, first, count the historical maximum response value information of the previous N-1 frames, calculate the average maximum response value m and the threshold th ₂ , m and The calculation formula of th ₂ is as follows:

th ₂ =δ ₂ *m (0<δ ₂ <1) in,

represents the maximum response value of the i-th frame, and δ ₂ represents the threshold coefficient of the average maximum response response value m; Then, determine the maximum response value of the Nth frame

Whether it is greater than th ₂ ; if so, no occlusion has occurred; if not, occlusion may have occurred; The indicator of whether occlusion occurs is ε ₂ , and ε ₂ is expressed as:

(3) Evaluation mechanism based on average correlation peak ratio Using the average correlation peak ratio APCE to evaluate the tracking reliability, including: on the basis of the response matrix, using the average correlation peak ratio to evaluate the tracking effect of the nuclear correlation filter tracking method; the APCE calculation formula is as follows:

Among them, F _max represents the maximum response value of the current frame, and F _min represents the minimum response value of the current frame; Calculate the average of the first N-1 frames

and the threshold th ₃ , which are used to evaluate the tracking effect of the current frame, that is, the Nth frame. The calculation formula is as follows:

where δ ₃ represents the average correlation peak ratio

The threshold coefficient of ; judge

Whether it is greater than th ₃ ; if so, no occlusion occurs; if not, occlusion may occur; The indicator of whether occlusion occurs is ε ₃ , and ε ₃ is expressed as:

The joint index ∈ is constructed as: ∈=ε ₁ +ε ₂ +ε ₃ When ∈=0 or 1, it means that the target is not occluded or slightly occluded, and the result of the KCF algorithm is used as the tracking result; When ∈=2 or 3, it means that the target is severely occluded, and the result of the Kalman filtering algorithm is used as the tracking result. 9. A kind of kernel correlation filtering tracking system oriented to severe occlusion scene according to claim 8, is characterized in that, the updating process of described updating module, comprises: If the result of the KCF algorithm is used as the tracking result, the KCF filter is updated, and the Kalman filter is updated with the result of the KCF algorithm as the Kalman measurement value; If the result of the Kalman filter algorithm is used as the tracking result, the Kalman filter is updated, and the KCF filter is stopped. 10. A kind of kernel correlation filtering tracking system oriented to severe occlusion scene according to claim 9, is characterized in that, the updating of described KCF filter, comprises: α _t =θ((1-ρ)α _t-1 +ρα _t )+(1-θ)α _t-1 x _t =θ((1-ρ)x _t-1 +ρx _t )+(1-θ)x _t-1

Among them, ρ represents the update coefficient of the KCF filter parameters; αt _-1 and αt represent the KCF filter coefficients of the t-1th frame and the tth frame, and _xt-1 and _xt represent the t-1th frame and the t- _th frame. The parameters of the target model selected by the KCF filter of the t frame.

Images