CN116049759A - Multi-sensor based monitoring target fusion method, device, and electronic equipment - Google Patents

Abstract

The invention discloses a monitoring target fusion method based on multiple sensors, a device and electronic equipment thereof, wherein the fusion method comprises the following steps: and (3) sampling the obtained state estimation values of the target states of the monitoring targets and the obtained contour estimation values of the target contours at the previous moment to obtain weighted particles of the predicted measurement values of the monitoring targets at the current moment, processing the measurement data of each sensor to obtain the current state estimation values and the current contour estimation values of all the monitoring targets corresponding to the sensors at the current moment, screening a target sensor set corresponding to each monitoring target, and fusing the current state estimation values and the current contour estimation values corresponding to each target sensor in the target sensor set to obtain the target state values and the target contour values of the monitoring targets at the current moment. The invention solves the technical problems of poor timeliness and low accuracy of calculating the state value and the contour value of the monitoring target in the related technology.

Description

Monitoring target fusion method based on multi-sensor and its device and electronic equipment

Technical Field

The present invention relates to the technical field of multi-sensor information fusion, and in particular to a multi-sensor based monitoring target fusion method and device, and electronic equipment.

Background Art

With the construction of "smart cities", more and more types of sensors are being used in various occasions. In order to avoid the disadvantage of visual sensors that easily infringe privacy, sensors such as millimeter wave radars can be used to quickly and accurately locate and track while avoiding privacy violations. However, due to the high resolution of the sensor, one monitoring target (for example, cars, pedestrians, etc.) will generate multiple measurements. Since identifying the shape of the monitoring target is conducive to further judgment, it is considered to be treated as an extended target.

In the related art, the positioning and tracking of extended targets often requires preprocessing operations such as clustering before tracking. This method has too much computational burden for embedded platforms with limited computing power and is difficult to meet real-time requirements.

To address the above-mentioned problems, no effective solution has been proposed yet.

Summary of the invention

The embodiment of the present invention provides a monitoring target fusion method based on multiple sensors, a device thereof, and an electronic device, so as to at least solve the technical problems of poor timeliness and low accuracy in calculating the state value and profile value of the monitoring target in the related art.

According to one aspect of an embodiment of the present invention, a monitoring target fusion method based on multiple sensors is provided, comprising: obtaining a state estimation value of a target state and a contour estimation value of a target contour of each monitoring target at a previous moment; performing particle sampling on the state estimation value and the contour estimation value of the monitoring target to obtain weighted particles of a predicted measurement value of the monitoring target at a current moment; processing measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitoring target at the current moment to obtain a current state estimation value and a current contour estimation value of all the monitoring targets corresponding to the sensor at the current moment; screening a set of target sensors corresponding to each monitoring target based on the current state estimation value and the current contour estimation value; fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain a target state value and a target contour value of the monitoring target at the current moment.

Optionally, after obtaining the state estimation value of the target state of each monitoring target at the previous moment and the contour estimation value of the target contour, it also includes: determining the state prediction value of the monitoring target at the current moment based on the state estimation value of the monitoring target at the previous moment; determining the contour prediction value of the monitoring target at the current moment based on the contour estimation value of the monitoring target at the previous moment; and obtaining the predicted measurement value of the monitoring target at the current moment based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitoring target at the current moment.

Optionally, the step of performing particle sampling on the state estimation value and the profile estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment includes: performing particle sampling on the state estimation value and the profile estimation value to generate multiple particles; calculating the initial particle weight of each particle based on preset parameters; normalizing each initial particle weight based on all the initial particle weights to obtain a target particle weight for each particle, wherein the particles carrying the target particle weights are characterized as weighted particles; updating the weighted particles to obtain weighted particles associated with the predicted measurement value of the monitoring target at the current moment.

Optionally, based on the weighted particles of the predicted measurement value of the monitoring target at the current moment, before processing the measurement data of each sensor, it also includes: obtaining the measurement data of each of the sensors, wherein the measurement data at least includes: a measurement vector set and a total number of measurements; based on the measurement data, constructing a target-associated variable for the monitoring target, wherein the target-associated variable is used to represent a measurement index indicating a first preset number of bits of measurement generated by a preset monitoring target at the current moment; based on the measurement data, constructing a measurement-associated variable for vector measurement, wherein the measurement-associated variable is used to represent a second preset number of bits of measurement generated by the monitoring target mapped by the measurement index at the current moment.

Optionally, based on the weighted particles of the predicted measurement values of the monitored target at the current moment, the measurement data of each sensor is processed to obtain the current state estimation values and current profile estimation values of all the monitored targets corresponding to the sensor at the current moment, including: based on the weighted particles of the predicted measurement values of the monitored target at the current moment and the measurement vector set, the target-associated variables are measured and evaluated to obtain a likelihood probability; under the profile estimation value when the likelihood probability belongs to a preset probability threshold range, an association factor graph is constructed based on the target-associated variables and the measurement-associated variables; based on the association factor graph, an iterative calculation is performed to obtain a first parameter value and a second parameter value; based on the first parameter value and the second parameter value, the measurement data of the monitored target of the sensor is updated to obtain an updated result; based on the updated result, the current profile estimation value of the monitored target corresponding to the sensor at the current moment is obtained, and based on the updated result, the weighted particles are weighted calculated to obtain the current state estimation value of the monitored target corresponding to the sensor.

Optionally, based on the current state estimation value and the current profile estimation value, the step of screening the target sensor set corresponding to each of the monitored targets includes: characterizing the current state estimation value as the mean of a preset Gaussian distribution, and characterizing the current profile estimation value as the covariance matrix of the preset Gaussian distribution; based on the mean and the covariance matrix, calculating the estimated probability of each sensor for the monitored target; normalizing all the estimated probabilities to obtain a target estimated probability; and when the target estimated probability is greater than a preset probability threshold, adding the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitored target.

Optionally, the step of fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain the target state value and the target contour value of the monitored target at the current moment includes: calculating the average value of all the current state estimation values corresponding to each of the target sensors to obtain the target state value; determining the first semi-axis length value, the second semi-axis length value and the deflection angle value of the current target contour of the monitored target based on the current contour estimation value; determining the target covariance matrix based on the covariance matrix of all the target sensors; determining the target deflection angle value based on the target covariance matrix; calculating the target first semi-axis length value and the target second semi-axis length value of each target sensor based on the target deflection angle value; obtaining the target semi-axis length value based on all the target first semi-axis length values and the target second semi-axis length values; obtaining the target contour value based on the target semi-axis length value and the target deflection angle value.

According to another aspect of an embodiment of the present invention, a monitoring target fusion device based on multiple sensors is also provided, including: an acquisition unit, used to acquire a state estimation value of a target state of each monitoring target and a contour estimation value of a target contour at a previous moment; a sampling unit, used to perform particle sampling on the state estimation value and the contour estimation value of the monitoring target to obtain weighted particles of a predicted measurement value of the monitoring target at a current moment; a processing unit, used to process the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitoring target at the current moment to obtain a current state estimation value and a current contour estimation value of all the monitoring targets corresponding to the sensor at the current moment; a screening unit, used to screen a set of target sensors corresponding to each monitoring target based on the current state estimation value and the current contour estimation value; and a fusion unit, used to fuse the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain a target state value and a target contour value of the monitoring target at the current moment.

Optionally, the fusion device also includes: a first determination module, which is used to determine the state prediction value of the monitoring target at the current moment based on the state estimation value of the monitoring target at the previous moment after obtaining the state estimation value of the target state of each monitoring target and the contour estimation value of the target contour at the previous moment; a second determination module, which is used to determine the contour prediction value of the monitoring target at the current moment based on the contour estimation value of the monitoring target at the previous moment; and a first output module, which is used to obtain the predicted measurement value of the monitoring target at the current moment based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitoring target at the current moment.

Optionally, the sampling unit includes: a first generation module, used to perform particle sampling on the state estimation value and the profile estimation value to generate multiple particles; a first calculation module, used to calculate the initial particle weight of each particle based on preset parameters; a first processing module, used to normalize each initial particle weight based on all the initial particle weights to obtain the target particle weight of each particle, wherein the particle carrying the target particle weight is characterized as a weighted particle; a first update module, used to update the weighted particles to obtain the weighted particles of the monitored target associated with the predicted measurement value at the current moment.

Optionally, the fusion device also includes: a first acquisition module, used to acquire the measurement data of each sensor before processing the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitoring target at the current moment, wherein the measurement data at least includes: a measurement vector set and a total number of measurements; a first construction module, used to construct a target-associated variable for the monitoring target based on the measurement data, wherein the target-associated variable is used to represent a measurement index indicating a first preset number of bits of measurement generated by a preset monitoring target at the current moment; a second construction module, used to construct a measurement-associated variable for vector measurement based on the measurement data, wherein the measurement-associated variable is used to represent a second preset number of bits of measurement generated by the monitoring target mapped by the measurement index at the current moment.

Optionally, the processing unit includes: a first evaluation module, which is used to perform measurement evaluation on the target-associated variables based on the weighted particles of the predicted measurement values of the monitored target at the current moment and the measurement vector set to obtain a likelihood probability; a third construction module, which is used to construct a correlation factor graph based on the target-associated variables and the measurement-associated variables when the likelihood probability belongs to a profile estimation value within a preset probability threshold range; a second calculation module, which is used to perform iterative calculation based on the correlation factor graph to obtain a first parameter value and a second parameter value; a second update module, which is used to update the measurement data of the monitored target of the sensor based on the first parameter value and the second parameter value to obtain an updated result; and a third calculation module, which is used to obtain a current profile estimation value of the monitored target corresponding to the sensor at the current moment based on the updated result, and to perform weighted calculation on the weighted particles based on the updated result to obtain a current state estimation value of the monitored target corresponding to the sensor.

Optionally, the screening unit includes: a first characterization module, used to characterize the current state estimation value as the mean of a preset Gaussian distribution, and to characterize the current profile estimation value as the covariance matrix of the preset Gaussian distribution; a fourth calculation module, used to calculate the estimated probability of each sensor for the monitoring target based on the mean and the covariance matrix; a second processing module, used to normalize all the estimated probabilities to obtain the target estimated probability; and a first adding module, used to add the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitoring target when the target estimated probability is greater than a preset probability threshold.

Optionally, the fusion unit includes: a fifth calculation module, used to calculate the average value of all the current state estimation values corresponding to each of the target sensors to obtain the target state value; a third determination module, used to determine the first semi-axis length value, the second semi-axis length value and the deflection angle value of the current target contour of the monitored target based on the current contour estimation value; a fourth determination module, used to determine the target covariance matrix based on the covariance matrix of all the target sensors; a fifth determination module, used to determine the target deflection angle value based on the target covariance matrix; a sixth calculation module, used to calculate the target first semi-axis length value and the target second semi-axis length value of each of the target sensors based on the target deflection angle value; a second output module, used to obtain the target semi-axis length value based on all the target first semi-axis length values and the target second semi-axis length values; a third output module, used to obtain the target contour value based on the target semi-axis length value and the target deflection angle value.

According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned multi-sensor based monitoring target fusion method.

According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned multi-sensor based monitoring target fusion method.

In the present disclosure, a state estimation value of a target state and a contour estimation value of a target contour of each monitored target at a previous moment are obtained, particle sampling is performed on the state estimation value and the contour estimation value of the monitored target to obtain weighted particles of predicted measurement values of the monitored target at the current moment, and based on the weighted particles of predicted measurement values of the monitored target at the current moment, measurement data of each sensor is processed to obtain current state estimation values and current contour estimation values of all monitored targets corresponding to the sensor at the current moment, and based on the current state estimation values and the current contour estimation values, a set of target sensors corresponding to each monitored target is screened, and the current state estimation values and the current contour estimation values corresponding to each target sensor in the target sensor set are fused to obtain the target state value and the target contour value of the monitored target at the current moment. In the present disclosure, particle sampling can be first performed on the state estimation value of the target state and the contour estimation value of the target contour of each monitored target at the previous moment to obtain weighted particles of the predicted measurement value of the monitored target at the current moment, and then the measurement data of each sensor is processed according to the weighted particles to obtain the current state estimation value and current contour estimation value of all monitored targets corresponding to each sensor at the current moment, and then the target sensor set of each monitored target is screened, and then the current state estimation value and current contour estimation value corresponding to each target sensor are fused to obtain the target state value and target contour value of the monitored target at the current moment, which can calculate the state value and contour value of the monitored target in real time, and the screening results can be used to efficiently obtain accurate estimates of the state value and contour value of each monitored target, thereby solving the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitored target in the related art.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

FIG1 is a flow chart of an optional multi-sensor based monitoring target fusion method according to an embodiment of the present invention;

2 is a flow chart of an optional multi-sensor multi-extended target fusion algorithm principle for lifeline safety according to an embodiment of the present invention;

FIG3 is a schematic diagram of an optional multi-sensor based monitoring target fusion device according to an embodiment of the present invention;

FIG4 is a hardware structure block diagram of an electronic device (or mobile device) for a multi-sensor based monitoring target fusion method according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

The present invention proposes a multi-sensor and multi-extended target fusion algorithm for urban lifeline safety, which is based on factor graph and sum-product algorithm, and uses voting mechanism to optimize sensor estimation results, so as to realize fast fusion of multi-sensor multi-extended targets. First, particle sampling can be performed on the state and contour estimation of each monitoring target at the previous moment, and then these weighted particles can be updated to obtain the predicted value of the state at the current moment. Then, each sensor performs iterative data association in parallel, so that each obtains an estimate of the state and contour of each target. After that, the estimation results are optimized through voting mechanism, and finally fused to obtain an estimate of the state and contour of each target. The algorithm has the advantages of low computational complexity and high computational speed. At the same time, fast fusion can be realized by using the optimized estimation results, and accurate estimates of the state and contour of each target can be obtained efficiently.

The present invention is described in detail below in conjunction with various embodiments.

Embodiment 1

According to an embodiment of the present invention, an embodiment of a monitoring target fusion method based on multiple sensors is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

FIG. 1 is a flow chart of an optional multi-sensor based monitoring target fusion method according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

Step S101, obtaining a state estimation value of a target state of each monitored target and a contour estimation value of a target contour at a previous moment.

Step S102, performing particle sampling on the state estimation value and the profile estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment.

Step S103, based on the weighted particles of the predicted measurement values of the monitoring targets at the current moment, the measurement data of each sensor is processed to obtain the current state estimation values and the current contour estimation values of all monitoring targets corresponding to the sensor at the current moment.

Step S104: based on the current state estimation value and the current profile estimation value, filter the target sensor set corresponding to each monitoring target.

Step S105 , fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain the target state value and the target contour value of the monitored target at the current moment.

Through the above steps, the state estimation value of the target state and the contour estimation value of the target contour of each monitored target at the previous moment can be obtained, and the state estimation value and the contour estimation value of the monitored target can be particle sampled to obtain the weighted particles of the predicted measurement value of the monitored target at the current moment. Based on the weighted particles of the predicted measurement value of the monitored target at the current moment, the measurement data of each sensor is processed to obtain the current state estimation value and the current contour estimation value of all monitored targets corresponding to the sensor at the current moment. Based on the current state estimation value and the current contour estimation value, the target sensor set corresponding to each monitored target is screened, and the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set are fused to obtain the target state value and the target contour value of the monitored target at the current moment. In an embodiment of the present invention, particle sampling can be first performed on the state estimation value of the target state and the contour estimation value of the target contour of each monitored target at the previous moment to obtain weighted particles of the predicted measurement value of the monitored target at the current moment, and then the measurement data of each sensor is processed according to the weighted particles to obtain the current state estimation value and current contour estimation value of all monitored targets corresponding to each sensor at the current moment, and then the target sensor set of each monitored target is screened, and then the current state estimation value and current contour estimation value corresponding to each target sensor are fused to obtain the target state value and target contour value of the monitored target at the current moment, so that the state value and contour value of the monitored target can be calculated in real time, and the screening result can be used to efficiently obtain accurate estimates of the state value and contour value of each monitored target, thereby solving the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitored target in the related art.

The embodiment of the present invention is described in detail below in combination with the above steps.

In this embodiment, it is assumed that S sensors are used to monitor K monitoring targets, where S and K are fixed and known. x _k (t) is used to represent the state of the monitoring target k∈{1,2,…,K} at time t.

First, set the model of the monitoring target as:

x _k (t)=Fx _k (t-1)+v _k (t-1)(1);

Among them, _vk (t-1) is Gaussian white noise with zero mean, that is, _vk (t-1)~N(0, _Qk (t-1)), _Qk (t-1) represents the covariance, F is the state transfer matrix, and _xk (t-1) represents the state of the monitored target at time t-1.

个量测，将第

个测量记为

定义量测向量

Sensor s∈{1,2,…,S} generates

A measurement,

The measurement is recorded as

Defining the measurement vector

The target contour can be described by using the major and minor semi-axis lengths and the deflection angle (l ₁ , l ₂ , θ). In this embodiment, it is assumed that the deflection angle of the target contour is consistent with the target motion direction.

The observation model of the target at sensor s is set as:

表示t时刻目标k在传感器s上产生的某个量测，μ_k(t)是服从零均值的高斯白噪声，即μ_k(t)～N(0,R_k(t))，R_k(t)表示协方差，X_k(t)表示监测目标在t时刻的轮廓，h(·)是一个非线性函数：in,

represents a measurement of target k on sensor s at time t, μ _k (t) is Gaussian white noise with zero mean, that is, μ _k (t) ~ N (0, R _k (t)), R _k (t) represents the covariance, X _k (t) represents the profile of the monitored target at time t, and h (·) is a nonlinear function:

Among them, h ₁ and h ₂ are random variables with mean 0 and uniform distribution in the interval [-1,1].

In this embodiment, the initial target states x _k (0) and initial target profiles X _k (0) of all monitored targets are known.

Step S101, obtaining a state estimation value of a target state of each monitored target and a contour estimation value of a target contour at a previous moment.

In an embodiment of the present invention, the state of each monitored target at time t-1 (i.e., the previous moment) and the estimation of the profile parameters can be obtained first (i.e., the state estimation value of the target state of each monitored target and the profile estimation value of the target profile at the previous moment are obtained).

Optionally, after obtaining the state estimation value of the target state of each monitoring target at the previous moment and the contour estimation value of the target contour, it also includes: determining the state prediction value of the monitoring target at the current moment based on the state estimation value of the monitoring target at the previous moment; determining the contour prediction value of the monitoring target at the current moment based on the contour estimation value of the monitoring target at the previous moment; and obtaining the predicted measurement value of the monitoring target at the current moment based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitoring target at the current moment.

In the embodiment of the present invention, for each monitoring target k, there are:

分别表示t-1时刻下监测目标k的状态以及轮廓的估计。in,

They respectively represent the state and contour estimation of the monitored target k at time t-1.

以及轮廓估计值

确定监测目标在当前时刻下的状态预测值以及轮廓预测值(即可以根据前一时刻下监测目标的状态以及轮廓的估计，通过公式(4)、公式(5)以及公式(1)得到监测目标在当前时刻下的状态预测值以及轮廓预测值)。In this embodiment, the state estimation value of the monitoring target at the previous moment can be used.

and the silhouette estimate

Determine the state prediction value and profile prediction value of the monitoring target at the current moment (that is, the state prediction value and profile prediction value of the monitoring target at the current moment can be obtained through formula (4), formula (5) and formula (1) based on the estimation of the state and profile of the monitoring target at the previous moment).

In this embodiment, the predicted measurement value is the measurement value corresponding to the predicted state of the monitoring target at the current moment (ie, the measurement value corresponding to the predicted state of the monitoring target at moment t).

Step S102, performing particle sampling on the state estimation value and the profile estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment.

Optionally, the step of performing particle sampling on the state estimation value and the profile estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment includes: performing particle sampling on the state estimation value and the profile estimation value to generate multiple particles; calculating the initial particle weight of each particle based on preset parameters; normalizing each initial particle weight based on all initial particle weights to obtain the target particle weight of each particle, wherein particles carrying the target particle weight are characterized as weighted particles; updating the weighted particles to obtain weighted particles associated with the predicted measurement value of the monitoring target at the current moment.

In an embodiment of the present invention, particle sampling may be performed on the state estimation value and the profile estimation value of each monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment. Specifically:

其中，w_k,j表示粒子j的粒子权重。For each set of target states and target profiles (x _k (t), X _k (t)), particle sampling can generate J particles (i.e., particle sampling is performed on the state estimation value and the profile estimation value to generate multiple particles), and a set of weighted particles is obtained.

Where w _k,j represents the particle weight of particle j.

Particle weights are calculated as follows:

w _k,j =f(x _k,j |x _k (t),cX _k (t))(6);

Among them, c>1 is the amplification factor (i.e., the preset parameter), and all weights are normalized after calculation (i.e., based on the preset parameters, the initial particle weight of each particle is calculated, and then based on all the initial particle weights, each initial particle weight is normalized to obtain the target particle weight of each particle):

In this embodiment, the particles carrying the weight of the target particles can be characterized as weighted particles. The weighted particles are then updated to obtain weighted particles with associated predicted measurement values of the monitoring target at the current moment (i.e., the weighted particles obtained by sampling are updated to obtain weighted particles with corresponding measurement values of the predicted state of each target at time t). The formula for updating the state in the weighted particles is:

x _k,j =Fx _k,j (8);

Where F represents the state transfer matrix.

Optionally, based on the weighted particles of the predicted measurement values of the monitored target at the current moment, before processing the measurement data of each sensor, it also includes: obtaining the measurement data of each sensor, wherein the measurement data at least includes: a measurement vector set and a total number of measurements; based on the measurement data, constructing a target-associated variable for the monitored target, wherein the target-associated variable is used to represent a measurement index indicating a first preset number of bits of measurement generated by a preset monitored target at the current moment; based on the measurement data, constructing a measurement-associated variable for vector measurement, wherein the measurement-associated variable is used to represent a second preset number of bits of measurement generated by the monitored target mapped by the measurement index at the current moment.

个量测，为了区分监测目标k产生哪些量测，传感器s使用面向监测目标的关联变量进行描述：In the embodiment of the present invention, for any sensor s, the monitoring target k generates at most

In order to distinguish which measurements are generated by the monitoring target k, the sensor s is described using associated variables oriented to the monitoring target:

In order to indicate whether the measurement is from the target or from clutter, the sensor s defines the following variables associated with the measurement:

表示传感器s产生的量测的总个数。假设各目标产生的最大量测数为

面向目标k的关联变量α_kq表示目标k产生的第q个量测对应的量测索引，其值范围为

其中，0表示未产生对应的量测；面向量测的关联变量β_m＝(k,q)，采用二元组表示索引为m的量测对应为目标k产生的第q个量测。In this embodiment, the sensor index can be defined as s∈{1,2,…,S}, where S is the total number of sensors; the target index can be defined as k∈{1,2,…,K}, where K is the total number of targets, and is assumed to be known. _z s is used to represent the set of measurements of sensor s,

represents the total number of measurements generated by sensor s. Assume that the maximum number of measurements generated by each target is

The associated variable α _kq for target k represents the measurement index corresponding to the qth measurement generated by target k, and its value range is

Wherein, 0 indicates that no corresponding measurement is generated; the associated variable β _m =(k,q) of the face vector measurement uses a two-tuple to indicate that the measurement with index m corresponds to the qth measurement generated for target k.

然后根据量测数据，构建面向监测目标的目标关联变量α_kq，该目标关联变量用于表示预设监测目标在当前时刻下产生的第一预设位数(例如，第q个)的量测指示的量测索引，并根据量测数据，构建面向量测的量测关联变量β_m＝(k,q)，该量测关联变量用于表示在当前时刻下由量测索引所映射的监测目标产生的第二预设位数(例如，第q个)的量测。In this embodiment, the measurement data of each sensor can be obtained first, and the measurement data at least includes: a measurement vector set zs, a measurement total number

Then, based on the measurement data, a target-associated variable α _kq for the monitoring target is constructed, and the target-associated variable is used to represent the measurement index of the measurement indication of the first preset number of bits (for example, the qth) generated by the preset monitoring target at the current moment. And based on the measurement data, a measurement-associated variable β _m = (k, q) for vector measurement is constructed, and the measurement-associated variable is used to represent the second preset number of bits (for example, the qth) of measurement generated by the monitoring target mapped by the measurement index at the current moment.

Step S103, based on the weighted particles of the predicted measurement values of the monitoring targets at the current moment, the measurement data of each sensor is processed to obtain the current state estimation values and the current contour estimation values of all monitoring targets corresponding to the sensor at the current moment.

Optionally, based on the weighted particles of the predicted measurement values of the monitored targets at the current moment, the measurement data of each sensor is processed to obtain the current state estimation values and the current profile estimation values of all the monitored targets corresponding to the sensor at the current moment, including: based on the weighted particles of the predicted measurement values of the monitored targets at the current moment and the set of measurement vectors, the target-associated variables are measured and evaluated to obtain the likelihood probability; in the case where the likelihood probability belongs to the profile estimation value within the preset probability threshold range, an association factor graph is constructed based on the target-associated variables and the measurement-associated variables; based on the association factor graph, an iterative calculation is performed to obtain a first parameter value and a second parameter value; based on the first parameter value and the second parameter value, the measurement data of the monitored targets of the sensor are updated to obtain an updated result; based on the updated result, the current profile estimation value of the monitored targets corresponding to the sensor at the current moment is obtained, and based on the updated result, the weighted particles are weighted calculated to obtain the current state estimation value of the monitored targets corresponding to the sensor.

In the embodiment of the present invention, all sensors s can perform internal data association iterations in parallel, so that each can obtain the state of each target at time t and the estimation of the profile parameters (that is, the measurement data of each sensor can be processed based on the weighted particles of the predicted measurement values of the monitoring targets at the current moment to obtain the current state estimation values and current profile estimation values of all monitoring targets corresponding to the sensor at the current moment). Specifically:

(1) Measurement evaluation: Use the weighted particles corresponding to the measurement value of the target's predicted state at time t to evaluate the measurement data by calculating the corresponding likelihood probability (i.e., based on the weighted particles and measurement vector set of the predicted measurement value of the monitoring target at the current time, the target-associated variable is measured and evaluated to obtain the likelihood probability):

对于各目标k的关联变量α_kq进行量测评估，具体评估公式如下：use

The associated variable α _kq of each target k is measured and evaluated. The specific evaluation formula is as follows:

表示带权粒子，J为粒子总数；j_kq(x_k,j,X_k,α_kq；z^s)计算公式如下：in,

represents weighted particles, J is the total number of particles; the calculation formula of j _kq (x _k,j ,X _k ,α _kq ;z ^s ) is as follows:

为

在以x_k,j为均值，X_k为协方差矩阵的高斯分布下的概率；μfa为在测量场景中虚警量测出现个数的均值，本实施例中假设均值的数量服从泊松分布，在量测范围内服从均匀分布；

表示传感器s产生的量测的总个数，

表示传感器s产生的第m个量测，

表示对应量测的虚警概率。Wherein, X _k represents the target profile of target k, and a random matrix is used in this embodiment;

for

The probability under Gaussian distribution with x _k,j as the mean and X _k as the covariance matrix; μfa is the mean of the number of false alarm measurements in the measurement scenario. In this embodiment, it is assumed that the number of means follows a Poisson distribution and follows a uniform distribution within the measurement range;

represents the total number of measurements produced by sensor s,

represents the mth measurement produced by sensor s,

Represents the false alarm probability of the corresponding measurement.

(2) Iterative data association, by constructing a data association factor graph for the association variables facing the monitoring target and the measurement, and obtaining the probability corresponding to each association situation through iterative calculation (that is, under the situation profile estimation value where the likelihood probability belongs to the preset probability threshold range, the association factor graph is constructed based on the target association variable and the measurement association variable, and iterative calculation is performed based on the association factor graph to obtain the first parameter value and the second parameter value):

其中，nit表示循环计算的总次数，每个迭代计算的过程中，对于所有可行的二元组

以及所有量测

进行如下计算：All sensors can be executed in parallel, using their own measurement data for iterative data association. Define the iterative calculation index as

Among them, nit represents the total number of loop calculations. During each iterative calculation, for all feasible binary pairs

And all measurements

Perform the following calculations:

表示对除选定的二元组(k,q)进行和积算法的运算；in,

It means to perform the sum-product algorithm operation on the selected two-tuple (k,q);

表示α_k包含的有效量测个数，

表示有效量测个数对应的概率。

represents the number of valid measurements contained in α _k ,

Represents the probability corresponding to the number of valid measurements.

进行如下计算：for

Perform the following calculations:

And the following initial calculation is performed for the iterative calculation:

By using the characteristics of factor graphs to optimize the iterative data association calculation, the above calculation formula can be replaced as follows:

in,

以及第二参数值

)，对加权粒子进行测量更新，同时根据关联概率选择合适的一部分量测值对目标的轮廓进行估计(即基于第一参数值以及第二参数值，更新传感器的监测目标的量测数据，得到更新结果，并基于更新结果，得到与传感器对应的在当前时刻下监测目标的当前轮廓估计值)：(3) Measurement update, using the result obtained by iterative data association (i.e., the first parameter value)

And the second parameter value

), measure and update the weighted particles, and select a suitable part of the measured values according to the associated probability to estimate the contour of the target (that is, based on the first parameter value and the second parameter value, update the measurement data of the monitoring target of the sensor to obtain the updated result, and based on the updated result, obtain the current contour estimation value of the monitoring target corresponding to the sensor at the current moment):

以及

进行如下量测更新：After iterative data association calculation, we get

as well as

Perform the following measurement updates:

并进行如下更新：For each target, select the measurement used to update the contour

And update as follows:

Among them, γ represents the association probability, _Zk represents the covariance matrix corresponding to the selected measurement, _τk and _νk are parameters in the random matrix update.

(4) Target estimation: weighted calculation is performed on the weighted particles to obtain an estimate of the target state (i.e., based on the update result, weighted calculation is performed on the weighted particles to obtain an estimate of the current state of the monitoring target corresponding to the sensor):

Use the obtained measurement update results and combine each group of weighted particle pairs to estimate the state of each target:

Step S104: based on the current state estimation value and the current profile estimation value, filter the target sensor set corresponding to each monitoring target.

Optionally, based on the current state estimation value and the current profile estimation value, the step of screening the target sensor set corresponding to each monitoring target includes: characterizing the current state estimation value as the mean of a preset Gaussian distribution, and characterizing the current profile estimation value as the covariance matrix of a preset Gaussian distribution; based on the mean and the covariance matrix, calculating the estimated probability of each sensor for the monitoring target; normalizing all estimated probabilities to obtain the target estimated probability; and when the target estimated probability is greater than a preset probability threshold, adding the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitoring target.

In the embodiment of the present invention, the fusion center can select the estimate with higher accuracy based on the voting mechanism according to the estimate obtained by each sensor (that is, based on the current state estimate value and the current profile estimate value, filter the target sensor set corresponding to each monitoring target). Specifically:

由融合中心对于各估计进行选择，过程如下：Estimation of target state and target contour obtained by each sensor

The fusion center makes a selection for each estimate, and the process is as follows:

)：For each target k, sensor s, the remaining sensors' estimates of the state and profile are regarded as the mean and covariance matrix of the Gaussian distribution, the corresponding probabilities are calculated, and cross-validation voting is performed (i.e., the current state estimate is represented as the mean of the preset Gaussian distribution, and the current profile estimate is represented as the covariance matrix of the preset Gaussian distribution; based on the mean and covariance matrix, the estimated probability of each sensor for the monitored target is calculated

):

After the calculation is completed, the probabilities of all sensors of the same target k are normalized (that is, all estimated probabilities are normalized to obtain the target estimated probability), and the sensor estimates greater than the threshold ε are selected for subsequent fusion. The sensor index set selected by target k is represented as S _k (that is, when the target estimated probability is greater than the preset probability threshold ε, the sensor corresponding to the target estimated probability is added to the target sensor set corresponding to the monitoring target).

Step S105 , fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain the target state value and the target contour value of the monitored target at the current moment.

Optionally, the step of fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain the target state value and the target contour value of the monitored target at the current moment includes: calculating the average of all the current state estimation values corresponding to each target sensor to obtain the target state value; determining the first semi-axis length value, the second semi-axis length value and the deflection angle value of the current target contour of the monitored target based on the current contour estimation value; determining the target covariance matrix based on the covariance matrix of all target sensors; determining the target deflection angle value based on the target covariance matrix; calculating the target first semi-axis length value and the target second semi-axis length value of each target sensor based on the target deflection angle value; obtaining the target semi-axis length value based on all target first semi-axis length values and target second semi-axis length values; obtaining the target contour value based on the target semi-axis length value and the target deflection angle value.

In an embodiment of the present invention, the estimates in the preferred target sensor set are fused to ultimately obtain estimates of the state and profile parameters of each target at time t (i.e., the current state estimate value and the current profile estimate value corresponding to each target sensor in the target sensor set are fused to obtain the target state value and the target profile value of the monitored target at the current moment).

For target k, the state and contour are estimated according to the determined sensor index set _Sk .

For the target state, the arithmetic average method is used for calculation (that is, the average value of all current state estimates corresponding to each target sensor is calculated to obtain the target state value):

For the contour of the target, it can be first converted into the form of semi-axis length plus deflection angle (that is, based on the current contour estimation value, the first semi-axis length value, the second semi-axis length value and the deflection angle value of the current target contour of the monitored target are determined, wherein the first semi-axis length value is the major semi-axis length, and the second semi-axis length value is the minor semi-axis length):

Among them, l _s1 ,l _s2 ,θ _s are the major semi-axis length, minor semi-axis length and deflection angle respectively.

进行相乘得到最终的协方差矩阵σ²(即基于所有目标传感器的协方差矩阵，确定目标协方差矩阵)，高斯分布相乘的协方差矩阵公式如下：For the estimation of the contour by each sensor, it is regarded as the covariance matrix of the Gaussian distribution.

The final covariance matrix σ ² is obtained by multiplication (i.e., the target covariance matrix is determined based on the covariance matrices of all target sensors). The formula of the covariance matrix of Gaussian distribution multiplication is as follows:

(即基于目标协方差矩阵，确定目标偏转角度值)，由此可以计算各传感器对应目标长短半轴的估计(即基于目标偏转角度值，计算每个目标传感器的目标第一半轴长值

以及目标第二半轴长值

)：The estimated target deflection angle is determined by the obtained covariance matrix σ2

(i.e., based on the target covariance matrix, determine the target deflection angle value), from which the estimation of the target long and short semi-axis corresponding to each sensor can be calculated (i.e., based on the target deflection angle value, calculate the target first semi-axis length value of each target sensor

And the target second semi-axis length value

):

再基于目标半轴长值以及目标偏转角度值，得到目标轮廓值)：Finally, the estimation of the major and minor semi-axis lengths is obtained by the arithmetic mean method (i.e., the target semi-axis length value is obtained based on all the target first semi-axis length values and the target second semi-axis length values).

Then, based on the target semi-axis length value and the target deflection angle value, the target contour value is obtained):

FIG2 is a flow chart of an optional multi-sensor multi-extended target fusion algorithm principle for lifeline safety according to an embodiment of the present invention. As shown in FIG2 , the flow chart includes the following steps:

Step (1): Estimation of the target at time t-1, that is, obtaining the state of each target at time t-1 and the estimation of the profile parameters;

Step (2): Particle sampling, i.e., performing particle sampling on the estimation of each target;

Step (3): state update, that is, updating the weighted particles obtained by sampling in step (2) to obtain the weighted particles of the measurement values corresponding to the predicted state of each target at time t;

Step (4): sensor parallel data association iteration, that is, all sensors 1, ..., sensor s, etc. perform internal data association iteration in parallel, and each obtains the state of each target at time t and the estimation of the profile parameters;

Step (5): Optimizing the estimation, i.e., the fusion center selects the estimation with the higher accuracy based on the estimation obtained by each sensor based on the voting mechanism;

Step (6): estimation fusion, that is, fusing the preferred estimates in step (5) to finally obtain an estimate of the target at time t (that is, an estimate of the state of each target and the profile parameters at time t).

In the embodiments of the present invention, sensor multi-extended target tracking can be realized based on factor graph optimization, and after optimizing sensor estimation results by using a voting mechanism, multi-sensor extended target estimation fusion can be realized, thereby achieving the following beneficial effects: (1) multi-extended target tracking is realized based on factor graph optimization, with fast calculation speed and no need for preprocessing operations such as clustering; (2) the voting mechanism optimization and multi-sensor extended target estimation fusion can ensure the optimization effect by using cross-validation voting, and effectively improve the accuracy of the estimation results obtained using the measurement data of multiple sensors.

The following is a detailed description in conjunction with another embodiment.

Embodiment 2

A multi-sensor based monitoring target fusion device provided in this embodiment includes multiple implementation units, each implementation unit corresponds to each implementation step in the above-mentioned embodiment 1.

FIG3 is a schematic diagram of an optional multi-sensor based monitoring target fusion device according to an embodiment of the present invention. As shown in FIG3 , the fusion device may include: an acquisition unit 30, a sampling unit 31, a processing unit 32, a screening unit 33, and a fusion unit 34, wherein:

An acquisition unit 30 is used to acquire a state estimation value of a target state of each monitoring target and a contour estimation value of a target contour at a previous moment;

The sampling unit 31 is used to perform particle sampling on the state estimation value and the profile estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment;

A processing unit 32 is used to process the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitoring target at the current moment, and obtain the current state estimation value and the current profile estimation value of all monitoring targets corresponding to the sensor at the current moment;

A screening unit 33, configured to screen a target sensor set corresponding to each monitoring target based on a current state estimation value and a current profile estimation value;

The fusion unit 34 is used to fuse the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain the target state value and the target contour value of the monitored target at the current moment.

The above-mentioned fusion device can obtain the state estimation value of the target state and the contour estimation value of the target contour of each monitored target at the previous moment through the acquisition unit 30, perform particle sampling on the state estimation value and the contour estimation value of the monitored target through the sampling unit 31, and obtain the weighted particles of the predicted measurement value of the monitored target at the current moment, and process the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitored target at the current moment through the processing unit 32 to obtain the current state estimation value and the current contour estimation value of all monitored targets corresponding to the sensor at the current moment, and screen the target sensor set corresponding to each monitored target based on the current state estimation value and the current contour estimation value through the screening unit 33, and fuse the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set through the fusion unit 34 to obtain the target state value and the target contour value of the monitored target at the current moment. In an embodiment of the present invention, particle sampling can be first performed on the state estimation value of the target state and the contour estimation value of the target contour of each monitored target at the previous moment to obtain weighted particles of the predicted measurement value of the monitored target at the current moment, and then the measurement data of each sensor is processed according to the weighted particles to obtain the current state estimation value and current contour estimation value of all monitored targets corresponding to each sensor at the current moment, and then the target sensor set of each monitored target is screened, and then the current state estimation value and current contour estimation value corresponding to each target sensor are fused to obtain the target state value and target contour value of the monitored target at the current moment, so that the state value and contour value of the monitored target can be calculated in real time, and the screening result can be used to efficiently obtain accurate estimates of the state value and contour value of each monitored target, thereby solving the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitored target in the related art.

Optionally, the fusion device also includes: a first determination module, which is used to determine the state prediction value of the monitoring target at the current moment based on the state estimation value of the monitoring target at the previous moment after obtaining the state estimation value of the target state of each monitoring target and the contour estimation value of the target contour at the previous moment; a second determination module, which is used to determine the contour prediction value of the monitoring target at the current moment based on the contour estimation value of the monitoring target at the previous moment; and a first output module, which is used to obtain the predicted measurement value of the monitoring target at the current moment based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitoring target at the current moment.

Optionally, the sampling unit includes: a first generation module, used to perform particle sampling on state estimation values and profile estimation values to generate multiple particles; a first calculation module, used to calculate the initial particle weight of each particle based on preset parameters; a first processing module, used to normalize each initial particle weight based on all initial particle weights to obtain a target particle weight for each particle, wherein particles carrying target particle weights are characterized as weighted particles; a first update module, used to update weighted particles to obtain weighted particles associated with predicted measurement values of the monitoring target at the current moment.

Optionally, the fusion device also includes: a first acquisition module, which is used to acquire the measurement data of each sensor before processing the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitoring target at the current moment, wherein the measurement data at least includes: a measurement vector set and a total number of measurements; a first construction module, which is used to construct a target-associated variable for the monitoring target based on the measurement data, wherein the target-associated variable is used to represent a measurement index indicating a measurement indication of a first preset number of bits generated by a preset monitoring target at the current moment; and a second construction module, which is used to construct a measurement-associated variable for vector measurement based on the measurement data, wherein the measurement-associated variable is used to represent a second preset number of bits of measurement generated by the monitoring target mapped by the measurement index at the current moment.

Optionally, the processing unit includes: a first evaluation module, which is used to measure and evaluate the target associated variables based on the weighted particles and measurement vector set of the predicted measurement values of the monitored target at the current moment to obtain the likelihood probability; a third construction module, which is used to construct a correlation factor graph based on the target associated variables and the measurement associated variables under the profile estimation value when the likelihood probability belongs to the preset probability threshold range; a second calculation module, which is used to perform iterative calculation based on the correlation factor graph to obtain the first parameter value and the second parameter value; a second update module, which is used to update the measurement data of the monitored target of the sensor based on the first parameter value and the second parameter value to obtain an updated result; a third calculation module, which is used to obtain the current profile estimation value of the monitored target corresponding to the sensor at the current moment based on the updated result, and to perform weighted calculation on the weighted particles based on the updated result to obtain the current state estimation value of the monitored target corresponding to the sensor.

Optionally, the screening unit includes: a first characterization module, used to characterize the current state estimation value as the mean of a preset Gaussian distribution, and to characterize the current profile estimation value as the covariance matrix of a preset Gaussian distribution; a fourth calculation module, used to calculate the estimated probability of each sensor for the monitored target based on the mean and the covariance matrix; a second processing module, used to normalize all estimated probabilities to obtain the target estimated probability; and a first adding module, used to add the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitored target when the target estimated probability is greater than a preset probability threshold.

Optionally, the fusion unit includes: a fifth calculation module, used to calculate the average value of all current state estimation values corresponding to each target sensor to obtain a target state value; a third determination module, used to determine the first semi-axis length value, the second semi-axis length value and the deflection angle value of the current target contour of the monitored target based on the current contour estimation value; a fourth determination module, used to determine the target covariance matrix based on the covariance matrix of all target sensors; a fifth determination module, used to determine the target deflection angle value based on the target covariance matrix; a sixth calculation module, used to calculate the target first semi-axis length value and the target second semi-axis length value of each target sensor based on the target deflection angle value; a second output module, used to obtain the target semi-axis length value based on all target first semi-axis length values and target second semi-axis length values; a third output module, used to obtain the target contour value based on the target semi-axis length value and the target deflection angle value.

The above-mentioned fusion device may also include a processor and a memory. The above-mentioned acquisition unit 30, sampling unit 31, processing unit 32, screening unit 33, fusion unit 34, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set are fused by adjusting the kernel parameters to obtain the target state value and the target contour value of the monitored target at the current moment.

The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining a state estimation value of a target state and a contour estimation value of a target contour of each monitored target at a previous moment, performing particle sampling on the state estimation value and the contour estimation value of the monitored target to obtain weighted particles of a predicted measurement value of the monitored target at a current moment, processing the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitored target at the current moment to obtain a current state estimation value and a current contour estimation value of all monitored targets corresponding to the sensor at the current moment, screening a set of target sensors corresponding to each monitored target based on the current state estimation value and the current contour estimation value, fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the target sensor set to obtain a target state value and a target contour value of the monitored target at the current moment.

According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned multi-sensor-based monitoring target fusion method.

According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned multi-sensor based monitoring target fusion method.

FIG. 4 is a hardware structure block diagram of an electronic device (or mobile device) for a multi-sensor-based monitoring target fusion method according to an embodiment of the present invention. As shown in FIG. 4 , the electronic device may include one or more (402a, 402b, ..., 402n are used in FIG. 4 to illustrate) processors 402 (the processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 404 for storing data. In addition, it may also include: a display, an input/output interface (I/O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a keyboard, a power supply and/or a camera. It can be understood by those skilled in the art that the structure shown in FIG. 4 is only for illustration and does not limit the structure of the above-mentioned electronic device. For example, the electronic device may also include more or fewer components than those shown in FIG. 4 , or have a configuration different from that shown in FIG. 4 .

The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims (10)

1. A multi-sensor-based monitoring target fusion method, comprising:

acquiring a state estimation value of a target state of each monitoring target and a contour estimation value of a target contour at the previous moment;

sampling particles of the state estimation value and the contour estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment;

processing the measurement data of each sensor based on weighted particles of the predicted measurement values of the monitoring targets at the current time to obtain current state estimation values and current contour estimation values of all the monitoring targets corresponding to the sensors at the current time;

screening a target sensor set corresponding to each monitoring target based on the current state estimated value and the current contour estimated value;

And fusing the current state estimated value and the current contour estimated value corresponding to each target sensor in the target sensor set to obtain a target state value and a target contour value of the monitoring target at the current moment.

2. The fusion method according to claim 1, further comprising, after acquiring the state estimation value of the target state and the contour estimation value of the target contour of each monitoring target at the previous time, the steps of:

determining a state predicted value of the monitoring target at the current time based on the state estimated value of the monitoring target at the previous time;

determining a contour predicted value of the monitoring target at the current time based on the contour estimated value of the monitoring target at the previous time;

and obtaining a predicted measurement value of the monitoring target at the current time based on the state predicted value and the contour predicted value, wherein the predicted measurement value is a measurement value corresponding to the predicted state of the monitoring target at the current time.

3. The fusion method according to claim 2, wherein the step of sampling the state estimation value and the contour estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current time, includes:

Performing particle sampling on the state estimation value and the contour estimation value to generate a plurality of particles;

calculating an initial particle weight of each particle based on preset parameters;

normalizing each initial particle weight based on all the initial particle weights to obtain a target particle weight of each particle, wherein the particles carrying the target particle weights are characterized as weighted particles;

and updating the weighted particles to obtain weighted particles of the monitoring target associated with the predicted measurement value at the current moment.

4. The fusion method of claim 1, wherein prior to processing the measurement data for each sensor based on weighted particles of the predicted measurement value of the monitored target at the current time, further comprising:

acquiring the measurement data of each sensor, wherein the measurement data at least comprises: a set of measurement vectors, a total number of measurements;

based on the measurement data, constructing a target associated variable facing a monitoring target, wherein the target associated variable is used for representing a measurement index of a measurement instruction of a first preset bit number generated by a preset monitoring target at the current moment;

And constructing a measurement-oriented measurement related variable based on the measurement data, wherein the measurement related variable is used for representing measurement of a second preset bit number generated by a monitoring target mapped by the measurement index at the current time.

5. The fusion method according to claim 4, wherein the step of processing the measurement data of each sensor based on weighted particles of the predicted measurement values of the monitoring targets at the current time to obtain the current state estimation values and the current contour estimation values of all the monitoring targets corresponding to the sensors at the current time includes:

based on weighted particles of the predicted measurement value of the monitoring target at the current time and the measurement vector set, carrying out measurement evaluation on the target associated variable to obtain likelihood probability;

under the condition that the likelihood probability belongs to a preset probability threshold range, constructing a correlation factor graph based on the target correlation variable and the measurement correlation variable;

performing iterative computation based on the associated factor graph to obtain a first parameter value and a second parameter value;

updating the measurement data of the monitoring target of the sensor based on the first parameter value and the second parameter value to obtain an updating result;

And based on the updating result, obtaining a current contour estimated value of the monitoring target corresponding to the sensor at the current time, and based on the updating result, carrying out weighted calculation on the weighted particles to obtain a current state estimated value of the monitoring target corresponding to the sensor.

6. The fusion method according to claim 1, wherein the step of screening the target sensor set corresponding to each of the monitoring targets based on the current state estimation value and the current contour estimation value comprises:

characterizing the current state estimation value as a mean value of a preset Gaussian distribution, and characterizing the current contour estimation value as a covariance matrix of the preset Gaussian distribution;

calculating the estimated probability of each sensor to the monitoring target based on the mean value and the covariance matrix;

normalizing all the estimated probabilities to obtain target estimated probabilities;

and adding the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitoring target under the condition that the target estimated probability is larger than a preset probability threshold.

7. The fusion method according to claim 6, wherein the step of fusing the current state estimation value and the current contour estimation value corresponding to each target sensor in the set of target sensors to obtain a target state value and a target contour value of the monitoring target at the current time, includes:

calculating the average value of all the current state estimation values corresponding to each target sensor to obtain the target state value;

determining a first half-axis length value, a second half-axis length value and a deflection angle value of a current target profile of the monitoring target based on the current profile estimation value;

determining a target covariance matrix based on the covariance matrices of all the target sensors;

determining a target deflection angle value based on the target covariance matrix;

calculating a target first half-axis length value and a target second half-axis length value of each target sensor based on the target deflection angle values;

obtaining a target half-axis length value based on all the target first half-axis length values and the target second half-axis length values;

and obtaining the target profile value based on the target half-axis length value and the target deflection angle value.

8. A multi-sensor based monitoring target fusion device, comprising:

the acquisition unit is used for acquiring a state estimation value of a target state of each monitoring target and a contour estimation value of a target contour at the previous moment;

the sampling unit is used for sampling particles of the state estimation value and the contour estimation value of the monitoring target to obtain weighted particles of the predicted measurement value of the monitoring target at the current moment;

the processing unit is used for processing the measurement data of each sensor based on weighted particles of the predicted measurement value of the monitoring target at the current time to obtain current state estimation values and current contour estimation values of all the monitoring targets corresponding to the sensors at the current time;

a screening unit, configured to screen a target sensor set corresponding to each monitoring target based on the current state estimation value and the current contour estimation value;

and the fusion unit is used for fusing the current state estimated value and the current contour estimated value corresponding to each target sensor in the target sensor set to obtain a target state value and a target contour value of the monitoring target at the current time.

9. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program, when run, controls a device in which the computer readable storage medium is located to perform the multi-sensor based monitoring target fusion method according to any one of claims 1 to 7.

10. An electronic device comprising one or more processors and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the multi-sensor based monitoring target fusion method of any of claims 1-7.

Images