CN116310900A - A target recognition and tracking method, device and system on a complex battlefield - Google Patents

Abstract

The embodiment of the invention provides a method, a device and a system for identifying and tracking targets on a complex battlefield, wherein the method comprises the following steps: acquiring a first image set containing a target object, wherein the types of first images in the first image set are different; deblurring and denoising the first image set to form a second image set; performing image fusion processing on the second image set to form a fusion image; constructing a target tracking algorithm model for identifying a target object in an image and tracking and marking, wherein the target tracking algorithm model is based on a single-stage target detection model and is obtained at least by adjusting the positions of a feature extraction module and a pooling layer and the number of convolution layers in the single-stage target detection model; and inputting the fusion image into the target tracking algorithm model to obtain a recognition and tracking result related to the target object based on the target tracking algorithm model.

Description

A target recognition and tracking method, device and system on a complex battlefield

technical field

Embodiments of the present invention relate to the technical field of target recognition and tracking, and in particular to a method and device for target recognition and tracking on complex battlefields.

Background technique

With the increasing unmanned trend of modern combat, the application of drones in the military field has become more and more important. It can not only undertake reconnaissance, strike, transportation and other tasks, but also effectively guarantee the safety of combatants. The subversive effect is self-evident. As an important component of battlefield target reconnaissance, drones are conducive to commanders to fully grasp the changing situation of the entire battlefield. In the future of information warfare, improving battlefield situation awareness can effectively improve the overall control of the war. All major military powers are strengthening the development and research of related technologies.

Due to the high dynamics of the battlefield situation, there are high requirements for the real-time performance and accuracy of typical target detection and tracking on the battlefield. In addition, when performing target detection on the battlefield image information obtained by UAV reconnaissance, military targets are affected by motion blur, noise pollution, climate interference, target size, and partial target occlusion during the process, which makes the battlefield situation awareness difficult. Performance is difficult to meet the needs of reality. The main problems are manifested in several aspects:

1) The battlefield environment is complex and changeable, and the reconnaissance missions of UAVs often include scenes with low visibility such as night, foggy days, and rainy days. At the same time, due to the flight jitter and sensor problems of the UAV, the captured pictures will inevitably be affected by blur and noise. In such scenarios, the performance of target detection and tracking algorithms based on conventional imaging systems will usually be greatly reduced, making it difficult to play a role.

2) Image data of different modalities, such as visible light images and infrared images, have their own characteristics, but relying solely on any one of the modal image samples for target detection has certain limitations, and cannot satisfy multi-scene, high-precision Performance object detection and tracking requirements.

Contents of the invention

An embodiment of the present invention provides a target recognition and tracking method on a complex battlefield, including:

Acquiring a first image set containing a target object, where the first images in the first image set are of different types;

performing deblurring and denoising processing on the first image set to form a second image set;

performing image fusion processing on the second image set to form a fusion image;

Constructing a target tracking algorithm model for identifying target objects in an image and performing tracking and marking, the target tracking algorithm model is based on a single-stage target detection model, at least by adjusting the feature extraction module and pooling in the single-stage target detection model The position of the layer and the number of convolutional layers are obtained;

The fused image is input into the target tracking algorithm model, so as to obtain the recognition and tracking results of the target object based on the target tracking algorithm model.

As an optional embodiment, also includes:

Performing image blur detection on the first image in the first image set, the image blur detection includes performing secondary blur processing on the first image, based on the comparison result of the first image before and after the secondary blur processing A blurriness of the first image is determined.

As an optional embodiment, the performing deblurring processing on the first image set includes:

Combining feature pyramid network, double discriminator structure and image deblurring algorithm to form the first image processing network;

Perform deblurring processing on the first image based on the first image processing network.

As an optional embodiment, the performing denoising processing on the first image set includes:

Forming a second image processing network by introducing an asymmetric loss function and a noise estimation sub-network into the image denoising network;

Denoising the first image based on the second image processing network.

As an optional embodiment, also includes:

Construct a feature fusion attention network based on a feature attention module that can handle different image features and pixels unequally, a basic block structure composed of a local residual learning module and a feature attention module, and a feature fusion structure that can distinguish different feature weights in rich images , the feature fusion attention network is used to dehaze the image;

Dehaze processing is simultaneously performed on the first images in the first image set based on the feature fusion attention network to obtain the second image set.

As an optional embodiment, the first image set includes the first visible light image and the first infrared image obtained by shooting the same scene, and the formed second image set includes the deblurred and denoised second visible light image. an image and a second infrared image;

The performing image fusion processing on the second image set to form a fusion image includes:

performing geometric registration on the second visible light image and the second infrared image;

performing image fusion processing on the registered second visible light image and the second infrared image to form the fusion image.

As an optional embodiment, the construction of a target tracking algorithm model for identifying and tracking a target object in an image includes:

Constructing the target tracking algorithm model based on a basic convolutional layer, a downsampling layer, a residual module, a feature extraction module and a pooling layer, the basic convolutional layer is composed of a plurality of convolutional layers, a BN layer and an activation function layer;

Wherein, the target tracking algorithm model includes six convolution stages, the feature extraction module is located after the fourth and fifth convolution stages, the pooling layer is located after the sixth convolution stage, and the second The number of residual modules from the convolution stage to the sixth convolution stage is 1, 2, 4, 4, and 2, and the number of convolution channels is 16, 24, 48, 96, 128, and 160, respectively.

As an optional embodiment, the collection includes a first image set of the target object, including:

The first image set including the target object is collected based on a drone, a visible light imaging sensor, and an infrared sensor.

Another embodiment of the present invention also provides a target recognition and tracking device on a complex battlefield, including:

An acquisition module, configured to acquire a first image set containing a target object, where the first images in the first image set are of different types;

A first processing module, configured to perform deblurring and denoising processing on the first image set to form a second image set;

A second processing module, configured to perform image fusion processing on the second image set to form a fusion image;

The first building block is used to construct a target tracking algorithm model for identifying a target object in an image and performing tracking and marking, the target tracking algorithm model is based on a single-stage target detection model, at least by adjusting the single-stage target detection The position of the feature extraction module and pooling layer in the model and the number of convolutional layers are obtained;

The input module is used to input the fused image into the target tracking algorithm model, so as to obtain the recognition and tracking results of the target object based on the target tracking algorithm model.

Another embodiment of the present invention also provides a target recognition and tracking system on a complex battlefield, including:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the target recognition and tracking method on a complex battlefield as described in any one of the above.

Additional features and advantages of the application will be set forth in the description which follows, and, in part, will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

The technical solutions of the present application will be described in further detail below with reference to the drawings and embodiments.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the present application, and do not constitute a limitation to the present application. In the attached picture:

FIG. 1 is a flowchart of a target recognition and tracking method on a complex battlefield in an embodiment of the present invention.

Figure 2 shows the overall composition of the system.

FIG. 3 is an algorithm block diagram of a target recognition and tracking method on a complex battlefield in an embodiment of the present invention.

Figure 4 is the algorithm flow chart of image fusion.

Figure 5 is a flowchart of the target tracking algorithm based on YOLOv3+deepsort

FIG. 6 is a structural block diagram of a target recognition and tracking method on a complex battlefield in an embodiment of the present invention.

Detailed ways

Below, specific embodiments of the present invention will be described in detail in conjunction with the accompanying drawings, but they are not intended to limit the present invention.

It should be understood that various modifications may be made to the embodiments applied for herein. Accordingly, the following description should not be viewed as limiting, but only as exemplifications of embodiments. Those skilled in the art will envision other modifications within the scope and spirit of the application.

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the general description of the application given above and the detailed description of the embodiments given below, serve to explain the embodiments of the application. principle.

These and other characteristics of the invention will become apparent from the following description of preferred forms of embodiment given as non-limiting examples with reference to the accompanying drawings.

It should also be understood that while the invention has been described with reference to a few specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and thus lie within the scope of the present invention. within the limited scope of protection.

The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

Specific embodiments of the present application are hereinafter described with reference to the accompanying drawings; however, it should be understood that the applied embodiments are merely examples of the present application, which can be implemented in various ways. Well-known and/or repetitive functions and constructions are not described in detail to avoid obscuring the application with unnecessary or redundant detail. Therefore, specific structural and functional details disclosed herein are not intended to be limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present invention in virtually any suitable detailed structure. Apply.

This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may refer to the same or one or more of the different embodiments.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

As shown in Figure 1, an embodiment of the present invention provides a target recognition and tracking method on a complex battlefield, including:

Acquiring a first image set containing the target object, where the types of the first images in the first image set are different;

performing deblurring and denoising processing on the first image set to form a second image set;

performing image fusion processing on the second image set to form a fusion image;

Construct a target tracking algorithm model for identifying and tracking the target object in the image. The target tracking algorithm model is based on the single-stage target detection model, at least by adjusting the position of the feature extraction module and the pooling layer in the single-stage target detection model And the number of convolutional layers is obtained;

The fused image is input into the target tracking algorithm model to obtain the recognition and tracking results of the target object based on the target tracking algorithm model.

The above method in this embodiment can be applied to a typical battlefield target detection tracking and control system. The assembly of the control system is shown in Figure 2. It mainly includes 6 parts from the business level: data layer, algorithm model layer, offline platform, online platform, data management system and target detection analysis and display system. The specific presentation system platform consists of three parts: (1) Deep learning training platform, which mainly carries out data labeling, preprocessing and model construction and training; (2) UAV real-time monitoring system, which mainly completes the real-time display and monitoring of UAV flight parameters. UAV attitude control; (3) target comprehensive analysis and display system, which mainly carries out algorithm verification analysis and data management functions, including local data storage module, database, data storage management system, result display system and target detection analysis and display system.

Further, the algorithm framework of the above method in this embodiment can be referred to as shown in Figure 3, wherein the image preprocessing algorithm model mainly includes an image blur degree detection model and an image optimization preprocessing model, wherein the image optimization preprocessing model includes an image deblurring algorithm model, an image denoising algorithm model and an image dehazing algorithm model (namely the first image processing network, the second image processing network and the feature fusion attention network described below respectively). The preprocessed image (that is, the second image set) is used for target detection and tracking, including processing through a single target detection algorithm and a multi-modal fusion target detection method, thereby realizing the YOLO+deepsort (described below) without Human-machine typical target tracking.

Specifically, the method in this embodiment also includes:

Performing image blur detection on the first image in the first image set, the image blur detection includes performing secondary blur processing on the first image, and determining the blur of the first image based on the comparison result of the first image before and after the secondary blur processing Spend.

For example, there are many types of typical targets collected by drones and are random, so there is usually no reference image. The blur detection proposed in this embodiment is used to evaluate the quality of the collected images without reference images. Since the sharpness of an image is an important index to measure the quality of an image, it can better correspond to people's subjective feelings, and the low sharpness of an image shows that the image is blurred. Therefore, this embodiment is implemented by applying an image blur degree detection algorithm of image secondary blur for image quality evaluation without a reference image.

Specifically, perform secondary blurring processing on the blurred image (that is, the first image in the first image set), and compare whether the difference between the secondary blurring result of the image and the original blurred image is significant. If the original image is blurred, the image after the secondary blurring , not much different from the original. If the original image is clear, perform a secondary blurring process on it. The difference between the image before and after blurring is large, indicating that the original image is clear. For image evaluation before and after blurring, this embodiment preferably adopts gray level variance product method (SMD2) to process, and this algorithm can have better performance in terms of accuracy and calculation time. The image secondary blurring method can give the blurring degree of the image without a clear picture as a reference, that is, evaluate the blurring degree of the image, and realize the blurring degree detection without a reference clear image.

When deblurring the first set of images, include:

Combining feature pyramid network, double discriminator structure and image deblurring algorithm to form the first image processing network;

Deblurring the first image is performed based on the first image processing network.

For example, when it is determined that the first image is blurred after blur detection, and deblurring processing is required, it is necessary to perform deblurring processing based on the first image processing network. The first image processing network in this embodiment uses the DeblurGANv2 algorithm. In this algorithm On the basis of the basic framework, a Feature Pyramid Network (Feature Pyramid Networks, FPN) structure is constructed for feature fusion, so as to facilitate the convergence of multi-scale features and achieve a balance between accuracy and efficiency. In order to consider processing larger and more complex real blurred regions and improve processing efficiency, the first image processing network in this embodiment also needs to adopt the following network structure:

(1) Taking Inception-ResNet as the basic network skeleton to form the FPN-Inception-ResNet-v2 network;

(2) With MobileNet as the basic network framework, the FPN-MobileNet network is formed, which achieves 10 to 100 times inference speed improvement and excellent performance, meeting real-time requirements.

In terms of discriminators, this embodiment adopts a dual discriminator structure, that is, discriminators are measured at two scales, global and local; during network training, network convergence can be achieved by optimizing the loss function of the above-mentioned network model, thereby Obtain a trained network model, that is, the first image processing network.

Further, when performing denoising processing on the first image set, including:

Forming a second image processing network by introducing an asymmetric loss function and a noise estimation sub-network into the image denoising network;

Denoising processing is performed on the first image based on the second image processing network.

For example, in this embodiment, the deep learning CBDNet model is preferably used to construct the second image processing network. This algorithm has the effect of eliminating real noise, and the consequences of image blurring are smaller, and at the same time, it can meet the real-time requirements. In addition, the denoising ability can be better improved by introducing the noise estimation sub-network into the model. At the same time, by introducing an asymmetric loss function in the model, the generalization ability of the model to real noise can be improved, and the noise intensity level can be reasonably adjusted to realize interactive denoising.

Preferably, the method in this embodiment also includes:

Construct a feature fusion attention network based on a feature attention module that can handle different image features and pixels unequally, a basic block structure composed of a local residual learning module and a feature attention module, and a feature fusion structure that can distinguish different feature weights in rich images , the feature fusion attention network is used to dehaze the image;

The first image in the first image set is dehazed simultaneously based on the feature fusion attention network to obtain the second image set.

Specifically, this embodiment preferably uses an end-to-end feature fusion attention network (FFA-Net) to directly restore the haze-free image. The FFA-Net architecture consists of the following three main parts: a. A new feature attention (FA) module is adopted, which combines channel attention with pixel attention mechanism, considering that the weighted information contained in different channel features is completely different, and the haze distribution on different image pixels is uneven, the FA in this embodiment treats different features and pixels unequally, which provides additional flexibility for processing different types of information, while improving the The expressive power of CNNs; b. The basic block structure consists of local residual learning and feature attention modules, the local residual learning module allows less important information, such as thin haze regions or low-frequency regions to be bypassed by multiple local residual connections , making the main network architecture focus on more effective information. c. Based on the feature fusion (FFA) structure of different levels of attention modules, feature weights are adaptively learned from feature attention (FA) modules, giving more weight to important features. This kind of structure can also preserve the information of the shallow layer and pass it to the deep layer. After the feature fusion attention network is prepared, it can process the first image set to achieve image dehazing to assist in obtaining a clear second image set.

In this embodiment, in order to improve the detection accuracy of the target detection algorithm in complex battlefield scenes, this embodiment collects visible light images and infrared images and fuses them, and complements the information of various types of images to improve the detection performance of complex targets the goal of. Specifically, the first image set in this embodiment includes the first visible light image and the first infrared image obtained by shooting the same scene, and the formed second image set includes the deblurred and denoised second visible light image and the first infrared image. Two infrared images: when the first image set containing the target object is collected, it is collected based on the UAV, the visible light imaging sensor and the infrared sensor.

Further, image fusion processing is performed on the second image set to form a fusion image, including:

performing geometric registration on the second visible light image and the second infrared image;

Perform image fusion processing on the registered second visible light image and the second infrared image to form a fusion image.

The specific processing process can be referred to as shown in Figure 4. Since the images of the same object obtained by different sensors and different imaging modes will have relative translation, rotation, different scaling and distortion, etc., when using visible light images and infrared images Geometrically strict registration must be guaranteed prior to fusion. For image fusion, it is a process of superimposing the information of multiple source images to obtain a fusion image, and then analyzing and processing the fusion image. According to the characteristics and abstraction of fusion processing, image fusion methods are divided into three categories, including: decision-level fusion, feature-level fusion and pixel-level fusion. These three image fusion methods can be flexibly selected and used according to the actual situation. only.

1) Decision-level fusion is a fusion method based on cognitive models. Based on feature extraction, the feature information of the image is identified and judged respectively, and the respective decisions are obtained, and then merged into a global image according to actual needs. optimal joint decision-making.

2) Feature-level fusion refers to extracting the target features of the region of interest in the source image, and performing comprehensive analysis and fusion processing on these feature information. In the process of feature extraction, only important information is retained, and unimportant information and redundant information are usually removed by means of dimensionality reduction. This fusion method compresses the information of the source image, and the calculation speed is significantly improved.

3) Pixel-level fusion refers to the process of selecting a fusion strategy to process the pixels of the strictly registered source image to obtain a fused image, such as using algorithms based on pyramid transformation or wavelet transformation. Pixel-level fusion can preserve the original information of the target image to the greatest extent, and has greater application value. It is possible to retain as much available information in the visible light image and infrared image as possible, so that the two information can effectively complement each other, and achieve the purpose of improving the accuracy of target detection. Through a specific algorithm, the visible light image and the infrared image are combined into a fusion image, and then Use the object detection model for detection.

Further, construct a target tracking algorithm model for identifying and tracking target objects in images, including:

The target tracking algorithm model is constructed based on the basic convolutional layer, downsampling layer, residual module, feature extraction module and pooling layer. The basic convolutional layer is composed of multiple convolutional layers, BN layers and activation function layers;

Among them, the target tracking algorithm model includes six convolution stages, the feature extraction module is located after the fourth and fifth convolution stages, the pooling layer is located after the sixth convolution stage, and the second convolution stage to The number of residual modules in the sixth convolution stage is 1, 2, 4, 4, and 2, and the number of convolution channels is 16, 24, 48, 96, 128, and 160, respectively.

Specifically, in this embodiment, YOLOv3 (single-stage target monitoring model) is preferably used as the basic structural framework for construction, and the model mainly includes three parts: basic structure, functional module and loss function. On this basis, this embodiment optimizes the model for typical target detection scenarios of UAVs from these three aspects. For example, the basic structure is consistent with YOLOv3, including Backbone, Neck and Head. The basic components of Backbone include: basic convolutional layer (Basicconv), downsampling layer (Downsample), residual module (Res), RFB module (feature extraction module) and SPP layer (pooling layer). The basic convolutional layers in the network are composed of convolutional layers, BN layers and ReLU activation function layers. Furthermore, the model includes initialization convolution, and there are 6 convolution stages in total, after 5 downsampling operations, so the convolution step of the featuremap on the deepest network layer is 32. The RFB module is added at the end of the 4th and 5th convolution stages to enhance the receptive field of the convolution stage; the SPP module is added at the end of the 6th convolution stage to obtain more advanced semantic information and enhance generalization performance. In view of the fact that the number of target categories in the UAV detection scene is small and the target pose and scale transformation are small, this embodiment adjusts the number of convolution layers in each stage, and the second convolution stage to the sixth convolution stage The number of residual modules in the stage is adjusted from the original 1, 2, 8, 8, 4 to 1, 2, 4, 4, 2, that is, from Darknet-53 to Darknet-33. The number of convolution channels in the 6 stages is also Adjustments have been made from the original 32, 64, 128, 256, 512, and 1024 to 16, 24, 48, 96, 128, and 160.

The above-mentioned YOLOv3 algorithm proposed in this embodiment reduces the complexity of the network structure, reduces the demand for computing power, shortens the time of model training and compresses the size of the model, which is conducive to the deployment of the algorithm on the embedded platform of the UAV. The real-time requirements of the algorithm are met, and the performance is at least equal to that of YOLOv5m. According to the experimental results, based on the COCO data set, when the IOU threshold is 0.5, the average precision (mAP) of the YOLOv3 algorithm proposed in this embodiment is slightly higher than that of YOLOv5m.

Further, as shown in FIG. 4 and FIG. 5 , after the target tracking algorithm model is prepared, the fusion image can be processed by it to obtain the recognition and tracking results of the target object. Specifically, since this embodiment uses the YOLOv3 algorithm as the target detection algorithm model, it is possible to directly use the model trained on typical battlefield targets as a detector in the target tracking algorithm, so that the technical route for multi-target tracking of typical battlefield targets is finally realized It is YOLOv3+deepsort. The target tracking algorithm based on YOLOv3+deepsort in this embodiment is specifically shown in Figure 5, mainly including two parts: Kalman filter and Hungarian algorithm. For example, the DeepSORT algorithm adopts the current mainstream tracking idea: detection+track (that is, real-time detection + tracking prediction). The specific process is: YOLOv3 detector generates detection frame detections → predicts the tracker tracks of the current frame from the tracking results of the previous frame → uses the Hungarian algorithm (calculates the cost matrix through appearance information, Mahalanobis distance, or IOU) to combine new tracks and The detections are matched → the Kalman filter update is performed on the successfully matched detections and the corresponding tracks. In the above algorithm, the YOLOv3 algorithm is used as the detector, and the detection result is used as input: boundingbox, confidence, feature. Confidence is mainly used to screen a part of the detection frame; boundingbox and feature (ReID) are used for the subsequent match calculation with the tracker; when applied, the prediction module first uses the Kalman filter to predict the tracker tracks. This embodiment uses the uniform motion and linear observation model of the Kalman filter (meaning that there are only four quantities and the detector will be used for constant value initialization during initialization). Then use the update module, which includes matching, tracker update and feature set update. In the part of updating the module, the fundamental method is to use IOU to match the Hungarian algorithm.

Based on the disclosure content of the various embodiments described above, the beneficial effects of the present application include:

In terms of image restoration: (1) For highly non-uniform blurred images of targets collected by drones, especially images containing complex target motion, an image deblurring algorithm based on DeblurGANv2 is proposed, which makes full use of global and local features, and uses feature pyramids The network improves the generalization of image features, introduces a global residual structure to ensure the accuracy of image restoration, and uses MobileNet as the basic network structure to improve the reasoning speed and ensure real-time performance. (2) The traditional image denoising algorithm is relatively "rude". Although it has a denoising effect, it will filter out some image details, making the image more blurred and affecting the subsequent target detection effect. This embodiment proposes a typical target image denoising CBDNet model based on a deep neural network, which consists of a noise estimation sub-network and a non-blind denoising sub-network. At the same time, synthetic noise images and real noise images are used to train CBDNet. Artificially synthesizing noise images and real noise images to train the model improves the generalization ability of the model to real noise images.

Further, to meet the real-time requirements of the typical target detection of UAVs, this embodiment proposes the improved YOLOv3 as the basic structural framework. By modifying the basic structure of the YOLOv3 network, the number of target categories in the UAV detection scene is small and Requirements such as small attitude and scale transformation; combined with spatial pyramid pooling and expanded receptive field, to achieve higher-level semantic information and enhance generalization for typical target detection of UAVs. Multi-task loss function is used to realize target classification and localization. When building a model, it can be based on the target detection algorithm model of visible light. The improved YOLOv3 algorithm has been well verified in terms of model reasoning speed, accuracy, and recall rate, and it has greatly compressed the model and reduced the detection time.

In addition, in this embodiment, by applying the dual-band fusion target detection and recognition algorithm to the detection and recognition of typical targets on the battlefield, it is mainly used for the detection of typical targets of unmanned aerial vehicles, which can effectively improve the detection performance and robustness. Among them, the target detection and recognition algorithm of the dual-band fusion is obtained by improving the algorithm using YOLOv3 as the basic structural framework. Feature-level multi-source sensor image fusion is to extract the target features of the region of interest in multi-source images, and perform comprehensive analysis and fusion processing on these feature information. The feature-level fusion algorithm of visible light images and infrared images proposed in this embodiment only retains important information during the feature extraction process, and usually removes unimportant information and redundant information by means of dimensionality reduction, and combines visible light and infrared images. The important features extracted by infrared are fused, which lays the foundation for subsequent accurate target detection and tracking, and improves the accuracy of detection and tracking.

As shown in Figure 6, another embodiment of the present invention also provides a target recognition and tracking device 100 on a complex battlefield, including:

An acquisition module, configured to acquire a first image set containing a target object, where the first images in the first image set are of different types;

A first processing module, configured to perform deblurring and denoising processing on the first image set to form a second image set;

A second processing module, configured to perform image fusion processing on the second image set to form a fusion image;

The first building block is used to construct a target tracking algorithm model for identifying a target object in an image and performing tracking and marking, the target tracking algorithm model is based on a single-stage target detection model, at least by adjusting the single-stage target detection The position of the feature extraction module and pooling layer in the model and the number of convolutional layers are obtained;

The input module is used to input the fused image into the target tracking algorithm model, so as to obtain the recognition and tracking results of the target object based on the target tracking algorithm model.

As an optional embodiment, also includes:

A detection module, configured to perform image blur detection on the first image in the first image set, the image blur detection includes performing secondary blur processing on the first image, based on the first blur before and after the secondary blur processing The image comparison result determines the blurriness of the first image.

As an optional embodiment, the performing deblurring processing on the first image set includes:

Combining feature pyramid network, double discriminator structure and image deblurring algorithm to form the first image processing network;

Perform deblurring processing on the first image based on the first image processing network.

As an optional embodiment, the performing denoising processing on the first image set includes:

Forming a second image processing network by introducing an asymmetric loss function and a noise estimation sub-network into the image denoising network;

Denoising the first image based on the second image processing network.

As an optional embodiment, also includes:

The second building block is based on a feature attention module that can handle different image features and pixels unequally, a basic block structure composed of a local residual learning module and a feature attention module, and feature fusion that can distinguish different feature weights in rich images Structural construction feature fusion attention network, described feature fusion attention network is used for dehazing processing to image;

The third processing module is to simultaneously perform defogging processing on the first images in the first image set based on the feature fusion attention network, so as to obtain the second image set.

As an optional embodiment, the first image set includes the first visible light image and the first infrared image obtained by shooting the same scene, and the formed second image set includes the deblurred and denoised second visible light image. an image and a second infrared image;

The performing image fusion processing on the second image set to form a fusion image includes:

performing geometric registration on the second visible light image and the second infrared image;

performing image fusion processing on the registered second visible light image and the second infrared image to form the fusion image.

As an optional embodiment, the construction of a target tracking algorithm model for identifying and tracking a target object in an image includes:

Constructing the target tracking algorithm model based on a basic convolutional layer, a downsampling layer, a residual module, a feature extraction module and a pooling layer, the basic convolutional layer is composed of a plurality of convolutional layers, a BN layer and an activation function layer;

Wherein, the target tracking algorithm model includes six convolution stages, the feature extraction module is located after the fourth and fifth convolution stages, the pooling layer is located after the sixth convolution stage, and the second The number of residual modules from the convolution stage to the sixth convolution stage is 1, 2, 4, 4, and 2, and the number of convolution channels is 16, 24, 48, 96, 128, and 160, respectively.

As an optional embodiment, the collection includes a first image set of the target object, including:

The first image set including the target object is collected based on a drone, a visible light imaging sensor, and an infrared sensor.

Furthermore, another embodiment of the present invention also provides a target recognition and tracking system on a complex battlefield, which is characterized in that it includes:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the target recognition and tracking method on a complex battlefield as described in any one of the above.

Furthermore, an embodiment of the present invention also provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned target recognition and tracking method on a complex battlefield is realized. It should be understood that each solution in this embodiment has the corresponding technical effects in the above method embodiments, and details are not repeated here.

Furthermore, an embodiment of the present invention also provides a computer program product, the computer program product is tangibly stored on a computer-readable medium and includes computer-readable instructions, and when executed, the computer-executable instructions cause at least A processor executes the target recognition and tracking method on a complex battlefield such as in the embodiments described above.

It should be noted that the computer storage medium in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or a combination of any of the above. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random-access storage media (RAM), read-only storage media (ROM), erasable Programmable read-only storage medium (EPROM or flash memory), optical fiber, portable compact disk read-only storage medium (CD-ROM), optical storage medium, magnetic storage medium, or any suitable combination of the above. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program configured to be used by or in conjunction with an instruction execution system, system, or device . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, antenna, optical cable, RF, etc., or any suitable combination of the foregoing.

In addition, those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a A system for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising a system of instructions, the The system implements the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims (10)

1. A target recognition and tracking method on a complex battlefield, characterized in that it comprises: Acquiring a first image set containing a target object, where the first images in the first image set are of different types; performing deblurring and denoising processing on the first image set to form a second image set; performing image fusion processing on the second image set to form a fusion image; Constructing a target tracking algorithm model for identifying target objects in an image and performing tracking and marking, the target tracking algorithm model is based on a single-stage target detection model, at least by adjusting the feature extraction module and pooling in the single-stage target detection model The position of the layer and the number of convolutional layers are obtained; The fused image is input into the target tracking algorithm model, so as to obtain the recognition and tracking results of the target object based on the target tracking algorithm model. 2. The target recognition and tracking method on the complex battlefield according to claim 1, is characterized in that, also comprises: Performing image blur detection on the first image in the first image set, the image blur detection includes performing secondary blur processing on the first image, based on the comparison result of the first image before and after the secondary blur processing A blurriness of the first image is determined. 3. The target recognition and tracking method on the complex battlefield according to claim 1, wherein said performing deblurring processing on said first set of images comprises: Combining feature pyramid network, double discriminator structure and image deblurring algorithm to form the first image processing network; Perform deblurring processing on the first image based on the first image processing network. 4. The target recognition and tracking method on the complex battlefield according to claim 1, wherein the denoising processing of the first set of images comprises: Forming a second image processing network by introducing an asymmetric loss function and a noise estimation sub-network into the image denoising network; Denoising the first image based on the second image processing network. 5. The target recognition and tracking method on the complex battlefield according to claim 1, is characterized in that, also comprises: Construct a feature fusion attention network based on a feature attention module that can handle different image features and pixels unequally, a basic block structure composed of a local residual learning module and a feature attention module, and a feature fusion structure that can distinguish different feature weights in rich images , the feature fusion attention network is used to dehaze the image; Dehaze processing is simultaneously performed on the first images in the first image set based on the feature fusion attention network to obtain the second image set. 6. The target recognition and tracking method on a complex battlefield according to claim 1, wherein the first image set includes the first visible light image and the first infrared image obtained by photographing the same scene, and the formed The second image set includes a second visible light image and a second infrared image after deblurring and denoising; The performing image fusion processing on the second image set to form a fusion image includes: performing geometric registration on the second visible light image and the second infrared image; performing image fusion processing on the registered second visible light image and the second infrared image to form the fusion image. 7. the target recognition and tracking method on the complex battlefield according to claim 1, is characterized in that, described construction is used to identify the target object in the image and carries out the target tracking algorithm model of tracking mark, comprises: Constructing the target tracking algorithm model based on a basic convolutional layer, a downsampling layer, a residual module, a feature extraction module and a pooling layer, the basic convolutional layer is composed of a plurality of convolutional layers, a BN layer and an activation function layer; Wherein, the target tracking algorithm model includes six convolution stages, the feature extraction module is located after the fourth and fifth convolution stages, the pooling layer is located after the sixth convolution stage, and the second The number of residual modules from the convolution stage to the sixth convolution stage is 1, 2, 4, 4, and 2, and the number of convolution channels is 16, 24, 48, 96, 128, and 160, respectively. 8. The target recognition and tracking method on the complex battlefield according to claim 1, wherein the collection includes the first image set of the target object, comprising: The first image set including the target object is collected based on a drone, a visible light imaging sensor, and an infrared sensor. 9. A target recognition and tracking device on a complex battlefield, characterized in that it includes: An acquisition module, configured to acquire a first image set containing a target object, where the first images in the first image set are of different types; A first processing module, configured to perform deblurring and denoising processing on the first image set to form a second image set; A second processing module, configured to perform image fusion processing on the second image set to form a fusion image; The first building block is used to construct a target tracking algorithm model for identifying a target object in an image and performing tracking and marking, the target tracking algorithm model is based on a single-stage target detection model, at least by adjusting the single-stage target detection The position of the feature extraction module and pooling layer in the model and the number of convolutional layers are obtained; The input module is used to input the fused image into the target tracking algorithm model, so as to obtain the recognition and tracking results of the target object based on the target tracking algorithm model. 10. A target recognition and tracking system on a complex battlefield, characterized in that it includes: at least one processor; and, a memory communicatively coupled to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to realize target recognition and tracking on a complex battlefield according to any one of claims 1-8 method.

Images