CN118262299A - Small ship detection method and system based on novel neck network and loss function - Google Patents

Abstract

The invention discloses a small ship detection method and a system based on a novel neck network and a loss function, belonging to the technical field of SAR image small target detection, and comprising the following steps: the neck network of the YOLOX model is improved, information from different levels is collected and fused through the SAM module and then distributed to different levels, the information fusion capacity of the neck network is enhanced, meanwhile, the positioning loss of a loss function of the YOLOX model is improved, and a gain function is added, so that a supervision signal of a small ship is more sufficient; performing target detection on the small ship in the SAR image according to the improved YOLOX model; the newly designed neck network realizes efficient information exchange and fusion, improves the fusion capacity of the neck of the original network, and is beneficial to detecting small targets. The loss function of the new design improves the optimization effect of the small ship detection algorithm by adaptively adjusting the positioning loss gain.

Description

Small ship detection method and system based on new neck network and loss function

Technical Field

The present invention relates to the technical field of small target detection in SAR images, and in particular to a small ship detection method and system based on a novel neck network and a loss function.

Background technique

SAR has the advantage of being available all day and all weather. SAR ship target detection is widely used in both military and civilian fields. The traditional detection method is the constant false alarm rate (CFAR) detection method, which cannot adapt to complex background environments. In recent years, with the development of deep learning, many researchers have adopted detection methods based on deep learning and achieved very good detection results.

Object detection methods based on deep learning can be divided into single-stage and two-stage. Single-stage methods can directly predict bounding boxes based on positioning boxes. They are fast but not accurate enough. Two-stage methods first generate a large number of region proposal boxes based on the features of CNN (convolutional neural network), and then use another network to identify the proposal boxes. They are very accurate but slightly slower. Although the above methods are very advanced, they cannot adapt to the special scenario of small target detection. This situation is even worse in SAR ship detection because there are a large number of small ships in SAR images.

Small-sized objects refer to those with fewer pixels. Taking the division criteria of the COCO dataset as an example, the area of a small object is less than 1024, the area of a medium object is between 1024-9216, and the area of a large object is greater than 9216. This means that the visual features of small objects are limited, so for deep learning-based detection algorithms, the detection performance of small objects is far inferior to that of large objects.

Compared with the datasets in computer vision, the proportion of small objects in the SAR ship detection dataset is extremely high. For example, the proportions of small objects, medium objects, and large objects in the COCO dataset are 41.4%, 34.3%, and 24.2%, respectively. But in the SAR-Ship-Dataset dataset, the proportions of small, medium, and large objects are 56.5%, 43.2%, and 0.3%, respectively. Therefore, due to the large proportion of small ships in the SAR ship detection dataset, the overall performance of the detection algorithm will drop significantly.

The reason for the poor performance of small object detection is that after CNN, the features of small objects are often contaminated by background noise, making it difficult to capture the discriminative information that is critical for classification, and the small position deviation of the bounding box in the IoU (intersection over union) metric interferes with small objects more than large objects, making it difficult to find a suitable IoU threshold and provide high-quality positive and negative samples to train the network. In order to solve the above problems, commonly used methods include anchor box design, small target data expansion, loss function design, etc.

Summary of the invention

In order to solve the above problems, the purpose of the present invention is to improve the detection performance of small ships by adopting multiple improvements (including neck and loss function) based on YOLOX.

In order to achieve the above technical objectives, the present application provides a small ship detection method based on a novel neck network and a loss function, comprising the following steps:

The neck network of the YOLOX model is improved. The SAM module collects and fuses information from different levels and distributes it to different levels to enhance the information fusion capability of the neck network.

at the same time,

The positioning loss of the loss function of the YOLOX model is improved, and the supervision signal of small ships will be more sufficient by adding the gain function;

Based on the improved YOLOX model, small ships in SAR images are detected.

Preferably, in the process of improving the neck network, local features and global features are obtained through the SAM module, two different convolutions are used for calculation in the global features, average pooling or bilinear interpolation is used to scale the global features, and at the end of each attention fusion, RepConv is added to further extract and fuse information.

Preferably, when the SAM module is used to improve the neck network, the SAM module consists of Conv convolution, Sigmoid activation function, Up Sampling, Down Sampling and RepConv units.

Preferably, in the process of acquiring the global features, the global features are acquired by fusing the local features, wherein the fusion of the local features is used to obtain high-resolution features that retain the information of the small ship.

Preferably, in the process of downsampling the input features, an average pooling operation is used to downsample the input features.

Preferably, in the process of improving the neck network of the YOLOX model, based on the SAM module, the features at different levels are globally fused to obtain global information, and the global information is injected into the features at different levels, so as to improve the information fusion capability of the neck network without significantly increasing the delay.

Preferably, in the process of improving the positioning loss, the gain function f(t) is expressed as:

Where t represents the area of the ground truth bounding box, τ represents the ratio of small ship loss to total loss, w represents the threshold, _wi represents, and _hi represents.

The present invention discloses a small ship detection system based on a novel neck network and a loss function, comprising:

Data acquisition module, used to acquire SAR images;

The detection and recognition module is used to detect small ships in SAR images through the improved YOLOX model. The improvement process of the YOLOX model includes: improving the neck network of the YOLOX model, collecting and fusing information from different levels through the SAM module and distributing it to different levels to enhance the information fusion capability of the neck network; improving the positioning loss of the loss function of the YOLOX model, and increasing the gain function so that the supervision signal of small ships will be more sufficient.

Preferably, the detection and recognition module is also used to obtain local features and global features through a SAM module, use two different convolutions to calculate the global features, use average pooling or bilinear interpolation to scale the global features, and add RepConv at the end of each attention fusion to further extract and fuse information.

Preferably, the detection and recognition module is also used to obtain global information by globally fusing features at different levels based on the SAM module, and inject the global information into features at different levels, so as to improve the information fusion capability of the neck network without significantly increasing the delay.

The present invention discloses the following technical effects:

The global information of the neck network newly designed in the present invention is obtained by globally integrating features at different levels and injected into features at different levels, achieving efficient information exchange and fusion. It avoids the information loss problem in FPN (Feature Pyramid Network), improves the fusion ability of the neck, and helps detect small targets. The newly designed loss function enhances the monitoring signal of small ships by adaptively adjusting the positioning loss gain, thereby improving the optimization effect of the small ship detection algorithm.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic diagram of the FPN and scale attention module of the present invention;

Fig. 2 is a schematic diagram of a SAM module according to the present invention;

FIG3 is a schematic diagram of a global functional module according to the present invention;

FIG. 4 is a schematic flow chart of the method described in the present invention.

Detailed ways

To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

As shown in Figures 1-4, the present invention discloses a method for detecting small-sized ship targets in SAR images based on a new neck network and loss function. The neck network globally integrates multiple layers of features and injects global and local information into different levels. It avoids the information loss inherent in the traditional feature pyramid network and enhances the information fusion capability without significantly increasing the delay. During the training process, the smaller the positioning loss of small ships, the greater the positioning loss gain. In this way, the supervision of small ships is strengthened in the loss function, which can make the calculation of the loss more biased towards small ships. A series of experiments were carried out on the SSDD and SAR-Ship-Dataset datasets, and the results showed that this method can greatly improve the detection performance of small ships.

Small-sized ship targets account for a very large proportion in SAR images, which poses great difficulties for detection. Existing research on small ship detection is rare, and the solutions are all based on feature fusion (feature map fusion at different levels and FPN, etc.). There is a lack of in-depth discussion on the proportion of small ships in the dataset, the reasons for the detection difficulties and solutions. This leads to limited improvement in the detection of small ships. Therefore, it is necessary to adopt special strategies to improve the detection results of small ships, which is the main motivation of this invention. The proposed method is based on YOLOX, because YOLOX is anchor-free and trained from scratch, and has good performance in SAR ship detection. On this basis, the neck network and loss function are newly designed. For the neck, the scale attention module is used together with FPN to fuse and interact information of different scales. It globally integrates multiple layers of features and injects global information into higher levels, which significantly enhances the information fusion ability of the neck without significantly increasing the delay and improves the performance of different object sizes. For the loss function, the supervision of small ships is strengthened during the training process, which makes the calculation of the loss more biased towards small ships.

Deep CNN generates feature maps with different spatial resolutions, where low-level features describe finer details and have more localization information, while high-level features capture richer semantic information. Due to downsampling, the response of small ships may disappear in deeper layers. To alleviate this problem, many researchers build feature pyramids by reusing multi-scale feature maps generated by forward propagation, and then use low-resolution feature maps to detect small ships, such as FPN and its variants.

FPN and its variants can only fully integrate features from adjacent layers and can only indirectly "recursively" obtain information from other layers. It can only exchange information selected by the intermediate layers, and the unselected information is discarded during the transmission process. This will cause the information of one layer to only affect the adjacent layers, and have limited impact on other global layers. Therefore, the overall effectiveness of information fusion is limited.

In order to avoid the information loss of the traditional FPN structure during transmission, the original recursive method is abandoned and a new fusion mechanism is constructed. By using a unified module to collect and fuse information from different levels and then distribute it to different levels, it not only avoids the inherent information loss of the traditional FPN, but also enhances the information fusion capability of the neck without significantly increasing the delay.

The input of the neck consists of feature maps B2, B3, B4 and B5 extracted from the backbone, with sizes of 160 × 160, 80 × 80, 40 × 40 and 20 × 20, respectively. The SAM (Scale Attention Module) module collects and aligns the features of each level and assigns them to different levels C3, C4, C5, as shown in Figure 1.

The structure of the SAM module is shown in Figure 2, where Local and Global represent local and global features respectively, Conv represents convolution, Sigmoid is the activation function, Up/Down Sampling represents upsampling and downsampling, and RepConv blocks represent RepConv units. The input is local features B2-B5 and global features. The global feature is generated by fusing the features of B2-B5. B2, B3, B4, and B5 are local features, and two different convolutions are used in the global feature for calculation. Due to the size difference between local and global features, average pooling or bilinear interpolation is used to scale the global features to ensure proper alignment. At the end of each attention fusion, RepConv is added to further extract and fuse information.

The B2, B3, B4, and B5 features are selected for fusion to obtain high-resolution features that retain the information of small ships. The input features are downsampled using an average pooling operation to achieve consistent sizes. Aligned features can be obtained by adjusting the feature size to the smallest feature size in the group. It can effectively aggregate information while minimizing the amount of computation.

Larger feature maps can retain more positioning information, but will increase computational complexity. Smaller feature maps will lose positioning information of smaller ships. Therefore, the present invention selects B4 as the size of feature alignment to achieve a balance between speed and accuracy.

After obtaining the fused global information, it is distributed to each level and injected using a simple attention operation to improve the detection ability of the branches.

By globally fusing features at different levels to obtain global information and injecting global information into features at different levels, efficient information exchange and fusion is achieved. Without significantly increasing latency, the SAM system significantly improves the information fusion capability of the neck and improves the model's ability to detect small ships.

During training, small ships contribute little to the total loss in most training iterations due to too few positive samples for them compared to large ships. Insufficient supervision signals for small ships lead to poor detection performance. Therefore, it is necessary to pay more attention to the loss of small ships. For object detection, the loss function usually consists of three parts: classification loss, confidence loss, and localization loss. The classification loss and confidence loss are independent of the ship size, while the localization loss is related to the ship size. In order to balance the loss distribution and alleviate the problem of insufficient supervision of small ships, a new localization loss function is proposed, which can supervise small ships more effectively and train the detection algorithm more evenly. The function before multiplying the designed loss function by f(t) the localization loss:

Loss _total = λ ₁ Loss _cls + λ ₂ Loss _obj + λ ₃ f(t)Loss _loc .

The left side of the equation is the total loss, and the three items on the right side are classification loss, likelihood loss, and positioning loss and their coefficients. In order to ensure that the loss function is more inclined to small ships, during the training iteration, the smaller the loss of small ships, the greater their loss gain f(t). Therefore, the formula of the gain function f(t) is as follows:

in,

t＝ _wi × _hi .

In the formula, t represents the area of the ground truth bounding box, τ represents the ratio of small ship losses to total losses, h represents, w represents, and π represents . The gain coefficient is It introduces τ as a feedback signal that is sensitive to the loss distribution. It adopts a nonlinear scale sensitivity coefficient to make it sensitive to scale changes. The smaller the ship size (t) and the smaller the ship loss (τ), the larger the gain coefficient.

In order to avoid the problem of negative gain, it is designed as a piecewise function. When t is less than the threshold ω, f(t) uses When it is greater than the threshold, the original scaled sensitivity gain factor (2-t) is used.

Through the above loss function, the supervision signal of small ships will be more sufficient, the training of the detection algorithm will be more balanced, and the accuracy will be steadily improved. This loss function can be easily integrated into any detection algorithm and imposes almost no additional burden during training and inference.

Verification of the technology proposed by the present invention:

The experiments were conducted on a Linux operating system using the Pytorch deep learning framework. The hardware was an Intel Xeon (Cascade Lake) Platinum 8269CY CPU @ 2.5 GHz and an NVIDIA GeForce TITAN V GPU (12 GB memory). The first 500 iterations were performed at one-third of the regular learning rate, and then the learning rate was gradually restored to 0.0025. The momentum of all models was set to 0.9 and the weight rate decay was set to 0.0025.

Comparison with state-of-the-art object detection algorithms on SSDD

In order to verify the detection performance of different detection algorithms for small, medium and large ships in SSDD, the following experiments are conducted in this section. 22 detection algorithms are selected in Table 1, and AP (average precision), APs (average precision of small-sized targets), APm (average precision of medium-sized targets), APl (average precision of large-sized targets) and FPS (frames per second) are used as evaluation indicators.

Table 1

YOLOX has significant advantages over other detection algorithms in terms of speed and accuracy, and it does not require anchor boxes and is trained from scratch. Therefore, the method proposed in this paper is based on YOLOX. It can be observed that the proposed detection algorithm increases the AP by 0.031. APs, APm, and APl are improved by 0.04, 0.018, and 0.046, respectively. The proposed method improves the performance of large, medium, and small ships, and the improvement for small ships is considerable. For a large number of small ships, improving AP is the best method for SAR ship detection.

Two phenomena can be observed from the table, the first is fluctuation, and the second is that many detection algorithms have lower AP for large ships compared to small and medium-sized ships. Intuitively, the AP should be much lower than APm or APl, but we find that sometimes the AP is higher for large ships and sometimes the AP is higher for small ships. By analyzing the dataset, we can find the reason behind this phenomenon: in SSDD, there are fewer large ships (62.8%, 34.6% and 2.6% for small, medium and large ships respectively). Missing or successfully detecting one during the test has a significant impact on the final AP, resulting in significant fluctuations. Many large ships have the abnormal phenomenon of low AP because there are fewer large ships in the dataset and the training frequency of large ships is low, resulting in poor detection performance.

Comparison with state-of-the-art object detection algorithms on SAR-Ship-Dataset:

In order to verify the detection performance of different detection algorithms for small, medium and large ships in the SAR ship dataset, the following experiments are conducted in this section. 14 detection algorithms are selected in Table 2, and AP, AP, APm, APl and FPS are used as evaluation indicators:

Table 2

From Table 2, we can also find that the single-stage detection algorithm has a higher AP but a lower speed. The anchor-free detection algorithm has a better trade-off between accuracy and latency. Compared with YOLOX, the proposed method improves the average AP, AP, APm, and APl by 0.037, 0.048, 0.029, and 0.034, respectively. Two phenomena similar to SSDD can be observed on the above data. Losing or successfully detecting one during the test has a significant impact on the final AP. Sometimes it is abnormal that APl is lower than AP and APm. The reason is that there are fewer large ships in the dataset and the large ships are trained less frequently, resulting in poor detection performance.

Evaluation of different components:

The improvements of each component on the SSDD and SAR ship datasets are shown in Tables 3 and 4. The proposed bottleneck improves the AP to 0.70 and 0.674, which proves the effectiveness of using global and local information when building FPN. The proposed loss function increases the AP to 0.717 and 0.695, indicating the importance of increasing the loss weight for small ship localization.

Table 3 Ablation results of the proposed detection algorithm on SSDD

Table 4 Ablation results of the proposed detection algorithm on the SAR ship dataset

From the above results, it can be found that the above experiment also proves that combining the above two strategies is effective.

Small object detection is challenging because small objects do not contain detailed information and may even disappear in deep networks. There are more small ships in the SAR ship detection dataset, so improving AP is a good choice to improve the overall performance. To the best of our knowledge, this paper is the first to systematically apply a small object detection strategy and achieve good performance. The neck effectively fuses features from different levels, and the loss function solves the problem of insufficient supervision information during training for small ships.

The present invention is described with reference to the flowchart and/or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (10)

1. The small ship detection method based on the novel neck network and the loss function is characterized by comprising the following steps of:

the neck network of YOLOX model is improved, the information from different levels is collected and fused through the SAM module and then distributed to different levels, the information fusion capability of the neck network is enhanced,

At the same time, the method comprises the steps of,

The positioning loss of the loss function of the YOLOX model is improved, and the supervision signal of the small ship is more sufficient by adding the gain function;

And detecting the small ship in the SAR image according to the improved YOLOX model.

2. A small ship detection method based on a novel neck network and a loss function according to claim 1, characterized in that:

In the process of improving the neck network, local features and global features are acquired through the SAM module, two different convolutions are used for calculation in the global features, an average pool or bilinear interpolation is used for scaling the global features, and RepConv is added at the end of each attention fusion to further extract and fuse information.

3. A small ship detection method based on a novel neck network and a loss function according to claim 2, characterized in that:

In the case of improving the neck network using the SAM module, the SAM module is composed of Conv convolution, sigmoid activation function, up Sampling Up, down Sampling Down and RepConv units.

4. A small ship detection method based on a novel neck network and a loss function according to claim 3, characterized in that:

And in the process of acquiring the global feature, acquiring the global feature by fusing the local features, wherein the fusion of the local features is used for acquiring the high-resolution feature for retaining the information of the small ship.

5. The small ship detection method based on the novel neck network and the loss function according to claim 4, wherein:

In downsampling the input features, the input features are downsampled using an average pooling operation.

6. The small ship detection method based on the novel neck network and the loss function according to claim 5, wherein:

in the process of improving the neck network of the YOLOX model, based on the SAM module, global information is obtained by globally fusing the features of different layers, and the global information is injected into the features of different layers, so that the information fusion capability of the neck network is improved under the condition of not obviously increasing delay.

7. The small ship detection method based on the novel neck network and the loss function according to claim 6, wherein:

In improving the positioning loss, the gain function f (t) is expressed as:

Where t represents the area of the ground truth bounding box, τ represents the ratio of small ship loss to total loss, w represents a threshold value, w _i represents h _i.

8. Small ship detection system based on novel neck network and loss function, characterized by comprising:

the data acquisition module is used for acquiring SAR images;

The detection and identification module is used for carrying out target detection on the small ship in the SAR image through an improved YOLOX model, wherein the improvement process of the YOLOX model comprises the following steps: the neck network of the YOLOX model is improved, information from different levels is collected and fused through a SAM module and then distributed to different levels, and the information fusion capability of the neck network is enhanced; the loss function of the YOLOX model is improved in positioning loss, and the supervision signal of the small ship is more sufficient by increasing the gain function.

9. The small vessel inspection system based on a novel neck network and loss function of claim 8, wherein:

The detection and identification module is further configured to obtain, through the SAM module, a local feature and a global feature, calculate the global feature using two different convolutions, scale the global feature using an average pool or bilinear interpolation, and add RepConv to further extract and fuse information at the end of each attention fusion.

10. A small vessel inspection system based on a new neck network and loss function according to claim 9, characterized in that:

The detection and identification module is also used for obtaining global information by globally fusing the features of different layers based on the SAM module, and injecting the global information into the features of different layers, so that the information fusion capability of the neck network is improved under the condition of not obviously increasing delay.