WO2025103079A1 - Method for fusing infrared light and visible light images - Google Patents

Abstract

The present invention relates to the technical field of image fusion. Disclosed is a method for fusing infrared light and visible light images. According to the present invention, design is carried out on the basis of a spatial channel attention mechanism and a gradient aggregation residual dense block, a strong texture and weak texture of a feature are retained by integrating Sobel and Laplacian operators while combining the advantages of ResNeXt and DenseNet, and channel information and spatial information of a feature map are refined by means of introducing a spatial and channel attention mechanism, thus the information capture capability of the feature map is increased, and refined spatial and channel feature maps are fused by using a pooled fusion block, to obtain a high-quality fusion feature.

Description

A method for fusion of infrared light and visible light images Technical Field

The present invention relates to the technical field of image fusion, and in particular to a method for fusing infrared light and visible light images.

Background Art

[Corrected 07.11.2024 in accordance with Rule 26]
Image fusion is an important image enhancement technology that can extract meaningful information from different source images and fuse them into new images. The fused images are usually robust, rich in information, and can show more complex and detailed scene expressions. It can not only reduce data redundancy, but also promote the development and decision-making of subsequent applications. Fusion has been widely used in the preprocessing modules of advanced visual tasks and has good application prospects in object detection, object tracking, semantic segmentation, etc.

Due to the practicality of infrared images and visible light images, many image fusion technologies have been born in recent years, which can be roughly divided into two categories: traditional methods and deep learning-based methods. According to different mathematical transformations, traditional methods can be further divided into multi-scale transformation-based methods, such as discrete wavelet transform (DWT), representation learning-based methods, such as sparse representation (SR) and joint sparse representation (JSR), subspace-based methods, saliency-based methods, and hybrid models. Deep learning-based methods can be divided into three categories according to the network architecture: models based on autoencoders (AE), models based on convolutional neural networks (CNN), and models based on generative adversarial networks (GAN).

Existing deep learning models are basically built on the basis of CNN or GAN networks. Although these methods have achieved good results in the field of image fusion, they all emphasize the quality of image fusion and ignore the needs of advanced visual tasks. In 2022, Tang et al. first proposed the image fusion framework SeAFusion combined with advanced visual tasks. They did not make great innovations in network architecture or learning paradigm, but they looked at the image fusion task from a new perspective, that is, using advanced visual tasks to drive image fusion. The emergence of SeAFusion has proposed new possibilities for image fusion.

In order to better promote the performance of infrared and visible light image fusion, the use of spatial and channel attention mechanisms can amplify useful information in the image and suppress the interference of harmful information by assigning different weights, while enriching the semantic information and spatial information of the image to promote the application of subsequent high-level visual tasks. The NestFuse network is a pioneering work in the field of image fusion. For multi-scale deep feature fusion, they proposed a fusion strategy based on spatial and channel attention models. In the spatial attention model, the deep features are calculated by l1 norm and soft-max operator to obtain enhanced deep features. In the channel attention model, the channel fusion features are obtained by global pooling and soft-max operator, and finally the spatial features and channel features are added to obtain the final fusion feature reference:

Disadvantages of existing technology:

1) Traditional methods use the same transformation to extract features from different source images. However, this operation does not take into account the feature differences of the source images, which may lead to poor expressiveness of the extracted features. On the other hand, the choice of fusion strategies and the complexity of human design in traditional methods limit the improvement of performance. These artificially designed fusion strategies are not only unable to learn, but also bring certain artifacts to the fusion results.

2) Before fusing infrared and visible light images, it is necessary to extract useful feature information from them, including texture, edges, etc. However, the existing network architecture cannot effectively extract coarse and fine granularity detail features and is easily disturbed by thermal radiation, thus affecting the quality of the fused image. How to effectively integrate coarse and fine texture features and suppress the interference of useless information still requires further breakthroughs.

3) Existing fusion methods tend to pursue better visual quality and higher evaluation indicators, and rarely systematically consider whether the fused image can promote high-level visual tasks. Although some studies have introduced perceptual loss at the feature level to constrain the fused image and the source image, the perceptual loss cannot effectively enhance the semantic information in the fused image. In addition, other researchers guide the image fusion process by segmentation masks, but the mask only segments some prominent objects, which is limited in enhancing semantic information.

Therefore, a new solution to the above problems needs to be proposed.

Summary of the invention

The purpose of the present invention is to provide a method for fusing infrared light and visible light images to solve the technical problems raised in the background technology.

To achieve the above object, the present invention provides the following technical solution: a method for fusing infrared light and visible light images, characterized in that it comprises at least the following steps:

S1: Build a fusion network, using CNN's end-to-end fusion network as the basic framework. The end-to-end CNN network has powerful feature extraction and learning capabilities;

S2: Build a gradient aggregation residual dense block, which combines the advantages of ResNext and DenseNet networks, retains shallow and deep features through split-conversion-merge and dense connection, and enhances the feature extraction ability of the network;

S3: Build a spatial channel attention module, which is used to amplify useful information, suppress the interference of useless information, and promote the development of semantic segmentation tasks to process the deep features of the input;

S4: Build a separation network to fully enhance the semantic information of the fused image.

Preferably, the application of the fusion network in S1 at least includes the following steps:

By sending the infrared light and visible light images to the feature extraction module respectively, the respective depth features are extracted;

The extracted deep features are then concatenated and sent to the spatial channel attention mechanism to further extract features and suppress the interference of useless information;

Finally, the fused image is generated through the feature reconstruction module.

Preferably, the branches in the gradient aggregated residual dense block in S2 perform a set of transformations, each transformation on a low-dimensional embedding, whose outputs are aggregated by summing.

Preferably, the gradient aggregated residual dense block in S2 sets the cardinality of the aggregated transformation to 2, and is composed of at least one residual block and one residual dense block, and each gradient aggregated residual dense block contains three branches to improve the diversity of extracted features so that it can make full use of the deep features extracted by each convolutional layer in the block;

Since the source image contains rich texture details, the gradient aggregation residual dense block also integrates the Laplacian operator and the Sobel operator to retain more coarse textures and fine textures in the image.

Preferably, the spatial channel attention module in S3 at least includes multi-scale feature extraction, spatial channel attention mechanism and pooling fusion block.

Preferably, the application of the spatial channel attention module in S3 at least comprises the following steps:

Four convolution blocks with different kernel sizes are used to capture multi-scale and multi-receptive field deep features, which are then fed into the spatial and channel attention mechanisms respectively;

Generate spatial and channel attention masks through self-attention functions to improve the network's information capture ability, thereby obtaining more accurate spatial and semantic information;

After the refinement of the attention branch, the channel feature map is upsampled to restore it to its original size. In addition, the present invention uses a convolution operation to control the number of channels of the spatial feature map;

The generated spatial and channel attention feature maps are fused using the pooling fusion block to generate a fused feature map that satisfies both the intra-class similarity and inter-class difference requirements.

Preferably, the application of the S4 separation network comprises at least the following steps:

A real-time segmentation model Bilateral attention decoder is introduced to segment the fused image, and the segmentation network outputs the segmentation result and auxiliary segmentation result;

The gap between the segmentation result and the semantic label reflects the richness of the semantic information contained in the fused image;

This gap is exploited to construct semantic loss;

Semantic loss is used to guide the training of the fusion network through back-propagation, forcing the fused image to contain more semantic information.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention is designed based on the spatial channel attention mechanism and the gradient aggregation residual dense block, combining the advantages of ResNeXt and DenseNet while integrating Sobel and Laplacian operators to retain the strong texture and weak texture of the features, and by introducing the spatial and channel attention mechanism, the channel information and spatial information of the feature map are refined to improve the information capture ability of the feature map, and a pooling fusion block is used to fuse the refined spatial and channel feature maps to obtain high-quality fused features; the experimental results of the present invention show that the proposed method performs better than the most advanced methods on the MSRS dataset, highlighting the potential research direction in the future to improve the accuracy of the fused image and promote the development of advanced visual tasks.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

Figure 1 is a spatial attention model under the prior art;

Figure 2 is a channel attention model under the prior art;

FIG3 is a schematic diagram of the overall network framework of SCGRFuse of the present invention;

FIG4 is a schematic diagram of a fusion network of the present invention;

FIG5 is a schematic diagram of a GRXDB module of the present invention;

FIG6 is a schematic diagram of a multi-scale spatial attention module of the present invention;

FIG7 is a schematic diagram showing a qualitative comparison of SCGRFuse of the present invention and nine most advanced methods on the MSRS dataset;

FIG8 is a schematic diagram of the process of generating channel attention and spatial attention masks according to the present invention;

FIG9 is a schematic diagram of the overall structure of the pooling fusion block of the present invention.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

Referring to FIG. 1 and FIG. 2 , it can be seen that the spatial attention model and the channel attention model under the prior art.

Embodiment 1:

This embodiment is used to disclose a method for fusing infrared and visible light images based on a spatial channel attention mechanism and a gradient aggregation residual dense block;

The design of SCGRFuse is a combination of image fusion and semantic segmentation tasks. It first uses the fusion network to obtain the fusion result of the input infrared and visible light images, and uses the content loss to guide the training of the entire fusion network. The fused image is then sent to the semantic segmentation network, and more semantic information is integrated into the fused image through the semantic loss generated by the segmentation network, so as to optimize the entire fused image. The overall architecture of the network is shown in Figure 3.

1) Fusion Network

The present invention uses an end-to-end fusion network based on CNN as the basic framework. The end-to-end CNN network has powerful feature extraction and learning capabilities. The infrared and visible light images are respectively sent to the feature extraction module to extract their respective deep features, and then the extracted deep features are spliced and sent to the spatial channel attention mechanism to further extract features and suppress the interference of useless information. Finally, the fused image is generated through the feature reconstruction module. Its network structure is shown in Figure 4.

2) Gradient Aggregation Residual Dense Block (GRXDB)

Gradient Aggregation Residual Dense Blocks (GRXDB) combines the advantages of ResNext and DenseNet networks to retain shallow and deep features through split-transform-merge and dense connections. The branches in the GRXDB module perform a set of transformations, each on a low-dimensional embedding, and its output is aggregated by summing. The present invention sets the cardinality of the aggregation transformation to 2, where the module of the present invention consists of a residual block and a residual dense block. More precisely, each GRXDB contains three branches to improve the diversity of extracted features, so that it can make full use of the deep features extracted by each convolutional layer in the block. Since the source image contains rich texture details, the Laplacian operator and the Sobel operator are also integrated in GRXDB to retain more coarse and fine textures in the image. GRXDB greatly enhances the feature extraction capability of the network, and its structure is shown in Figure 5.

3) Spatial Channel Attention Module (SCAM)

In order to amplify useful information, suppress the interference of useless information, and promote the development of semantic segmentation tasks, the present invention proposes to use a multi-scale spatial channel attention mechanism to process the input deep features. SCAM mainly consists of three parts: multi-scale feature extraction, spatial channel attention mechanism, and pooling fusion block. Since the same-scale convolution will hinder the extraction of multi-scale information in different features, before calculating the attention weight, in order to obtain more comprehensive source image representation information, the present invention uses four convolution blocks with different kernel sizes to capture multi-scale and multi-receptive field deep features, and then sends the captured deep features to the spatial and channel attention mechanisms respectively. In order to improve the information capture ability of the network and obtain more accurate spatial information and semantic information, the present invention generates spatial and channel attention masks through self-attention functions. After the refinement of the attention branch, the present invention uses an upsampling operation on the channel feature map to restore it to its original size. In addition, the present invention uses a convolution operation to control the number of channels of the spatial feature map. In order to generate a feature map that meets the requirements of both intra-class similarity and inter-class difference, the present invention uses a pooling fusion block (PFB) to fuse the generated spatial and channel attention feature maps, thereby generating a fused feature map that meets the requirements. The structure of the spatial channel attention module is shown in Figure 6.

4) Segmentation Network

In order to fully enhance the semantic information of the fused image, the present invention introduces a real-time segmentation model, Bilateral Attention Decoder, to segment the fused image. The segmentation network outputs the segmentation result and the auxiliary segmentation result. The gap between the segmentation result and the semantic label can reflect the richness of the semantic information contained in the fused image. Therefore, this gap can be used to construct the semantic loss. Finally, the semantic loss is used to guide the training of the fusion network through back propagation, forcing the fused image to contain more semantic information.

Embodiment 2:

This embodiment is used to further disclose an application of performance evaluation based on the above embodiment 1;

In order to comprehensively evaluate the proposed method, the present invention conducts extensive quantitative and qualitative evaluations on the MSRS dataset. In addition, the present invention is compared with nine state-of-the-art methods, including a traditional method, namely GTF, an ae-based method, namely DenseFuse, two GAN-based methods, namely FusionGAN, GANMcC, four CNN-based methods, namely IFCNN, SDNet, U2Fusion, SeAFusion, and an image decomposition-based method, namely DeFusion. The implementations of these nine methods are all public, and the parameters set by the present invention are also consistent with the original paper. Among them, DenseFuse and IFCNN use element-wise addition and element-wise maximum fusion strategies to fuse deep features, respectively.

In terms of quantitative evaluation, the present invention selects seven indicators, EN, MI, VIF, SF, SD, SCD, and Qabf, to objectively evaluate the fusion effect. In addition, the present invention also uses IoU to quantify the segmentation performance. The larger the value of these indicators, the better the fusion performance. The quantitative results of 7 statistical indicators for 361 pairs of images are shown in Table 1. The segmentation results are shown in Table 2.

Table 1 Quantitative comparison of 7 indicators for 361 pairs of images from the MSRS dataset

Method EN SD SF MI SCD. VIF Qabf GTF 5.47 19.56 8.48 1.67 0.76 0.51 0.40 DenseFuse 5.94 23.57 6.03 2.65 1.25 0.69 0.37 FusionGAN 5.44 17.07 4.42 1.87 0.98 0.44 0.14 IFCNN 6.28 31.50 10.76 2.82 1.38 0.77 0.60 GANMcC 5.91 22.84 4.92 2.53 1.24 0.59 0.25 SDNet 5.25 17.35 8.67 1.65 0.99 0.45 0.38 U2Fusion 5.56 27.71 9.24 1.96 1.26 0.55 0.42 DeFusion 6.46 37.63 8.60 2.16 1.35 0.77 0.54 SeAFusion 6.65 41.84 11.11 4.04 1.69 0.94 0.67 SCGRFuse(Ours) 6.68 42.76 11.28 5.08 1.67 1.04 0.69

As shown in Table 1. The present invention shows significant advantages in SD, SF, and MI. The higher the SD, the higher the contrast of the fused image. The higher the SF value, the clearer the fused image is and the better the quality of the fused image. The higher the MI value, the more information the fused image conveys. In addition, the present invention presents the best VIF, which shows that the fused image of the present invention is more in line with the human visual system. The method of the present invention also obtains the best Qabf, which means that the fused image retains more edge information. In addition, the present invention also presents the highest EN, which shows that the fused image contains the most information. The present invention follows SeaFusion only slightly in the SCD indicator. An example of the qualitative results of SCGRFuse on the MSRS dataset is shown in Figure 7. The present invention shows the fusion results of two scenes dominated by day and night, respectively, and uses a red frame to enlarge an area to illustrate the phenomenon that the texture details are polluted to different degrees by the spectrum. In addition, the present invention uses a green frame to highlight the problem of useless information weakening the prominent target. In daytime scenes, neither GTF nor FusionGAN can well preserve the texture details of visible light images, and other methods are inevitably interfered by useless information. For night scenes, it can be seen that all methods have integrated the complementary information in infrared and visible light images to some extent, but most of them have introduced useless information in the fused image, which is manifested in the weakening of salient targets and the pollution of texture background details. Only the present invention and SeAFusion can retain rich texture details and highlight targets, but the fused image of the present invention has higher contrast than SeAFusion.

Table 2 Segmentation performance of visible light, infrared and fused images on the MSRS dataset (miou).

The segmentation results are shown in Table 2. It can be seen that the present invention is basically in a leading position in various types of IoU and ranks first in MIoU. This is due to the two advantages of the present invention. On the one hand, the present invention effectively integrates the complementary information of infrared images and visible light images, which helps the segmentation model to fully understand the imaging scene. On the other hand, SCGRFuse improves the information capture capability under the action of spatial and channel attention mechanisms, and enhances the spatial information and semantic information under the guidance of semantic loss, so that the segmentation network can more accurately describe the imaging scene.

Embodiment three:

This embodiment is used to disclose the specific operation process under the above embodiment.

The purpose of image fusion is to make the fused image maintain the advantages of different images. However, the existing fusion methods are complex in design and ignore the impact of attention mechanism on deep features. To solve these problems, benfam proposed a method for infrared and visible light image fusion called SCGRFuse. First, we constructed a gradient aggregation residual dense block (GRXDB), which combines the advantages of ResNeXt and DenseNet while integrating Sobel and Laplacian operators to retain the strong and weak textures of the features. Then, we introduced the spatial and channel attention mechanism to refine the channel information and spatial information of the feature map to improve the information capture ability of the feature map, and used a pooling fusion block to fuse the refined spatial and channel feature maps to obtain high-quality fused features; the experimental results of this invention show that the proposed method performs better than the most advanced methods on the MSRS dataset, highlighting the potential research direction in the future to improve the accuracy of fused images while promoting the development of advanced visual tasks.

Among them, the gradient aggregation residual dense block (GRXDB) and the spatial channel attention module (SCAM) are the key technologies of the present invention.

1) Gradient Aggregation Residual Dense Block

Gradient aggregation residual dense block overall operation flow (as shown in Figure 5): The GRXDB of the present invention contains three branches. The main branch deploys three 3x3 convolution blocks and two 1x1 convolution blocks. In order to make full use of various convolution layers to extract features, the present invention introduces dense connections into the main branch, and uses 1x1 convolution blocks to deal with the differences between channel dimensions. In addition, the main branch also introduces the Laplacian operator to further extract the weak texture of the feature. In addition, in addition to two 3x3 convolution blocks and one 1x1 convolution block, the residual branch also integrates the Sobel operator to retain the strong texture of the feature. The third branch does not do any processing, which we call the residual branch to retain the information of the input feature. Finally, the main branch, residual branch and residual branch outputs are added by element addition to integrate the deep features. It is worth noting that the Relu activation function is equivalent to directly discarding negative activations. This approach may be effective for classification tasks, but it is not suitable for image fusion tasks. In order to better meet the needs of image fusion, the present invention sets the activation function of GRXDB to Leaky ReLU that can retain negative activation information.

2) Spatial channel attention module

The operation flow of the spatial channel attention module (as shown in Figure 6): The present invention uses four convolution blocks with different kernel sizes to capture multi-scale and multi-receptive field depth features. The sizes of the convolution blocks are 1x1, 3x3, 5X5, and 7x7 respectively. The activation function of the convolution block is still Leaky ReLU. Then the obtained deep features are spliced in the channel direction. Finally, the 1x1 convolution block is used to change the number of channels and send them to the spatial and channel attention layers. In the attention mechanism, generating attention masks Ms and Mc is the most important process. In order to improve the information capture ability of the network, more accurate spatial information and semantic information can be obtained. The present invention generates spatial and channel attention masks through self-attention functions, and its main process is shown in Figure 8. In order to generate the channel attention mask, first, the present invention uses global average pooling to average all pixels of each channel map, and then obtains a new 1*1 channel map. Then, the Tanh activation function and the 1x1 convolution block are used to adjust the value of the feature map. It is worth noting that the present invention divides the calculated value of the tanh activation function by 2 and adds 0.5 to enable it to be mapped to the range of [0,1]. Finally, the present invention multiplies the calculated channel map with the input feature map to generate the output of the channel attention branch. Similarly, in order to obtain the spatial attention mask, the present invention first uses a 1x1 convolution block to reduce the number of channels of the input feature, and then uses maximum pooling and average pooling to obtain two feature maps, both of which have only one channel and are the same size as the input feature. Next, the two feature maps are spliced together, and the number of channels is changed to 1 by using a 1x1 convolution block. The activation function of the convolution block is still tanh, which is consistent with the channel attention mechanism. Finally, the generated spatial attention mask is multiplied with the input feature map to obtain the output of the final spatial attention branch. After the refinement of the attention branch, the present invention uses an upsampling operation on the channel feature map to restore it to its original size, and uses a convolution operation to control the number of channels of the spatial feature map. In order to generate a feature map that meets both the intra-class similarity and inter-class difference requirements, the present invention uses a Pooling fusion block (PFB) to fuse the generated spatial and channel attention feature maps. The structure of PFB is shown in Figure 9. The present invention uses 3x3 average pooling to smooth the upsampled channel feature map, and uses reflection filling to construct the true boundary to reduce boundary artifacts. Finally, it is spliced with the spatial feature map, and the boundary is corrected using a 1x1 convolution block and a leaky ReLU activation function to generate a fused feature map that meets the requirements.

It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims (7)

A method for fusing infrared light and visible light images, characterized in that it comprises at least the following steps: S1: Build a fusion network, using CNN's end-to-end fusion network as the basic framework. The end-to-end CNN network has powerful feature extraction and learning capabilities; S2: Build a gradient aggregation residual dense block, which combines the advantages of ResNext and DenseNet networks, retains shallow and deep features through split-conversion-merge and dense connection, and enhances the feature extraction ability of the network; S3: Build a spatial channel attention module, which is used to amplify useful information, suppress the interference of useless information, and promote the development of semantic segmentation tasks to process the deep features of the input; S4: Build a separation network to fully enhance the semantic information of the fused image. According to the infrared and visible light image fusion method of claim 1, it is characterized in that the application of the fusion network in S1 at least comprises the following steps: By sending the infrared light and visible light images to the feature extraction module respectively, the respective depth features are extracted; The extracted deep features are then concatenated and sent to the spatial channel attention mechanism to further extract features and suppress the interference of useless information; Finally, the fused image is generated through the feature reconstruction module. According to the infrared and visible light image fusion method of claim 1, it is characterized in that: the branches in the gradient aggregation residual dense block in S2 perform a set of transformations, each transformation is on a low-dimensional embedding, and its output is aggregated by summing. According to the infrared and visible light image fusion method of claim 3, it is characterized in that: the gradient aggregation residual dense block in S2 sets the cardinality of the aggregation transformation to 2, and is composed of at least one residual block and one residual dense block, and each gradient aggregation residual dense block contains three branches to improve the diversity of extracted features, so that it can make full use of the deep features extracted by each convolutional layer in the block; Since the source image contains rich texture details, the gradient aggregation residual dense block also integrates the Laplacian operator and the Sobel operator to retain more coarse textures and fine textures in the image. According to the infrared and visible light image fusion method described in claim 1, it is characterized in that the spatial channel attention module in S3 at least includes multi-scale feature extraction, spatial channel attention mechanism and pooling fusion block. According to the infrared and visible light image fusion method of claim 5, it is characterized in that: the application of the spatial channel attention module in S3 at least comprises the following steps: Four convolution blocks with different kernel sizes are used to capture multi-scale and multi-receptive field deep features, which are then fed into the spatial and channel attention mechanisms respectively; Generate spatial and channel attention masks through self-attention functions to improve the network's information capture ability, thereby obtaining more accurate spatial and semantic information; After the refinement of the attention branch, the channel feature map is upsampled to restore it to its original size. In addition, the present invention uses a convolution operation to control the number of channels of the spatial feature map; The generated spatial and channel attention feature maps are fused using the pooling fusion block to generate a fused feature map that satisfies both the intra-class similarity and inter-class difference requirements. According to the infrared and visible light image fusion method of claim 1, the application of the S4 separation network at least comprises the following steps: A real-time segmentation model Bilateral attention decoder is introduced to segment the fused image, and the segmentation network outputs the segmentation result and auxiliary segmentation result; The gap between the segmentation result and the semantic label reflects the richness of the semantic information contained in the fused image; This gap is exploited to construct semantic loss; Semantic loss is used to guide the training of the fusion network through back-propagation, forcing the fused image to contain more semantic information.