CN115601661A - A building change detection method for urban dynamic monitoring - Google Patents

Abstract

The invention discloses a building change detection method for urban dynamic monitoring, which receives an urban surface double-time-phase image detected by a remote sensing satellite through a satellite technology, cuts an original image, inputs the cut image into an urban building automatic detection model, and outputs a change detection result of a building in the double-time-phase image; the automatic detection model of the urban building comprises an encoding stage and a decoding stage. In the encoding stage, a twin network shared by weight is adopted to carry out down-sampling operation on the input double-time phase image, rich multi-scale characteristic information is extracted, and meanwhile, the expression of the characteristic information is enhanced by utilizing a twin cross attention mechanism; in the decoding stage, a multi-scale feature fusion module is adopted to carry out progressive fusion on the extracted multi-scale features; and the detection result is pushed to be closer to the actual change condition by using the differential context discrimination module. The method can efficiently judge and fuse multiple features, thereby improving the accuracy of detecting the change of the urban buildings.

Description

A building change detection method for urban dynamic monitoring

technical field

The invention belongs to the field of urban dynamic monitoring, and more particularly relates to a building change detection method for urban dynamic monitoring.

Background technique

At present, most of the automatic monitoring systems of urban buildings need to lay large-scale detection equipment and cables around the city, and the power supply and maintenance of the equipment require high costs. If it is large, it will lead to problems such as false monitoring and missing monitoring in the detection system. Remote sensing technology can acquire the information of the earth's surface at fixed time intervals and extract the dynamic changes of the same surface in multiple time periods. The automatic detection model of urban buildings is based on remote sensing change detection technology. Its task is to observe the difference changes of the same target in different periods, and to classify each image pixel, that is, label 0 (unchanged) and label 1 (change). So far, researchers have done a lot of work on the theory and application of remote sensing change detection. These contributions are of great significance to land resource management, urban construction and planning, and illegal construction management.

Over the past few decades, many algorithms have been proposed for models of change detection in remote sensing imagery. These algorithms can be roughly divided into two categories: traditional methods and deep learning-based methods. For traditional methods, limited by the resolution of remote sensing images at the beginning, pixel-based methods are mostly used for change detection, and change vector analysis (change vector analysis, CVA) and principal component analysis (principal component analysis, PCA) are used to analyze each pixel The spectral features of the points are analyzed for change detection. With the rapid development of aerospace and remote sensing technology, the ability to obtain high-resolution remote sensing images has been enhanced. Scholars have introduced the concept of objects in the field of change detection, mainly using object-level spectral, texture and spatial background information for change detection. Although these methods were able to achieve good results at the time, the traditional methods required manual design of features and specified thresholds to ensure the final detection effect, and could only extract shallow features, which could not fully represent the changes of buildings in high-resolution remote sensing images. , so it is difficult to meet the accuracy requirements in reality.

On the other hand, with the development of computing power and the accumulation of massive data, the change detection algorithm based on deep learning has become the mainstream because of its powerful performance. At present, most of the deep learning-based change detection methods are developed from networks that have a good effect on contrastive learning and segmentation tasks. Some scholars use focused contrast loss for change detection, which reduces intra-class variance and increases inter-class differences, and finally obtains binarized detection results through threshold restrictions. The segmentation network uses the idea of image segmentation for change detection. Representative examples include U-shaped network (UNet), fully convolutional neural network (Fully Convolutional Networks, FCN) and DeepLab series networks.

Although these methods have also achieved high performance, there are still the following problems: First, when there are a large number of spurious changes in some front and back time-series images, the current attention mechanism cannot efficiently and targetedly focus on the no-change area and changing regions, which can lead to serious false detection phenomena. Secondly, there are a large number of down-sampling and up-sampling operations in the existing network, which leads to the loss of feature information in the front and rear time-series images, and the rough fusion strategy aggravates this problem, which makes the network unable to perform well in the final change detection. To restore the original features of the image, the final detection result will have problems such as missed detection and irregular edges. Finally, the current algorithm cannot perform differential processing on context information well, so the detection effect on urban building images with many spurious changes is not good.

Contents of the invention

Aiming at the defects and improvement needs of the prior art, the present invention provides a building change detection method for urban dynamic detection, which can accurately realize automatic detection of urban buildings. Including the following steps:

S1, using the images of urban buildings collected by remote sensing satellites as a data set, obtaining the actual change images corresponding to each building in the data set, and dividing the actual change images and the corresponding dual-temporal images into a training set and a test set;

S2. Build an automatic building detection model consisting of an encoder and a decoder. The encoder includes a weight-sharing dual-channel Siamese network and a Siamese cross-attention module. The decoder includes a multi-scale feature fusion and differential context discrimination module. ;

The weight-shared dual-channel Siamese network includes a batch normalization layer and multiple upsampling blocks, inputting bitemporal images for obtaining feature maps of different scales;

The twin cross-attention module first performs embedding operations on feature maps of different scales, and then uses the multi-head cross-attention mechanism to extract deeper semantic information of changing features to improve global attention to feature information;

The multi-scale feature fusion module adopts a double progressive fusion strategy of reconstruction and up-sampling blocks to fuse the extracted features containing rich multi-scale semantic information;

The input of the differential context discrimination module is the output image of the multi-scale fusion module and the front and rear temporal difference images. The purpose is to combine the context information in the image to improve the discrimination ability of the network, so that the detection result image is closer to the real change image, thereby Improve detection accuracy;

S3, using the training set in S1 to train the building automatic detection model in S2, and using the trained model to realize building change detection.

In some optional embodiments, step S1 includes:

Using artificially produced city building change images as a data set, the actual change images are produced according to the front and back time series images in the data set, where the actual change images are the changing areas in the front and back time series images, and each pixel in the front and rear time series images represents a Category (unchanged or changed).

The front and back time series images and their corresponding actual change images are composed of urban building automatic detection image data set, in which the training set and test set are divided according to the ratio of 8:2.

In some optional implementations, the encoder includes a weight-sharing dual-channel Siamese network and a Siamese cross-attention module, and the decoder includes a multi-scale feature fusion and differential context discrimination module.

In this implementation, the weight-sharing dual-channel Siamese network in the encoder is implemented using a multi-scale densely connected UNet, which contains skip connections and can fully extract low-level features and high-level features. The twin cross-attention module in the encoder is combined with the Transformer multi-head attention mechanism. The twin cross-attention module first independently embeds the bitemporal images to obtain the corresponding multi-stage embedded tokens. The feature information is further divided into query queue, query vector and query value through the multi-head attention mechanism, the Sigmoid function further activates the feature information concerned, and the multi-layer perceptron block effectively reduces the time complexity of the network, and finally makes the attention channels respectively Focus on the changed area and the unchanged area in the image, and divide the image information into sliding windows for self-attention calculation to improve the network's ability to model global information.

The multi-scale fusion module in the decoder uses multi-scale feature fusion technology to fuse the multi-stage embedded tokens extracted in the encoder with the channel attention output rich in context information, and then uses the upsampling operation to fuse the features, This enables the network to restore the original image information to the greatest extent and reduce the missed detection rate of the network. Second, the embedded tokens are fused with the rich context output of the channel transformer using multi-scale feature fusion technique. Then, the extracted multi-scale information content is up-sampled and fused to maximize the recovery of the original image information; the input of the differential context discrimination module in the decoder is the output image of the multi-scale fusion module in the decoder and the front and back timing The purpose of the difference image is to combine the context information in the image to improve the discriminative ability of the network, so that the detection result image is closer to the real change image, thereby improving the detection accuracy.

In some optional implementations, the dual-channel Siamese network with weight sharing in step S2 performs batch normalization operations on the input dual-temporal images, including convolution kernel 3, two-dimensional convolution with a step size of 1, two-dimensional Dimension BatchNorm and the ReLU activation function with 64 output channels, then extract feature information through 3 downsampling blocks, define x ^{i, j} as the output nodes of the downsampling block, and the objective function of the downsampling block is:

Among them, N(·) represents the nested convolution function, D(·) represents the downsampling layer, U(·) represents the upsampling layer, [] represents the feature connection function, x ^{i, j} represents the output feature map, i represents The number of layers, j represents the jth convolutional layer of the layer, and k represents the kth connection layer; finally, the Siamese network channel outputs four kinds of multi-scale feature information.

In some optional embodiments, the Siamese cross-attention module in step S2 performs an embedding operation on the four outputs of the dual-channel Siamese network. First, a 2D convolution is performed to extract features, and then the features are expanded into a two-dimensional sequence T ₁ , T ₂ , T ₃ and T ₄ , whose patch sizes are 32, 16, 8 and 4 respectively, merge T ₁ -T ₄ to get T _∑ , and then use the multi-head cross-attention mechanism for processing, the goal of the first stage The function is:

W_K和W_V为不同输入的权重系数，T_l表示特征信息令牌，l表示第l个尺度的特征信息，T_∑表示四个令牌的特征联合，得到查询向量Q_u，查询键 K，查询值V，l＝1，2，3，4，u＝1，2，3，4；in,

W _K and W _V are the weight coefficients of different inputs, T _l represents the feature information token, l represents the feature information of the lth scale, T _∑ represents the feature union of four tokens, and the query vector _Qu is obtained, and the query key K , query value V, l=1, 2, 3, 4, u=1, 2, 3, 4;

The objective function of the second stage is:

分别表示softmax函数和实例规范化函数，C_∑表示通道数的总和；Among them, σ(·) and

Represents the softmax function and the instance normalization function, respectively, and C _∑ represents the sum of the number of channels;

The objective function of the third stage of multi-head cross attention is:

Among them, CA _h represents the output of the second stage of multi-head cross attention, h represents the output of the hth attention head, and N is the number of attention heads;

The objective function of the final stage of multi-head cross attention is:

O _r ＝MCA _p +MLP(Q _u +MCA _p )

Determine the final output of multi-head cross attention, where MCA _p represents the output of the third stage of multi-head cross attention, p represents the pth output, MLP( ) is the multi-layer perceptron function, Qu represents the query vector, _u represents uth query vector.

In some optional embodiments, in step S2, the objective function of the multi-scale feature fusion module is:

M _i =W ₁ ·V(T _l )+W ₂ ·V(O _r )

Among them, W ₁ and W ₂ are the weight parameters of two linear layers, T _l represents the feature information token, l represents the feature information of the l-th scale, O _r represents the output of the multi-head cross-attention module, r represents the r-th The output of the attention head.

In some optional implementations, in step S2, the differential context discrimination module includes a generator and a discriminator, and the generator receives two inputs, the detected image obtained from the last layer of the multi-scale feature fusion module and the first The generated image is obtained by difference operation with the second phase, and the loss of the two is calculated to push the result closer to the actual changing image. In the generator, the weighted sum of SCAD and the least squares LSGAN loss function is used as the loss function to reduce the model The false positive rate; the least squares LSGAN loss function is used in the discriminator to improve the detection accuracy, and the loss function of the generator and the discriminator is accumulated to obtain the final probability loss.

In some optional implementations, in step S2, the objective function of the differential context discrimination module is:

L(P)=L(D)+L(G)

L(D)＝L _LSGAN (D)

L(G)＝L _LSGAN (D)+αL _SCAD

Among them, L(P) represents the probability loss, L(D) represents the discriminator loss, L(G) represents the generator loss, L _LSGAN (D) represents the least squares LSGAN loss of the discriminator, L _LSGAN (G) represents the generated LSCAD is the least squares LSGAN loss of the filter, and L _SCAD is the SCAD loss.

In some optional implementations, the SCAD loss is defined as:

Among them, C represents the detection type, v(c) represents the pixel error value of the detection type, J _C is the loss item, ρ is the parameter for continuous optimization, and v(c) is defined as follows:

Among them, y _i is the actual change image, s _g (c) is the detection score, and g represents the gth pixel.

In some alternative implementations, the least squares LSGAN loss is:

及

表示第一时相影像的检测期望，

及

表示第二时相影像的检测期望，x₁，x₂分别表示判别器输入的第一和第二时相影像，y表示实际变化影像。Among them, D(x ₁ , y) and D(x ₁ , G(x ₁ )) represent the output of the discriminator to the first phase image, G(x ₁ ) represents the output of the generator to the first phase image, D(x ₂ , y) and D(x ₂ , G(x ₂ )) represent the output of the discriminator to the second phase image, G(x ₂ ) represents the output of the generator to the second phase image,

and

Denotes the detection expectation of the first-phase image,

and

Indicates the detection expectation of the second time-phase image, x ₁ and x ₂ respectively represent the first and second time-phase images input by the discriminator, and y represents the actual change image.

In some alternative implementations, the least squares LSGAN loss is:

表示第一时相影像的检测期望，

表示第二时相影像的检测期望，D(x₁，G(x₁))表示判别器对第一时相影像的输出，G(x₁)表示生成器对第一时相影像的输出，D(x₂，G(x₂))表示判别器对第二时相影像的输出， G(x₂)表示生成器对第二时相影像的输出，x₁，x₂分别表示判别器输入的第一和第二时相影像。in,

Denotes the detection expectation of the first-phase image,

Denotes the detection expectation of the second phase image, D(x ₁ , G(x ₁ )) represents the output of the discriminator for the first phase image, G(x ₁ ) represents the output of the generator for the first phase image, D(x ₂ , G(x ₂ )) represents the output of the discriminator to the second time-phase image, G(x ₂ ) represents the output of the generator to the second time-phase image, x ₁ , x ₂ represent the input of the discriminator respectively The first and second phase images of .

Generally speaking, compared with the prior art, the above technical solutions conceived by the present invention can achieve the following beneficial effects: Based on the deep convolutional neural network, construct an automatic building detection model composed of an encoder and a decoder , which can effectively discriminate and fuse multi-scale feature information in bitemporal images, and effectively improve the accuracy of building change detection. Eventually, only the bitemporal images need to be fed into the trained model to automatically detect changes in urban buildings.

Description of drawings

Fig. 1 is a schematic flow diagram of an urban building automatic detection system and method provided by an embodiment of the present invention;

Fig. 2 is a schematic diagram of a building automatic detection model provided by an embodiment of the present invention;

FIG. 3 is a network structure diagram of a multi-head cross-attention mechanism provided by an embodiment of the present invention;

Fig. 4 is a detection comparison diagram of a different method provided by an embodiment of the present invention.

detailed description

The present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments.

The multi-scale dense connection UNet network is used to extract rich feature information in bitemporal images; the twin attention mechanism pays attention to the changed areas and unchanged areas in bitemporal images, strengthens the representation of feature information, and improves the overall information Attention: the multi-scale feature fusion module is used to gradually fuse the feature information of each scale; at the same time, the weighted sum of the generator and the discriminator is calculated by the differential context discrimination module as the probability loss, so as to promote the detection result close to the real change image. And adopt 8 evaluation indexes to evaluate the performance of the present invention, comprise precision rate (Precision), recall rate (Recall), comprehensive evaluation index (F1-score), intersection (IOU), unchanged intersection (IOU_0), change intersection ( IOU_1), overall accuracy (OA), and Kappa coefficient (Kappa) are used as evaluation indicators. The present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments.

Figure 1 is a schematic flow diagram of a system and method for automatic detection of urban buildings provided by an embodiment of the present invention, which specifically includes the following steps:

S1: Dataset construction: use the images of urban buildings collected by remote sensing satellites to construct a data set, obtain the actual change images corresponding to each building in the data set, and use the actual change images and the corresponding dual-temporal images as a data set;

Constructing a reasonable building change detection dataset can effectively improve the detection accuracy of the model. In the experiments of the embodiments of the present invention, the LEVIR-CD dataset is used, which contains a wide variety of architectural images from 20 regions. The original size of each image is 1024×1024 pixels, and the spatial resolution is 0.5m. Considering the limitation of GPU memory capacity, an image segmentation algorithm was used to cut each image into 16 regional images with a size of 256 × 256 pixels, and finally obtained 4450 time-series image pairs before and after. The invention adopts professional computer vision labeling software to label images of urban buildings. For each pair of time-series images before and after, the ground truth corresponding to the actual change image is obtained. Each pixel in the actual change image represents a category, where the class labels in the actual change image are represented by 0 and 1, and 0 represents no Change area (can be displayed as black), 1 represents change area (can be displayed as white).

After the above processing, the front and rear time-series images and their corresponding actual change images are obtained, and each front and rear time-series images and their corresponding actual change images are composed of urban building automatic detection image data sets, and the training set is divided into the training set according to the ratio of 8:2. (a total of 3560 images) and a test set (a total of 890 images).

S2: Building automatic detection model construction: Construct a twin cross-attention discriminant network composed of an encoder and a decoder as a building automatic detection model;

As shown in Fig. 2, the building automation detection model of the embodiment of the present invention includes two main modules: an encoder and a decoder. Among them, the encoder includes a weight-sharing dual-channel Siamese network and a Siamese cross-attention module, and the decoder includes a multi-scale feature fusion and differential context discrimination module.

The encoder is responsible for extracting multi-scale feature information and high-level semantic information in the input image. The decoder will extract the multi-scale features for progressive fusion, and calculate the probability loss in combination with the context difference information, so as to continuously push the result map close to the Ground Truth.

As shown in (a) of Figure 2, firstly, a weight-sharing dual-channel Siamese network is used to perform batch normalization operations on the input dual-temporal images, including convolution kernel 3, two-dimensional convolution with a step size of 1, two-dimensional Dimension BatchNorm and ReLU activation function with 64 output channels. Then extract feature information through the downsampling block, define x ^{i, j} as the output node of the downsampling block, and the objective function of the downsampling block is:

Among them, N(·) represents the nested convolution function, D(·) represents the downsampling layer, U(·) represents the upsampling layer, [] represents the feature connection function, x ^{i, j} represents the output feature map, i represents The number of layers, j represents the jth convolutional layer of the layer, and k represents the kth connection layer. In order to better describe the network parameters, the number of output channels of the three downsampling blocks is defined as 128, 256 and 512, respectively. Finally, Siamese network channels output four kinds of multi-scale feature information.

As shown in (b) of Figure 2, the Siamese cross-attention module embeds the four outputs of the weight-shared dual-channel Siamese network. First, a 2D convolution is performed to extract features, and then the features are expanded into a two-dimensional sequence T ₁ , T ₂ , T ₃ and T ₄ , whose patch sizes are 32, 16, 8 and 4, respectively. Combine T ₁ -T ₄ to get T _∑ .

As shown in Figure 3, the twin cross-attention module uses the multi-head cross-attention mechanism to extract deeper semantic information of changing features and improve the global attention to feature information. The objective function of the first stage of multi-head cross attention is:

Q _u ＝T _l W _Qi , K＝T _∑ W _K , V＝T _∑ W _V

W_K和W_V为不同输入的权重系数，T_l表示特征信息令牌，l表示第 l个尺度的特征信息，T_∑表示四个令牌的特征联合。得到查询向量Q_u(u＝1，2，3，4)，查询键K，查询值V。四个查询向量的通道数分别为[64，128，256，512]。in,

W _K and W _V are the weight coefficients of different inputs, T _l represents the feature information token, l represents the feature information of the l-th scale, T _∑ represents the feature union of four tokens. A query vector Q _u (u=1, 2, 3, 4), a query key K, and a query value V are obtained. The channel numbers of the four query vectors are [64, 128, 256, 512], respectively.

和V^T分别为查询向量Q_u和查询值V的转置。因此多头交叉注意力第二阶段的目标函数为：Since the global attention mechanism will lead to a large time complexity of the network, the transposed attention mechanism is used to reduce the computational load of the network. in

and V ^T are the transpose of query vector Q _u and query value V respectively. Therefore, the objective function of the second stage of multi-head cross attention is:

分别表示softmax 函数和实例规范化函数，

W_K和

为不同输入的权重系数，

表示特征信息令牌，l表示第l个尺度的特征信息，T_∑表示四个令牌的特征联合。C_∑表示通道数的总和。Determine the output of the second stage of multi-head cross attention, where σ( ) and

represent the softmax function and the instance normalization function, respectively,

W _K and

are the weight coefficients for different inputs,

Represents the feature information token, l represents the feature information of the l-th scale, and T _∑ represents the feature union of four tokens. C _Σ represents the sum of the number of channels.

The objective function of the third stage of multi-head cross attention is:

Among them, CA _h represents the output of the second stage of multi-head cross attention (h=1, 2, 3, 4), h represents the output of the hth attention head, and N is the number of attention heads. It has been proved by experiments that When N is set to 4, the detection effect of the network is the best.

The objective function of the final stage of multi-head cross attention is:

O _r ＝MCA _p +MLP(Q _u +MCA _p )

Determine the final output of multi-head cross-attention, where MCA _p represents the output of the third stage of multi-head cross-attention, p represents the pth output, MLP( ) is the multi-layer perceptron function, Qu represents the query vector, and _u represents The uth query vector (u=1, 2, 3, 4). Finally four outputs O ₁ , O ₂ , O ₃ and O ₄ are obtained.

As shown in Figure 2(c), the multi-scale feature fusion module adopts a double progressive fusion strategy of reconstruction and up-sampling blocks to fuse the extracted features containing rich multi-scale semantic information. The reconstruction strategy first combines the four embedded tokens T ₁ , T ₂ , T ₃ and T ₄ in the Siamese cross-attention module and the four outputs O ₁ , O ₂ , O ₃ and O ₄ in the multi-head cross-attention mechanism Perform fusion.

The objective function in the reconstruction strategy is:

M _i =W ₁ ·V(T _l )+W ₂ ·V(O _r )

Among them, W ₁ and W ₂ are the weight parameters of two linear layers, T _l represents the feature information token, l represents the feature information of the l-th scale, O _r represents the output of the multi-head cross-attention module, r represents the r-th The output of the attention head (r = 1, 2, 3, 4). Four outputs M ₁ , M ₂ , M ₃ and M ₄ are obtained.

In order to better integrate multi-scale feature information, the above four outputs are subjected to upsampling block operation, and the output channels of the four upsampling blocks are 256, 128, 64, and 64, respectively. The upsampling block consists of a 2D convolution with a kernel size of 2, an average pooling layer, and an activation function ReLu. Finally, we perform a one-dimensional convolution with a convolution kernel of 1 and a step size of 1 on the output of the fourth upsampling block to obtain the detection image.

As shown in Figure 2(d), the differential context discrimination module includes a generator and a discriminator. The generator receives two inputs, the detected image obtained by the last layer of the multi-scale feature fusion module and the generated image obtained by the differential operation of the first and second phases. The loss of both is calculated to push the result closer to the actual changing image. In the generator, the weighted sum of SCAD and least squares LSGAN loss function is used as the loss function to reduce the false detection rate of the model. The least squares LSGAN loss function is used in the discriminator to improve the detection accuracy. The loss functions of the generator and the discriminator are accumulated to obtain the final probability loss. The objective function of the differential context discrimination module is:

L(P)=L(D)+L(G)

L(D)＝L _LSGAN (D)

L(G)＝L _LSGAN (D)+αL _SCAD

Define the SCAD loss as:

Determine the SCAD loss, where C represents the detection type, v(c) represents the pixel error value of the detection type, J _C is the loss term, and ρ is a parameter for continuous optimization. v(c) is defined as follows:

Among them, y _i is the actual change image, s _g (c) is the detection score, and g represents the gth pixel.

The discriminator least squares LSGAN loss described in the present invention is:

及

表示第一时相影像的检测期望，

及

表示第二时相影像的检测期望，x₁，x₂分别表示判别器输入的第一和第二时相影像，y表示实际变化影像。Determine the least squares LSGAN loss of the discriminator, where D(x ₁ , y) and D(x ₁ , G(x ₁ )) represent the output of the discriminator for the first phase image, and G(x ₁ ) represents the generated D(x ₂ , y) and D(x ₂ , G(x ₂ )) represent the output of the discriminator to the second phase image, and G(x ₂ ) represents the output of the generator to the The output of the second phase image,

and

Denotes the detection expectation of the first-phase image,

and

Indicates the detection expectation of the second time-phase image, x ₁ and x ₂ respectively represent the first and second time-phase images input by the discriminator, and y represents the actual change image.

The generator least squares LSGAN loss described in the present invention is:

表示第一时相影像的检测期望，

表示第二时相影像的检测期望，D(x₁，G(x₁))表示判别器对第一时相影像的输出，G(x₁)表示生成器对第一时相影像的输出，D(x₂，G(x₂)) 表示判别器对第二时相影像的输出，G(x₂)表示生成器对第二时相影像的输出， x₁，x₂分别表示判别器输入的第一和第二时相影像。Determine the least squares LSGAN loss for the generator, where,

Denotes the detection expectation of the first-phase image,

Denotes the detection expectation of the second phase image, D(x ₁ , G(x ₁ )) represents the output of the discriminator for the first phase image, G(x ₁ ) represents the output of the generator for the first phase image, D(x ₂ , G(x ₂ )) represents the output of the discriminator to the second time phase image, G(x ₂ ) represents the output of the generator to the second time phase image, x ₁ , x ₂ represent the input of the discriminator respectively The first and second phase images of .

Therefore, the objective function of the differential context discriminant module is:

L(P)=L(D)+L(G)

L(D)＝L _LSGAN (D)

L(G)＝L _LSGAN (D)+αL _SCAD

Among them, L(P) represents the probability loss, L(D) represents the discriminator loss, L(G) represents the generator loss, L _LSGAN (D) represents the least squares LSGAN loss of the discriminator, and L _LSGAN (G) represents the generated LSCAD is the least squares LSGAN loss of the filter, and L _SCAD is the SCAD loss. α is a weight parameter that controls the relative importance between the two losses. With the aid of the objective function, the generator and the discriminator iteratively generate probability loss until the probability loss is lower than the set threshold and output the detection result.

S3: Use the training set in S1 to train the building automatic detection model in S2, use the trained model to realize building change detection, and finally use the building automatic detection model evaluation index to evaluate the detection results;

Use the network structure proposed by the present invention to train on the LEVIR-CD data set constructed in the S1 step, and obtain model weights for model evaluation. The training process is based on the PyTorch deep learning framework, the software environment is Ubuntu20.04, the hardware environment is a 3090 graphics card, and the video memory is 24GB. Set the batchsize to 8, and the total number of training times is 100epoch. Each input contains three images: the first temporal image, the second temporal image, and the actual change image. One test is performed after one training session. During the network training process, the urban buildings in the dual temporal image and the real change image are continuously learned. change information. The loop iterates until the epoch reaches 100, then the training ends.

Select precision rate (Precision), recall rate (Recall), comprehensive evaluation index (F1-score) intersection (IOU), unchanged intersection (IOU_0), overall accuracy (OA) change intersection (IOU_1), Kappa coefficient (Kappa) as The evaluation index and its evaluation index calculation formula are as follows:

In order to verify the performance of the building automation detection model proposed in the present invention, the present invention provides the final experimental results. Figure 4 is a visual comparison diagram of various methods, and Table 1 is the quantitative indicators of various methods.

Among them, FIG. 4 shows images of building detection results obtained by various methods. (a) is the previous image, (b) is the subsequent image, (c) is the actual change image (Ground Truth, GT), (d)-(g) are the detection result images of different methods. By comparing the actual changed images, black represents the unchanged area, white represents the changed area, red represents the false detection area, and green represents the missing detection area.

Table 1: Building detection accuracy of various methods in the LEVIR-CD dataset

Note that all metrics are in percentages and that higher numbers are better. For ease of observation, the best results are bolded.

It should be pointed out that according to the needs of implementation, each step/component described in this application can be split into more steps/components, and two or more steps/components or part of the operations of steps/components can also be combined into a new Step/component, to realize the object of the present invention.

It is easy for those skilled in the art to understand that the above description is only for preferred embodiments of the present invention, and is not intended to limit the present invention, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention , are all included in the protection scope of the present invention.

Claims (10)

1. A building change detection method for urban dynamic monitoring, characterized in that, comprising the following steps: S1, using the images of urban buildings collected by remote sensing satellites as a data set, obtaining the actual change images corresponding to each building in the data set, and dividing the actual change images and the corresponding dual-temporal images into a training set and a test set; S2. Build an automatic building detection model consisting of an encoder and a decoder. The encoder includes a weight-sharing dual-channel Siamese network and a Siamese cross-attention module. The decoder includes a multi-scale feature fusion and differential context discrimination module. ; The weight-shared dual-channel Siamese network includes a batch normalization layer and multiple upsampling blocks, inputting bitemporal images for obtaining feature maps of different scales; The twin cross-attention module first performs embedding operations on feature maps of different scales, and then uses the multi-head cross-attention mechanism to extract deeper semantic information of changing features to improve global attention to feature information; The multi-scale feature fusion module adopts a double progressive fusion strategy of reconstruction and up-sampling blocks to fuse the extracted features containing rich multi-scale semantic information; The input of the differential context discrimination module is the output image of the multi-scale fusion module and the front and rear temporal difference images. The purpose is to combine the context information in the image to improve the discrimination ability of the network, so that the detection result image is closer to the real change image, thereby Improve detection accuracy; S3, using the training set in S1 to train the building automatic detection model in S2, and using the trained model to realize building change detection. 2. The method according to claim 1, wherein step S1 comprises: Using artificially produced urban building change images as a data set, the actual change images are produced according to the bitemporal images in the dataset, where the actual change images are the change areas in the bitemporal images, and each pixel in the actual change images Represents a class, as unchanged or changed; The before and after time-series images and their corresponding actual change images are composed of urban building automatic detection image data set, in which the training set and test set are divided according to the ratio of 8:2. 3. The method according to claim 1, characterized in that: in step S2, the weight-shared dual-channel Siamese network performs a batch normalization operation on the input dual-temporal images, including a convolution kernel 3 and a step size of 1 Two-dimensional convolution, two-dimensional BatchNorm and ReLU activation function with 64 output channels, then extract feature information through three downsampling blocks, define x ^{i, j} as the output node of the downsampling block, and the objective function of the downsampling block is :

Among them, N(·) represents the nested convolution function, D(·) represents the downsampling layer, U(·) represents the upsampling layer, [] represents the feature connection function, x ^i,j represents the output feature map, and i represents The number of layers, j represents the jth convolutional layer of the layer, and k represents the kth connection layer; finally, the Siamese network channel outputs four kinds of multi-scale feature information. 4. The method according to claim 1, characterized in that: the twin-cross attention module in step S2 performs an embedding operation on the four outputs of the dual-channel twin network, first performs a 2D convolution to extract features, and then expands the features into a two-dimensional sequence T ₁ , T ₂ , T ₃ and T ₄ , whose patch sizes are 32, 16, 8 and 4 respectively, and T ₁ -T ₄ are merged to obtain T _∑ , and then processed by a multi-head cross-attention mechanism , the objective function of the first stage is:

K＝T _∑ W _K , V＝T _∑ W _V in,

W _K and W _V are the weight coefficients of different inputs, T _l represents the feature information token, l represents the feature information of the lth scale, T _∑ represents the feature union of four tokens, and the query vector _Qu is obtained, and the query key K , query value V, l=1,2,3,4, u=1,2,3,4; The objective function of the second stage is:

Among them, σ(·) and

Represents the softmax function and the instance normalization function, respectively, and C _∑ represents the sum of the number of channels; The objective function of the third stage of multi-head cross attention is:

Among them, CA _h represents the output of the second stage of multi-head cross attention, h represents the output of the hth attention head, and N is the number of attention heads; The objective function of the final stage of multi-head cross attention is: O _r ＝MCA _p +MLP(Q _u +MCA _p ) Determine the final output of multi-head cross-attention, where MCA _p represents the output of the third stage of multi-head cross-attention, p represents the pth output, MLP( ) is the multi-layer perceptron function, Qu represents the query vector, and _u represents uth query vector. 5. The method according to claim 4, characterized in that: in step S2, the objective function of the multi-scale feature fusion module is: M _i =W ₁ ·V(T _l )+W ₂ ·V(O _r ) Among them, W ₁ and W ₂ are the weight parameters of two linear layers, T _l represents the feature information token, l represents the feature information of the l-th scale, O _r represents the output of the multi-head cross-attention module, r represents the r-th The output of the attention head. 6. The method according to claim 1, characterized in that: in step S2, the differential context discrimination module includes a generator and a discriminator, two inputs are received in the generator, and the last layer of the multi-scale feature fusion module obtains The detection image and the generated image obtained by the difference operation of the first and second phases, calculate the loss of the two to push the result closer to the actual changing image, and use the weighted sum of SCAD and the least squares LSGAN loss function in the generator As a loss function, the false positive rate of the model is reduced; the least squares LSGAN loss function is used in the discriminator to improve the detection accuracy, and the loss functions of the generator and the discriminator are accumulated to obtain the final probability loss. 7. The method according to claim 6, characterized in that: in step S2, the objective function of the differential context discrimination module is: L(P)=L(D)+L(G) L(D)＝L _LSGAN (D) L(G)＝L _LSGAN (D)+αL _SCAD Among them, L(P) represents the probability loss, L(D) represents the discriminator loss, L(G) represents the generator loss, L _LSGAN (D) represents the least squares LSGAN loss of the discriminator, and L _LSGAN (G) represents the generated LSCAD is the least squares LSGAN loss of the filter, and L _SCAD is the SCAD loss. 8. method according to claim 7, is characterized in that: define SCAD loss as:

Among them, C represents the detection type, v(c) represents the pixel error value of the detection type, J _C is the loss item, ρ is the parameter for continuous optimization, and v(c) is defined as follows:

Among them, y _i is the actual change image, s _g (c) is the detection score, and g represents the gth pixel. 9. The method according to claim 7, characterized in that: the least squares LSGAN loss is:

Among them, D(x ₁ ,y) and D(x ₁ ,G(x ₁ )) represent the output of the discriminator to the first phase image, G(x ₁ ) represents the output of the generator to the first phase image, D(x ₂ ,y) and D(x ₂ ,G(x ₂ )) represent the output of the discriminator to the second phase image, G(x ₂ ) represents the output of the generator to the second phase image,

and

Denotes the detection expectation of the first-phase image,

and

Indicates the detection expectation of the second time-phase image, x ₁ and x ₂ respectively represent the first and second time-phase images input by the discriminator, and y represents the actual change image. 10. The method according to claim 7, characterized in that: the least squares LSGAN loss is:

in,

Denotes the detection expectation of the first-phase image,

Represents the detection expectation of the second phase image, D(x ₁ ,G(x ₁ )) represents the output of the discriminator for the first phase image, G(x ₁ ) represents the output of the generator for the first phase image, D(x ₂ , G(x ₂ )) represents the output of the discriminator to the second time-phase image, G(x ₂ ) represents the output of the generator to the second time-phase image, x ₁ , x ₂ represent the input of the discriminator respectively The first and second phase images of .

Images