CN111914611A - High score remote sensing monitoring method and system for urban green space - Google Patents

Abstract

The invention relates to a high-resolution remote sensing monitoring method and system for urban green space. The method includes a training sample set construction step, a multi-dimensional feature space construction step, a U-Net+ model construction step, and an image post-processing step. At the same time, the U-Net+ deep learning model is constructed, and the image post-processing method is combined to improve the generalization and robustness of the monitoring method, and solve the problem of over-fitting due to limited training samples, so as to improve the height of urban green space. Accuracy and timeliness of high-resolution remote sensing monitoring.

Description

High score remote sensing monitoring method and system for urban green space

technical field

The invention relates to the technical field of remote sensing monitoring, in particular to a high-resolution remote sensing monitoring method and system for urban green space.

Background technique

Urban green space plays a very important role in the urban ecosystem and is closely related to human health, quality of life of residents, biodiversity, and social security. The distribution of urban green space is heterogeneous and highly dispersed. How to realize the precise quantification of the spatial and temporal pattern of urban green space is a hot issue in current research, which is crucial to the planning and management of urban green space.

At present, there have been a lot of studies on the classification and extraction of urban green space. With the development of remote sensing technology, more and more high-resolution remote sensing images are applied to urban green space mapping and change analysis, such as SPOT, IKONOS, Quick-Bird, Worldview, series, etc. High-resolution remote sensing images contain rich ground object information, with obvious spectral features, clearer geometric features, and richer texture details. It can identify urban street trees and residential gardens at the sub-meter scale, providing finer urban green space features. .

There are many classification methods using remote sensing. Traditional remote sensing image classification methods are mainly based on pixel classification and object-oriented classification. Pixel-based classification is mainly based on the spectral information of reference pixels, while high-resolution remote sensing images are relatively lacking in spectral information due to their high spatial resolution, while relatively many features such as shape and texture. important role in image classification. The object-oriented classification method integrates the spectral features, geometric features and texture features of ground objects, and its experimental steps are "segmentation-feature selection-classification". The object-oriented method has no fixed segmentation parameters. If the segmentation scale is too large, a large number of mixed pixels will appear. If the segmentation scale is too small, the shape information will be missing. increased workload. Traditional machine learning methods such as support vector machines and decision trees cannot fully learn from complex features, nor can they adapt to high-resolution image samples with a large amount of data, and the obtained classification results cannot meet the needs of classification. Although the traditional urban green space remote sensing information classification and extraction methods are relatively mature, due to the limitations of the methods themselves, the optimal segmentation parameters and object feature selection need to be manually selected, and the accuracy is relatively low.

At present, the deep learning method has strong application potential in the classification of high-resolution remote sensing images, especially the U-Net model has also emerged in the application of remote sensing image classification. Deep learning has been widely used in many different tasks such as image recognition, object detection and image classification, and has achieved good results. At present, the more common deep learning neural network models for remote sensing image classification include Deep Belief Network (DBN), Stacked Autoencoder Network (SAE) and Convolutional Neural Network (CNN). Fields such as classification and segmentation excel and are among the most widely used deep learning models. However, the current research has the following problems:

(1) There are still many defects in the existing deep learning models. For example, the structure of the model, the generalization and robustness of the model, the calculation mode of the loss function, etc.; especially the small-scale image data set will cause the overfitting of the deep learning model, the robustness of the model and the deterioration of the generalization ability of the model. And other issues. (2) The richness of feature learning is insufficient. Gaofen-2 remote sensing images have a limited number of bands, relatively lack of spectral information, and less richness of features, which limits the richness of deep learning in feature learning to a certain extent. (3) The problem of model misclassification. There are usually small misclassified areas in the classified images, and the boundaries of the objects are slightly smooth. Therefore, it is necessary to add image post-processing methods to optimize the classification results and obtain classification results that are closer to the real objects.

SUMMARY OF THE INVENTION

Aiming at the problems existing in the prior art, the present invention proposes a high-resolution remote sensing monitoring method for urban green space. By constructing a multi-dimensional feature space, enhancing feature richness, and simultaneously constructing a U-Net+ deep learning model, combined with an image post-processing method, monitoring is improved. The generalization and robustness of the method can solve the problem of over-fitting due to limited training samples, so as to improve the accuracy and timeliness of high-resolution remote sensing monitoring of urban green space. The invention also relates to an urban green space high score remote sensing monitoring system.

The technical scheme of the present invention is as follows:

A high-score remote sensing monitoring method for urban green space, comprising the following steps:

The training sample set construction step, according to the high-resolution remote sensing image features, select the sample area, and construct the training sample data set;

The multi-dimensional feature space construction step is to perform data enhancement, random cropping and feature calculation processing on the training sample data set to construct a multi-dimensional feature space including vegetation features, spatial features, contrast features, texture features and phenological features; by constructing a normalized vegetation index Taking it as the vegetation feature of urban green space, construct the nDSM digital surface model to take it as the spatial feature of urban green space, calculate the local contrast feature map through AC algorithm and take it as the contrast feature of urban green space, and obtain the image texture feature map through the gray level co-occurrence matrix. Take it as the texture feature of urban green space, and add the winter images of the same period as the phenological feature of urban green space;

The U-Net+ model construction step is based on the constructed multi-dimensional feature space, facing the characteristics of urban green space, using the image edge zero padding improvement method, the batch normalization processing improvement method and the regularization improvement method to improve the U-net model in turn, and establish a face-to-face Urban green space and U-Net+ deep learning model based on multi-dimensional features; then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, and after the training is completed, predict the spatial distribution of urban green space to obtain U-Net+ Model prediction results;

The image post-processing step is to post-process the prediction results of the U-Net+ model to obtain the high-resolution remote sensing monitoring results of urban green spaces.

Preferably, in the step of constructing the training sample set, according to the features of high-resolution remote sensing images, some typical regions are used as sample regions, and a visual interpretation method is used to extract samples of high-resolution remote sensing images, and the vector files obtained by visual interpretation are extracted. Convert elements to raster to obtain label images, record real surface images of different types of vegetation locations in remote sensing images, and construct training sample data sets; the multi-dimensional feature space construction step first uses data enhancement methods to perform image processing to highlight effective spectral information, Then, the label image and the data-enhanced remote sensing image are cropped by the method of random cropping, and feature calculation processing is performed based on the cropped label image and remote sensing image to construct a multi-dimensional feature space.

Preferably, the improved method of batch normalization processing in the U-Net+ model building step is to add batch normalization processing after the model convolution layer, and then input the normalized data into the model convolution layer. The next layer of ; the regularization improvement is to add a dropout layer with a specific probability of dropping neurons after each deconvolution of the model.

Preferably, in the U-Net+ model building step, the multi-dimensional feature data of the training sample is added to the U-net+ deep learning model for training, which is to input the remote sensing image, multi-dimensional feature data and corresponding label image of the training sample into the U-net+ depth In the learning model, the U-net+ deep learning model extracts the features of the input image in the encoding part, restores its spatial position and resolution in the decoding part, and classifies each pixel in the pixel classification layer to obtain category information; Calculate the loss value in the cross entropy function with the input label map, pass the loss value to the U-net+ deep learning model for backpropagation, and optimize the parameters in the model layer by layer. When the loss value reaches a certain threshold, the training stops.

Preferably, in the image post-processing step, the fully-connected CRFs image post-processing method is used to post-process the prediction results of the U-Net+ model, and the classification results are processed in combination with the relationship between all pixels in the original high-resolution remote sensing image, so as to make The prediction results are optimized, and the high-resolution remote sensing monitoring results of urban green space are obtained.

An urban green space high score remote sensing monitoring system is characterized in that it includes a training sample set building module, a multi-dimensional feature space building module, a U-Net+ model building module and an image post-processing module connected in sequence, the training sample set building module and U-Net+ model building blocks are connected,

The training sample set building module selects a sample area according to the features of high-resolution remote sensing images, and constructs a training sample data set;

The multi-dimensional feature space building module performs data enhancement, random cropping and feature calculation processing on the training sample data set of the training sample set building module, and constructs a multi-dimensional feature space including vegetation features, spatial features, contrast features, texture features and phenological features. ; By constructing the normalized vegetation index and taking it as the vegetation feature of urban green space, constructing nDSM digital surface model as the spatial feature of urban green space, calculating the local contrast feature map by AC algorithm and taking it as the contrast feature of urban green space, through grey The degree co-occurrence matrix is used to obtain the image texture feature map and use it as the texture feature of urban green space, and add the winter images of the same period to use it as the phenological feature of urban green space;

The U-Net+ model building module, based on the multi-dimensional feature space constructed by the multi-dimensional feature space building module, is oriented towards the features of urban green spaces, and sequentially uses the image edge zero-filling improvement method, the batch normalization processing improvement method and the regularization improvement method to improve. U-net model, establish a U-Net+ deep learning model based on multi-dimensional features for urban green space; then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, and after the training is completed, the urban green space is distributed. to obtain the prediction results of the U-Net+ model;

The image post-processing module performs post-processing on the prediction results of the U-Net+ model to obtain high-resolution remote sensing monitoring results of urban green spaces.

Preferably, in the training sample set building module, for the features of high-resolution remote sensing images, some typical areas are used as sample areas, and a visual interpretation method is used to extract samples of high-resolution remote sensing images, and the vector files obtained by visual interpretation are extracted. Convert elements to raster to obtain label images, record real surface images of different types of vegetation locations in remote sensing images, and construct training sample data sets; the multi-dimensional feature space building module first uses data enhancement methods to perform image processing to highlight effective spectral information, Then, the label image and the data-enhanced remote sensing image are cropped by the method of random cropping, and feature calculation processing is performed based on the cropped label image and remote sensing image to construct a multi-dimensional feature space.

Preferably, the improved method of batch normalization processing in the U-Net+ model building module is to add batch normalization processing after the model convolution layer, and then input the normalized data into the model convolution layer. The next layer of ; the regularization improvement is to add a dropout layer with a specific probability of dropping neurons after each deconvolution of the model.

Preferably, in the U-Net+ model building module, the multi-dimensional feature data of the training samples are added to the U-net+ deep learning model for training, and the remote sensing images, multi-dimensional feature data and corresponding label images of the training samples are input into the U-net+ deep learning model. In the learning model, the U-net+ deep learning model extracts the features of the input image in the encoding part, restores its spatial position and resolution in the decoding part, and classifies each pixel in the pixel classification layer to obtain category information; Calculate the loss value in the cross entropy function with the input label map, pass the loss value to the U-net+ deep learning model for backpropagation, and optimize the parameters in the model layer by layer. When the loss value reaches a certain threshold, the training stops.

Preferably, the image post-processing module uses the fully connected CRFs image post-processing method to post-process the prediction results of the U-Net+ model, and processes the classification results in combination with the relationship between all the pixels in the original high-resolution remote sensing images, so as to make The prediction results are optimized, and the high-resolution remote sensing monitoring results of urban green space are obtained.

The technical effect of the present invention is as follows:

The present invention relates to a high-score remote sensing monitoring method for urban green space. Sufficient training sample data sets are obtained in a sample area, and a combination of data enhancement, random clipping and feature calculation is used to process the training sample data sets to construct a specific training sample data set. The multi-dimensional feature space avoids overfitting while generalizing the network model. In particular, on the basis of the first four characteristics, combined with phenological characteristics, deciduous trees wither in winter, high-resolution remote sensing images and multi-temporal remote sensing images are used, and winter images are added to the deep learning model in combination with the phenology theory of vegetation to distinguish evergreen trees. and deciduous trees, optimize the classification results of single summer images, and provide a basis for the accuracy of subsequent high-score remote sensing monitoring of urban green spaces. Based on the constructed multi-dimensional feature space, facing the characteristics of urban green space, the U-net model is improved by successively using the improved method of image edge zero padding, the improved method of batch normalization and the improved method of regularization, and a multi-dimensional feature-oriented urban green space is established. U-Net+ deep learning model, proposes the concept of U-Net+ deep learning model based on multi-dimensional features, realizes the optimization of U-Net model, and adopts a series of specific improvement methods. The U-Net+ deep learning model has extremely high performance. Then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, realize the training of the samples and the training of the model, and predict the spatial distribution of urban green space after the training is completed. Net+ model prediction results, combined with image post-processing methods to obtain high-resolution remote sensing monitoring results of urban green space, improve the generalization and robustness of monitoring methods, and solve the problem of over-fitting due to limited training samples. Accuracy and timeliness of high-resolution remote sensing monitoring of urban green space.

The present invention also relates to an urban green space high-score remote sensing monitoring system, which corresponds to the above-mentioned urban green space high-score remote sensing monitoring method, and can be understood as a system for implementing the above-mentioned urban green space high-score remote sensing monitoring method, and sets corresponding training samples Set building module, multi-dimensional feature space building module, U-Net+ model building module and image post-processing module, all modules work together with each other. A U-Net+ deep learning model based on multi-dimensional features is established for urban green space, combined with image post-processing, to improve the generalization and robustness of the monitoring method, thereby improving the accuracy of high-resolution remote sensing monitoring of urban green space.

Description of drawings

FIG. 1 is a flow chart of the method for monitoring high-resolution remote sensing of urban green space according to the present invention.

FIG. 2 is a preferred flow chart of the method for high-resolution remote sensing monitoring of urban green space of the present invention.

Fig. 3 is a partial image of the dataset and the corresponding label image in the step of constructing the training sample set.

Figures 4a and 4b are comparison diagrams of the improved methods of zero-filling image edges in the U-Net+ model building step.

Figure 5a and Figure 5b are comparison diagrams of regularization improvements in the U-Net+ model building step.

FIG. 6 is a structural diagram of the U-Net+ deep learning model based on multi-dimensional features of the present invention.

FIG. 7 is a flowchart of training a U-Net+ deep learning model with multi-dimensional features.

FIG. 8 is a comparison diagram between the prediction result of the U-Net+ model of the present invention and the real surface.

FIG. 9 is a structural diagram of an urban green space high score remote sensing monitoring system of the present invention.

Detailed ways

The present invention will be described below with reference to the accompanying drawings.

The present invention relates to a high-resolution remote sensing monitoring method for urban green space, the process of which is shown in FIG. 1 , including: a training sample set construction step, selecting a sample area according to the characteristics of high-resolution remote sensing images, and constructing a training sample data set within the sample area; The multi-dimensional feature space construction step is to perform data enhancement, random clipping and feature calculation processing on the training sample data set, that is to say, carry out feature calculation processing on the training sample data set after data enhancement and random clipping processing. The multi-dimensional feature space of five types of features including feature, contrast feature, texture feature and phenological feature; by constructing the normalized vegetation index as the vegetation feature of urban green space, and constructing the nDSM digital surface model as the spatial feature of urban green space, through The AC algorithm calculates the local contrast feature map and takes it as the contrast feature of the urban green space, obtains the image texture feature map through the gray level co-occurrence matrix and takes it as the texture feature of the urban green space, and adds the winter images of the same period as the phenological feature of the urban green space; U -Net+ model construction step, based on the constructed multi-dimensional feature space, facing the characteristics of urban green space, using the image edge zero padding improvement method, the batch normalization processing improvement method and the regularization improvement method to improve the U-net model in turn, and establish a city-oriented improvement method. Green space and U-Net+ deep learning model based on multi-dimensional features; then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, and after the training is completed, predict the spatial distribution of urban green space to obtain the U-Net+ model Prediction results; image post-processing step, post-processing the prediction results of the U-Net+ model to obtain high-resolution remote sensing monitoring results of urban green spaces. This method builds a multi-dimensional feature space, enhances feature richness, builds a U-Net+ deep learning model and performs sample training and model training, and then combines image post-processing methods to optimize the prediction results and improve the generalization and robustness of the monitoring method. It can solve the problem of over-fitting due to limited training samples, so as to improve the accuracy and timeliness of high-resolution remote sensing monitoring of urban green space.

FIG. 2 is a preferred flow chart of the method for high-resolution remote sensing monitoring of urban green space of the present invention. Preferably,

First, the training sample set construction steps:

According to the characteristics of high-resolution remote sensing images, some typical areas are used as sample areas, and the visual interpretation method is used to extract samples of high-resolution remote sensing images. The ground-truth images of the locations of different types of vegetation in the imagery are obtained to obtain sufficient training sample datasets.

As shown in Figure 2, after the high-resolution remote sensing data is input, sample area selection and target interpretation are performed. First, according to the spatial form, organization and vegetation type of urban green space, select some typical areas as sample areas, which can also be called typical sample areas. For example, select several typical areas for parks, select several typical areas for golf courses, residents Select several typical areas and so on. Then, the visual interpretation method is used to extract samples of high-resolution remote sensing images, and the vector files obtained by visual interpretation are converted into elements to obtain label images, that is, the real surface images that record the locations of different types of vegetation in remote sensing images. The partial images of the dataset and the corresponding label images shown in Figure 3 show the label images labeled as deciduous trees, evergreen trees, grasslands, and non-vegetated areas.

Second, the multi-dimensional feature space construction steps:

First, the data enhancement method is used for image processing to highlight the effective spectral information, and then the label image and the data-enhanced remote sensing image are cropped by the random cropping method, and feature calculation processing is performed based on the cropped label image and remote sensing image to construct a multi-dimensional feature space. Data enhancement and random cropping of the training sample data set can provide sufficient data sets, improve the efficiency of training sample set construction, avoid over-fitting problems while generalizing the network model, and are the basic prerequisite for constructing multi-dimensional feature space data.

First, image processing is performed using data enhancement methods to highlight effective spectral information and improve the convergence efficiency in the deep learning process. The present invention adopts the min-max normalization method, which is also called dispersion normalization, and reduces the data range of the pixel value of each channel from the interval [0, 255] to [0, 1].

Among them, x _min represents the minimum value of the sample data, and x _max represents the maximum value of the sample data.

Secondly, the random cropping method is used to increase the number of training samples to ensure the number of samples required for deep learning. Set the size of the cropping size (such as 256×256), and at the same time, according to the set quantity, combined with the random function, set the cropping starting position point. According to the starting point and cropping size, the sample area and high-resolution remote sensing image after visual interpretation are cropped out of the sample data, thereby greatly increasing the data volume of the sample data.

Then, according to the training sample dataset after data enhancement and random cropping, feature calculation processing is performed to construct a multi-dimensional feature space of five types of features including vegetation features, spatial features, contrast features, texture features and phenological features. In view of the current situation that the high-resolution remote sensing image data has fewer bands and the richness of model feature learning is limited, the present invention constructs five types of remote sensing image features from five different dimensions, namely, vegetation features, spatial features, contrast features, texture features and phenological characteristics. That is, from the five aspects of spectrum, space, contrast, texture and phenology, a multi-dimensional feature space is constructed to enrich the information of deep learning.

First, by constructing the normalized vegetation index as the vegetation feature of urban green space; second, by constructing the nDSM digital surface model as the spatial feature of urban green space; third, by calculating the local contrast feature map through the AC algorithm, the It is used as the contrast feature of urban green space; fourth, the image texture feature map is obtained through the gray level co-occurrence matrix, and it is used as the texture feature of urban green space; fifth, the winter images of the same period are added to use it as the phenological feature of urban green space.

(1) Vegetation characteristics

We select the normalized vegetation index for vegetation characteristics, which is an important index to measure the degree of vegetation coverage and detect the growth status of plants. It can synthesize relevant spectral information, highlight the vegetation in the image, and reduce the non-vegetation information.

In the formula, NIR represents the value in the near-infrared band, and R represents the value in the red band. When NDVI<0, there may be water, snow or clouds in the area; when NDVI>0, it means that the area may be covered with vegetation; when NDVI=0, the area is likely to be covered with rocks or bare land, etc. features.

(2) Spatial features

The present invention selects nDSM as the spatial feature of urban green space. nDSM refers to a data model that eliminates the influence of terrain and records the height information of all ground objects relative to the ground. Digital Surface Model (DSM, Digital Surface Model) refers to a ground elevation model that includes the heights of buildings, bridges, and trees on the ground. Digital Elevation Model (DEM, Digital Elevation Model) is a spatial data model that describes the morphological characteristics of surface relief. The data model nDSM containing only the height of the ground object can be obtained by subtracting the DSM including the height of the ground object and the height of the terrain relief from the DEM. Calculated as follows:

nDSM(i,j)=DSM(i,j)-DEM(i,j) (3)

Among them: nDSM(i,j) represents the elevation value of nDSM in row i, column j; DSM(i,j) represents the elevation value of DSM in row i, column j; DEM(i,j) represents the elevation value of DEM in row i and column j The elevation value of row i and column j.

(3) Contrast feature

The AC algorithm is a feature extraction algorithm based on local contrast. Its basic idea is to convert the image from the RGB color space to the LAB color space, and then for each perceptual unit in the image, calculate the local image separately on windows of different sizes. Contrast value, and then add multiple local contrast values obtained under different window sizes, and traverse the entire image in this way to obtain the final image feature map.

For example, assuming the inner region R1 and the outer region R2, when calculating the local contrast of the inner region R1 and the outer region R2, the multi-scale saliency calculation is realized by changing the size of R2. The perceptual unit R1 may be a pixel or a pixel block, and its neighborhood is R2, and the average value of the feature values of all pixels included in (R1)R2 is used as the feature value of (R1)R2. Let pixel p be the center of R1 and R2, and the local contrast at the position of p is:

where N1 and N2 are the number of pixels in R1 and R2, respectively. v _k is the eigenvalue or eigenvector of the position k. Since the AC method uses the Lab color space, the Euclidean distance is used to calculate the feature distance. R1 is one pixel by default, R2 is a square area between [L/8, L/2], and L is the smaller of the length and width. Feature saliency maps of multiple scales are directly added to obtain a complete saliency map.

(4) Texture features

The texture feature of the image can better take into account both the macroscopic properties and the fine structure of the image, and is an important feature for the classification and extraction of ground objects. There are many methods for extracting texture features, such as features based on local statistical properties, features based on random field models, features based on spatial frequency, fractal features, etc. The most widely used feature is based on gray value co-occurrence matrix.

The gray-scale co-occurrence matrix method proposed in 1973 is an important texture analysis method currently recognized. The co-occurrence matrix is a function of distance and direction, which describes a pair of pixels separated by a distance of d pixels in the θ direction, with the probability of occurrence of gray values i and j respectively, and its elements can be recorded as P(i,j| d, θ). The calculation formula of each element of the gray level co-occurrence matrix is as follows:

It is assumed that the image to be analyzed has Nx and Ny pixels in the horizontal and vertical directions respectively. Let Zx={1,2,...,Nx} be the horizontal space domain, and Zy={1,2,...,Ny} be the vertical space domain. When the distance is 1 and θ is 0 degrees, 45 degrees, 90 degrees, and 135 degrees, the formulas are:

Among them, k, m and l, n represent the variation in the selected calculation window, respectively, and # represents the number of pixel pairs that make the curly brackets true.

Carry out the calculation of eigenvalues, construct co-occurrence matrices in four directions according to the above formula (6-9), and then calculate the four texture parameters of energy, entropy, inertia moment, and correlation of these four co-occurrence matrices respectively, and obtain the texture features of the image. .

(5) Phenological characteristics

Deciduous trees wither in winter, the present invention uses high-resolution remote sensing images and multi-temporal remote sensing images, and combines the phenology theory of vegetation to add winter images to the deep learning model to distinguish evergreen trees and deciduous trees, and optimize the classification results of single summer images.

3. U-Net+ model construction steps:

Based on the constructed multi-dimensional feature space, facing the characteristics of urban green space, the U-net model is improved by successively using the improved method of image edge zero padding, the improved method of batch normalization and the improved method of regularization, and a multi-dimensional feature-oriented urban green space is established. U-Net+ deep learning model.

The image edge filling strategy is adopted to ensure that the image sizes of the input network and output network are consistent; in order to improve the generalization and robustness of the model, Dropout regularization and BN layer are added to improve the U-Net model, which can ensure that the model avoids excessive The training speed is accelerated while fitting; in order to eliminate the influence of low image edge prediction accuracy on the calculation result of the loss function, the cross-entropy function is improved to improve the prediction accuracy.

At the same time, for the first four types of features, high-resolution remote sensing images (true color/false color) are combined with previously constructed vegetation, space, contrast, and texture features to obtain 4-channel multi-feature high-resolution images. According to the phenological characteristics, the deciduous vegetation in winter has entered a dormant period, and the spectral difference is significantly different from that of the summer vegetation. Combined with the theory of vegetation phenology, adding the Gaofen-2 winter image to the model can eliminate the winter deciduous trees and correct the summer image classification results. Input the prepared training set into the U-Net+ deep learning model for training to obtain the best parameter combination, and use the inflation prediction to obtain the classification result during model prediction. The obtained vegetation features, spatial features, contrast features, texture features, and phenological feature classification maps are fused by voting method, and the best prediction results, such as urban green space in the main urban area of Beijing, are further optimized.

The U-Net model is an improved FCN structure and is currently the most scalable fully connected neural network. Its structure clearly presents the letter U shape, which consists of the left half of the compression channel (Contracting Path) and the right half of the expansion channel (Expansive Path). U-Net cleverly combines the characteristics of the encoding-decoding structure and the skip network, and the compression channel is an encoder, which is used to extract the features of the image layer by layer. It adopts the structure of 2 convolutional layers and 1 maximum pooling layer, and the dimension of the feature map is doubled after each pooling operation. The expansion channel is a decoder, which is used to restore the position information of the image. In the expansion channel, a deconvolution operation is performed first to halve the dimension of the feature map, and then the feature map cropped by the corresponding compression channel is spliced and reconstituted. A feature map of 2 times the size, and then 2 convolutional layers are used for feature extraction, and the structure is repeated. In the final output layer, 2 convolutional layers are used to map the 64-dimensional feature map into a 2-dimensional output map. However, the existing U-Net model still has some deficiencies. The present invention makes specific improvements to the original network structure, and establishes a U-Net+ deep learning model based on multi-dimensional features for urban green space.

(1) Improvement of image edge zero padding

In order to ensure that the size of the input image and the output image are consistent, the image edge zero-filling strategy is adopted, as shown in Figure 4a and Figure 4b for comparison, Figure 4a is the valid mode, that is, there is no zero-padding; Figure 4b is the same mode, that is, zero-padding Case.

(2) Improvement of batch normalization processing method

Deep learning continuously updates parameters during training, so in addition to the input layer, the update of training parameters will lead to changes in the distribution of input data in subsequent layers, which can easily lead to the occurrence of gradient dispersion and reduce the speed of training. Since there are many network training parameters, in order to prevent gradient disappearance and gradient explosion, the present invention adopts a batch normalization (BatchNormalization, BN) processing method, adding batch normalization processing after the convolution layer, and then the normalized data is processed. Input the next layer of the above-mentioned convolutional layer, thereby preventing the gradient from disappearing and exploding, and improving the training speed of the network. Doing so can better connect the output of the upper network to the result of the upsampling of the last layer. The normalization layer here is a learnable, parameterized network layer, as shown in Equation 1.

为标准差归一化结果，即

γ^(k)、β^(k)为学习参数，其中

β^(k)＝E[x^(k)]。in,

Normalize the result for the standard deviation, that is

γ ^(k) and β ^(k) are learning parameters, where

β ^(k) = E[x ^(k) ].

(3) Dropout layer improvement

Dropout is a method of regularization in deep learning, which means that during network training, neurons are temporarily discarded with a certain probability to prevent the model from over-learning and over-fitting with a small number of samples. If the dropout layer is added after each deconvolution of the model, the overfitting situation can be prevented to a certain extent, as shown in the comparison diagrams shown in Figure 5a and Figure 5b. Figure 5a shows the original network without adding the Dropout layer. Figure 5 5b is the network after adding the Dropout layer.

The structure of the improved U-Net+ deep learning model based on multi-dimensional features for urban green space is shown in Figure 6. The input image size of the U-Net+ model structure is 256×256×c (c represents the multi-source data added. The total number of bands in the image), first a 3×3 convolution and BN operation (horizontal black crop) are performed to convert the input data into a 32-dimensional feature map, and then 1 max pooling layer and 2 volumes are repeated. Layered structure. The extended channel is a decoder that gradually restores the details and position of the image. In the expansion channel, a deconvolution operation (vertical hollow arrow) is first performed to halve the dimension of the feature map, and then the feature map corresponding to the compressed channel (slash filled box) is spliced to form a 2-fold dimension. The number of feature maps, and then two convolution layers are used, and this structure is repeated. The splicing operation can enable the network to learn multi-scale and different-level features, increase the robustness of the network, and help improve the classification accuracy; in the final output layer, A 6-dimensional convolutional layer with a convolution kernel of 1 × 1 is used to map the feature map obtained by the previous layer into a 6-dimensional output feature map. The letter "D" in the figure represents the addition of a dropout layer with a probability of dropping neurons of 0.5 after the convolutional layer at that position. The linear activation function uses Relu, the pooling selects max-pooling, and the optimizer uses Adam.

Add the multi-dimensional feature data of the training samples to the U-net+ model for training. As shown in the training flow chart in Figure 7, in the model training, a total of 350 remote sensing images of 256×256, multi-dimensional feature data and corresponding labels are input into the U-net+ model. The decoding part restores its spatial position and resolution, and classifies each pixel in the pixel classification layer to obtain class information. Calculate the loss value (loss) in the cross entropy function between the predicted classification map and the input label map, pass the loss value to the model for back propagation, and optimize the parameters in the model layer by layer.

The activation function of the convolutional layer adopts the RELU function, and the calculation formula of the convolutional layer can be expressed as:

分别为第l层第k维的权重值与偏置项，

为第l层第k维的特征图，

表示第l-1层第k维的输出特征图，

表示卷积运算。in,

are the weight value and bias term of the k-th dimension of the l-th layer, respectively,

is the feature map of the kth dimension of the lth layer,

represents the output feature map of the kth dimension of the l-1th layer,

Represents a convolution operation.

The deep learning model generally uses the loss function to calculate the loss value between the ground truth data and the predicted probability to quantify the gap between the two. When the loss value is smaller, the classification is more accurate. The present invention uses the Categorical Crossentropy function (Categorical Crossentropy) to calculate the loss value, and the formula is:

The process of model training is the process of optimizing the loss function and reducing the loss value, that is, backward propagation. In the present invention, the Adam optimization algorithm is preferably used for model training, and the parameters in the model are updated layer by layer, and the Adam algorithm is easy to implement, with high computational efficiency and low memory requirements. When the loss value reaches a certain threshold, the training stops.

After the training of the U-net+ model is completed, the parameters of each layer have achieved the optimal value. At this time, the forward propagation of the test set image is used again, which is the process of model prediction. The model prediction results shown in Figure 8 are compared with the real surface.

Four, image post-processing steps:

Post-processing the prediction results of the U-Net+ model, preferably using the fully connected CRFs image post-processing method to post-process the prediction results of the U-Net+ model, and combine the relationship between all pixels in the original high-resolution remote sensing image to process the classification results. Therefore, the prediction results are optimized, and the high-resolution remote sensing monitoring results of urban green space are obtained.

Fully connected CRFs connect all pixels in the image pairwise, describe the relationship between each pixel and all other pixels, and use the color and actual relative distance between pixels to measure the gap between pixels, while encouraging the allocation of pixels with large gaps Different labels enable the fully connected CRFs to be segmented at the boundary as much as possible, and it is not easy to cause excessive smoothing of the boundary.

In fully connected CRFs, the energy of the predicted label value x is defined as:

Among them, i,j∈{1,2,...N,N is the total number of image pixels}, ψ _u (x _i ) is the unary potential energy, and ψ _p (x _i ,x _j ) is the binary potential energy. The unary potential energy ψ _u ( _xi ) represents the class probability distribution map independently predicted by the network model for each pixel i in the image, which contains a lot of noise and discontinuity. The binary potential energy ψ _p (x _i ,x _j ) represents a fully connected graph that connects all pixel pairs in the image. In the actual application process, the information provided by the original image is used as the binary potential energy. Its model expression is:

为特征函数，公式如下：Among them, u(x _i , y _j ) is a label compatible term, which constrains the condition of conduction between pixels, that is, only under the condition of the same label, energy can be transmitted to each other. ω ^(m) is the weight parameter,

is the characteristic function, the formula is as follows:

以特征的形式表示了不同像素之前的“亲密度”，第一项被称作表面核，第二项被称作平滑核。在全连接CRFs进行影像后处理的实际过程中，一元势能是概率分布图，是每一个像素的类别分配概率，由网络模型输出的特征图经过softmax函数运算后得到的结果；二元势能中的位置信息和颜色信息是由原始影像提供。Among them, I _i , I _j are color vectors, and p _i , p _j are position vectors. Characteristic Function

The "closeness" before different pixels is represented in the form of features, the first term is called the surface kernel and the second term is called the smooth kernel. In the actual process of image post-processing with fully connected CRFs, the unary potential energy is the probability distribution map, which is the class distribution probability of each pixel. The feature map output by the network model is the result obtained by the softmax function; Position information and color information are provided by the original image.

The U-Net+ model prediction results are post-processed using the fully connected CRFs method. Fully connected CRFs can combine the relationship between all pixels in the original image to process the classification results, improve the misclassification phenomenon, refine the edge of objects, optimize the prediction results, and form high-precision remote sensing monitoring results of urban green space.

The present invention also relates to an urban green space high-score remote sensing monitoring system, which corresponds to the above-mentioned urban green space high-score remote sensing monitoring method, and can be understood as a system for realizing the above-mentioned urban green space high-score remote sensing monitoring method. The system structure is shown in the figure As shown in 9, it includes a training sample set building module, a multi-dimensional feature space building module, a U-Net+ model building module, and an image post-processing module that are connected in sequence. The training sample set building module and the U-Net+ model building module are connected. Working together, after constructing sufficient training sample data sets, by constructing a multi-dimensional feature space, enhancing feature richness, and establishing a U-Net+ deep learning model based on multi-dimensional features for urban green space, combined with image post-processing methods, improve monitoring methods The generalization and robustness of the method can improve the accuracy of high-resolution remote sensing monitoring of urban green space.

Among them, the training sample set building module selects a sample area and constructs a training sample data set in the sample area according to the features of high-resolution remote sensing images; the multi-dimensional feature space building module performs data enhancement, Random cropping and feature calculation processing to construct a multi-dimensional feature space including vegetation features, spatial features, contrast features, texture features and phenological features; by constructing the normalized vegetation index as the vegetation feature of urban green space, the nDSM digital surface model is constructed to As the spatial feature of urban green space, the local contrast feature map is calculated by the AC algorithm as the contrast feature of urban green space, and the image texture feature map is obtained through the grayscale co-occurrence matrix as the texture feature of urban green space. It is used as the phenological feature of urban green space; the U-Net+ model building module, based on the multi-dimensional feature space constructed by the multi-dimensional feature space building module, is oriented towards the characteristics of urban green space, and successively uses the image edge zero-filling improvement method and batch normalization processing improvement method. And the regularization improvement method to improve the U-net model, establish a U-Net+ deep learning model based on multi-dimensional features for urban green space; then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, and after the training is completed Then, predict the spatial distribution of urban green space, and obtain the prediction results of the U-Net+ model; the image post-processing module performs post-processing on the prediction results of the U-Net+ model, and obtains the high-resolution remote sensing monitoring results of urban green space.

Preferably, in the training sample set building module, for the features of high-resolution remote sensing images, some typical areas are used as sample areas, and a visual interpretation method is used to extract samples of high-resolution remote sensing images, and the vector files obtained by visual interpretation are extracted. Convert elements to raster to obtain label images, record real surface images of different types of vegetation locations in remote sensing images, and build a training sample dataset, as shown in Figure 3. Partial images of the dataset and corresponding label images. The multi-dimensional feature space building module first uses the data enhancement method to process the image to highlight the effective spectral information, and then uses the random cropping method to crop the label image and the data-enhanced remote sensing image, and perform feature based on the cropped label image and remote sensing image. Computational processing to construct a multidimensional feature space.

Preferably, the improved method of zero-filling image edges in the U-Net+ model building module is shown in Figure 4b; the improved method of batch normalization (BN) is to add batch normalization after the model convolution layer, and then The normalized data is input into the next layer of the convolutional layer of the model, so that the output of the upper layer network is connected to the result of the upsampling of the last layer, so as to prevent gradient disappearance and gradient explosion, and improve network training speed; regularization An improvement (dropout), shown in Figure 5b, is to add a dropout layer with a specific probability of dropping neurons after each deconvolution of the model.

Preferably, the structure of the multi-dimensional feature-based U-Net+ deep learning model is shown in Figure 6, and the training process is shown in Figure 7. In the U-Net+ model building module, the multi-dimensional feature data of the training sample is added to the U-net+ depth For training in the learning model, the remote sensing images, multi-dimensional feature data and corresponding label images in the multi-dimensional feature data of the training samples are input into the U-net+ deep learning model, and the U-net+ deep learning model extracts the features of the input image in the coding part. , restore its spatial position and resolution in the decoding part, and classify each pixel in the pixel classification layer to obtain category information; calculate the loss value of the predicted classification map and the input label map in the cross entropy function, and pass the loss value to Backpropagation is performed in the U-net+ deep learning model, and the parameters in the model are optimized layer by layer. When the loss value reaches a certain threshold, the training stops.

Preferably, the image post-processing module uses the fully connected CRFs image post-processing method to post-process the prediction results of the U-Net+ model, and processes the classification results in combination with the relationship between all the pixels in the original high-resolution remote sensing images, so as to make The prediction results are optimized, and the high-resolution remote sensing monitoring results of urban green space are obtained.

It should be pointed out that the above-mentioned specific embodiments can make those skilled in the art understand the present invention more comprehensively, but do not limit the present invention in any way. Therefore, although this specification has described the invention in detail with reference to the accompanying drawings and embodiments, those skilled in the art should understand that the invention can still be modified or equivalently replaced. The technical solutions and improvements of the spirit and scope shall be covered by the protection scope of the invention patent.

Claims (10)

1. an urban green space high score remote sensing monitoring method, is characterized in that, comprises the following steps: The training sample set construction step, according to the high-resolution remote sensing image features, select the sample area, and construct the training sample data set; The multi-dimensional feature space construction step is to perform data enhancement, random cropping and feature calculation processing on the training sample data set to construct a multi-dimensional feature space including vegetation features, spatial features, contrast features, texture features and phenological features; by constructing a normalized vegetation index Taking it as the vegetation feature of urban green space, construct the nDSM digital surface model to take it as the spatial feature of urban green space, calculate the local contrast feature map through AC algorithm and take it as the contrast feature of urban green space, and obtain the image texture feature map through the gray level co-occurrence matrix. Take it as the texture feature of urban green space, and add the winter images of the same period as the phenological feature of urban green space; The U-Net+ model construction step is based on the constructed multi-dimensional feature space, facing the characteristics of urban green space, using the image edge zero padding improvement method, the batch normalization processing improvement method and the regularization improvement method to improve the U-net model in turn, and establish a face-to-face Urban green space and U-Net+ deep learning model based on multi-dimensional features; then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, and after the training is completed, predict the spatial distribution of urban green space to obtain U-Net+ Model prediction results; The image post-processing step is to post-process the prediction results of the U-Net+ model to obtain the high-resolution remote sensing monitoring results of urban green spaces. 2. The high-resolution remote sensing monitoring method for urban green space according to claim 1, wherein, in the training sample set construction step, for high-resolution remote sensing image features, some typical areas are used as sample areas, and a visual interpretation method is used. Extracting samples of high-scoring remote sensing images, converting the vector files obtained by visual interpretation into elements to raster to obtain label images, recording real surface images of different types of vegetation locations in remote sensing images, and constructing training sample datasets; the multi-dimensional features The spatial construction step first uses the data enhancement method to process the image to highlight the effective spectral information, and then uses the random cropping method to crop the label image and the data-enhanced remote sensing image, and perform feature calculation processing based on the cropped label image and remote sensing image. Thereby, a multi-dimensional feature space is constructed. 3. urban green space high score remote sensing monitoring method according to claim 1 and 2, is characterized in that, the batch normalization processing improvement mode in described U-Net+ model building step is to add batch normalization after model convolution layer. Normalization processing, and then input the normalized data into the next layer of the model convolution layer; the regularization improvement method is to add dropout with a specific probability of dropping neurons after each deconvolution of the model. Floor. 4. The high-resolution remote sensing monitoring method for urban green space according to claim 3, wherein in the U-Net+ model building step, the multi-dimensional feature data of the training sample is added to the U-net+ deep learning model for training, which is to The remote sensing images, multi-dimensional feature data and corresponding label images of the training samples are input into the U-net+ deep learning model. The U-net+ deep learning model extracts the features of the input image in the encoding part, and restores its spatial position and resolution in the decoding part. The pixel classification layer classifies each pixel to obtain category information; calculates the loss value in the cross entropy function between the predicted classification map and the input label map, and transfers the loss value to the U-net+ deep learning model for backpropagation. The layer optimizes the parameters in the model, and when the loss value reaches a certain threshold, the training stops. 5. The high-resolution remote sensing monitoring method for urban green space according to claim 1 or 2, wherein the image post-processing step utilizes a fully connected CRFs image post-processing method to perform post-processing on the U-Net+ model prediction results, combined with The relationship between all pixels in the original high-resolution remote sensing image is used to process the classification results, so as to optimize the prediction results and obtain the high-resolution remote sensing monitoring results of urban green space. 6. An urban green space high score remote sensing monitoring system, characterized in that it comprises a training sample set building module, a multi-dimensional feature space building module, a U-Net+ model building module and an image post-processing module that are connected in sequence, and the training sample set builds The module is connected to the U-Net+ model building module, The training sample set building module selects a sample area according to the features of high-resolution remote sensing images, and constructs a training sample data set; The multi-dimensional feature space building module performs data enhancement, random cropping and feature calculation processing on the training sample data set of the training sample set building module, and constructs a multi-dimensional feature space including vegetation features, spatial features, contrast features, texture features and phenological features. ; By constructing the normalized vegetation index and taking it as the vegetation feature of urban green space, constructing nDSM digital surface model as the spatial feature of urban green space, calculating the local contrast feature map by AC algorithm and taking it as the contrast feature of urban green space, through grey The degree co-occurrence matrix is used to obtain the image texture feature map and use it as the texture feature of urban green space, and add the winter images of the same period to use it as the phenological feature of urban green space; The U-Net+ model building module, based on the multi-dimensional feature space constructed by the multi-dimensional feature space building module, is oriented towards the features of urban green spaces, and sequentially uses the image edge zero-filling improvement method, the batch normalization processing improvement method and the regularization improvement method to improve. U-net model, establish a U-Net+ deep learning model based on multi-dimensional features for urban green space; then add the multi-dimensional feature data of the training samples to the U-net+ deep learning model for training, and after the training is completed, the urban green space is distributed. to obtain the prediction results of the U-Net+ model; The image post-processing module performs post-processing on the prediction results of the U-Net+ model to obtain high-resolution remote sensing monitoring results of urban green spaces. 7. The high-resolution remote sensing monitoring system for urban green space according to claim 6, wherein the training sample set building module, for high-resolution remote sensing image features, takes part of typical areas as sample areas, and uses a visual interpretation method. Extracting samples of high-scoring remote sensing images, converting the vector files obtained by visual interpretation into elements to raster to obtain label images, recording real surface images of different types of vegetation locations in remote sensing images, and constructing training sample datasets; the multi-dimensional features The spatial building module first uses the data enhancement method to process the image to highlight the effective spectral information, and then uses the random cropping method to crop the label image and the data-enhanced remote sensing image, and perform feature calculation processing based on the cropped label image and remote sensing image. Thereby, a multi-dimensional feature space is constructed. 8. The high-resolution remote sensing monitoring system for urban green space according to claim 6 or 7, wherein the batch normalization processing improvement method in the U-Net+ model building module is to add batch normalization after the model convolution layer. Normalization processing, and then input the normalized data into the next layer of the model convolution layer; the regularization improvement method is to add a certain probability of discarding neurons after each deconvolution of the model. dropout layer. 9. The high-resolution remote sensing monitoring system for urban green space according to claim 8, characterized in that, in the U-Net+ model building module, the multi-dimensional feature data of the training sample is added to the U-net+ deep learning model for training. The remote sensing images, multi-dimensional feature data and corresponding label images of the training samples are input into the U-net+ deep learning model. The U-net+ deep learning model extracts the features of the input image in the encoding part, and restores its spatial position and resolution in the decoding part. The pixel classification layer classifies each pixel to obtain category information; calculates the loss value in the cross entropy function between the predicted classification map and the input label map, and transfers the loss value to the U-net+ deep learning model for backpropagation. The layer optimizes the parameters in the model, and when the loss value reaches a certain threshold, the training stops. 10. The high-resolution remote sensing monitoring system for urban green space according to claim 6 or 7, wherein the image post-processing module utilizes a fully connected CRFs image post-processing method to post-process the U-Net+ model prediction results, combined with The relationship between all pixels in the original high-resolution remote sensing image is used to process the classification results, so as to optimize the prediction results and obtain the high-resolution remote sensing monitoring results of urban green space.

Images