CN107358260A - A kind of Classification of Multispectral Images method based on surface wave CNN - Google Patents

Abstract

The invention discloses a method for classifying multispectral images based on surface wave CNN. The multispectral images to be classified are input, the multispectral data are normalized to obtain a matrix, and the normalized matrix is obtained by taking the center pixel. block to obtain the training data set and test set; construct a classification model based on surface wave CNN; use the training data set to train the classification model; use the trained classification model to classify the test data set. The invention introduces a multi-scale depth filter, improves the classification accuracy of multi-spectral images, and can be used for object classification.

Description

A Multispectral Image Classification Method Based on Surface Wave CNN

technical field

The invention belongs to the technical field of image processing, and in particular relates to a multispectral image classification method based on surface wave CNN.

Background technique

Multispectral remote sensing is a remote sensing technology that uses sensors with more than two spectral channels to simultaneously image ground objects. It divides the electromagnetic wave information reflected by objects into several spectral segments for reception and recording. Multi-spectral remote sensing can not only distinguish ground objects according to the difference in image shape and structure, but also distinguish ground objects according to the difference in spectral characteristics, which expands the amount of remote sensing information. Both the multi-spectral photography used in aerial photography and the multi-spectral scanning used in land satellites can obtain remote sensing data of different spectral bands, and the images or data of sub-spectral segments can be obtained through photographic color synthesis or computer image processing, which is more abundant than conventional methods. The image also provides the possibility for the computer recognition and classification of ground object images.

The key to remote sensing image classification is the fusion of remote sensing data. The existing remote sensing image fusion technologies mainly include pixel-level fusion, feature-level fusion, and decision-level fusion.

Pixel-based image fusion refers to the combination of measured physical parameters, that is, fusion is performed directly on the collected raw data layer. It emphasizes the integration of different image information on the basis of pixels, and the need for basic geocoding, that is, the geometric registration of raster data, and the image pixel level on the premise of one-to-one correspondence between each pixel. Merge processing to improve the effect of image processing, so that image segmentation, feature extraction, etc. can be performed on a more accurate basis, and better image visual effects may be obtained.

Feature-based image fusion refers to the use of different algorithms to first extract features such as edge extraction and classification from various data sources for target recognition, that is, to extract feature information from the initial image—spatial structure information such as range and shape , field, texture, etc.; and then comprehensively analyze and fuse these feature information. These similar targets or areas from multiple sources are spatially one-to-one, but not pixel by pixel, and are assigned to each other, and then fused using statistical methods, neural networks, fuzzy stimulation and other methods for further evaluation.

Image fusion based on the decision-making layer refers to the fusion based on image understanding and image recognition. That is, fusion after the "feature extraction" and "feature recognition" processes. It is a high-level integration, often directly oriented to applications, and serving for decision support.

Since these feature extraction methods do not take into account the multi-scale, multi-direction, and multi-resolution characteristics of multispectral images, it is difficult to obtain high classification accuracy for multispectral images with complex backgrounds.

Contents of the invention

The technical problem to be solved by the present invention is to provide a multi-spectral image classification method based on surface wave CNN to improve the classification accuracy.

The present invention adopts following technical scheme:

A multi-spectral image classification method based on surface wave CNN, input the multi-spectral image to be classified, normalize the multi-spectral data to obtain a matrix, and take blocks from the center pixel of the normalized matrix to obtain training Data set and test set; construct a classification model based on surface wave CNN; use the training data set to train the classification model; use the trained classification model to classify the test data set.

Further, the following steps are included:

S1. Take the data of each band of the multispectral remote sensing data as a matrix, divide the data in each matrix minus the minimum value by the maximum value minus the minimum value, and multiply by 255 to obtain the matrix corresponding to each band after normalization , and then perform pixel-level fusion of the matrix data of each band, and the numerical interval [0,255] after normalization is obtained as a matrix and recorded as D1;

S2. Correspond the position of the marked part to D1 to take the block with the center pixel, remove the surrounding 32×32 block to represent the original element class value, and form the training data set, which is recorded as train_data; the data without the block is used as the test data Set, denoted as test_data;

S3. Select a seventeen-layer convolutional neural network, given the feature map of each layer, determine the filter size of the convolutional layer and initialize the filter randomly, construct a multi-scale depth filter with Surfacelet, and replace the convolutional neural network Randomly initialized filters in the convolutional layer of the network to obtain a classification model based on multi-scale depth filters;

S4. The training data set train_data is used as the input of the classification model, and the category of each pixel in the training data set train_data is used as the output of the classification model. By solving the error between the above-mentioned category and the correct category manually marked and reverse the error Propagate and optimize the network parameters of the classification model to obtain the trained classification model;

S5. The test data set test_data is used as the input of the trained classification model, and the output of the trained classification model is the classification category obtained by classifying each pixel in the test data set.

Further, in step S1, the rules for normalizing spectral images of each band are as follows:

Wherein, n _i is the minimum value in the band, m _i is the maximum value in the band, B _i is the data of each band, and i=1, 2, . . . , 10 are 10 bands.

Further, in step S1, the matrix D1 is a matrix D1 with a size of M1×M2×10, wherein M1 is the length of the multispectral image, and M2 is the width of the multispectral image.

Further, in step S2, the method of removing the minimum value, dividing by the maximum value, and then multiplying by 255 is used to normalize the pixel-based feature matrix F, and first obtain the matrix D1 based on each dimension of the pixel point The maximum value max(Di); each element in the pixel-based feature matrix Di is divided by the maximum value max(F), multiplied by 255, and the normalized feature matrix F1 is obtained.

Further, in step S3, the classification model based on the multi-scale depth filter sequentially includes from the first layer: input layer→multi-scale depth filter layer→pooling layer→ReLU→convolution layer→pooling layer→ ReLU→convolutional layer→pooling layer→ReLU→convolutional layer→pooling layer→ReLU→full connection layer→Dropout→full connection layer→Softmax classifier.

Further, the number of feature maps of the input layer in the first layer is 10, the number of feature maps of the multi-scale depth filter layer in the second layer is 9, and the downsampling size of the pooling layer in the third layer is 2. The number of feature maps of the convolutional layer in the fourth layer is 64, and the filter size is 3.

Further, the downsampling size of the pooling layer in the fifth layer is 2, the number of feature maps of the convolutional layer in the sixth layer is 128, the filter size is 3, and the downsampling size of the pooling layer in the seventh layer is The sampling size is 2, the number of feature maps of the eighth convolutional layer is 256, and the filter size is 3.

Further, the downsampling size of the pooling layer in the ninth layer is 2, the number of feature maps in the fully connected layer in the tenth layer is 1000, and the number of feature maps in the fully connected layer in the eleventh layer is 500 , the number of feature maps of the Softmax classifier in the twelfth layer is 17.

Compared with the prior art, the present invention has at least the following beneficial effects:

The polarimetric SAR image classification method based on the multi-scale depth filter of the present invention is characterized in that, the multi-spectral image to be classified is input, the multi-spectral data is normalized to obtain a matrix, and the normalized matrix is processed by the central pixel Take the block of points to get the training data set and test set; construct the classification model based on multi-scale depth filter; use the training data set to train the classification model; use the trained classification model to classify the test data set, because the pixel The level features are extended into image block features, which can obtain spectral information and spatial information at the same time, and introduce multi-scale depth filters into the convolutional neural network. Therefore, it can effectively capture and represent the singularity of smooth surface signals, and has multi-directional decomposition, various Properties such as anisotropy, high-efficiency tree-structured filter banks, complete reconstruction, and low redundancy are well suited for multispectral image processing.

Further, the data is normalized first, and then the data is cut into pieces and sent to the network for feature learning. It can make the data have uniform distribution characteristics before being sent to the network, and the values can be unified in a smaller range through normalization, which can speed up the calculation speed.

Furthermore, normalization can unify the statistical distribution of samples. First of all, the dimensions are unified. If the magnitudes of different data are too different, the changes in large numbers will cover up the changes caused by small numbers. Secondly, it can speed up the convergence speed. After normalization, the convergence speed is faster.

Furthermore, the usually obtained remote sensing image data is often disturbed by noise, and the external environment such as changes in illumination conditions will also affect the global characteristics of the image. In addition, the image also has redundant information that has nothing to do with the target characteristics. Therefore, the multi-scale transformation method can be used to decompose the remote sensing image, and the corresponding low-frequency and high-frequency sub-bands of the image can be obtained. The low-frequency sub-band retains the spatial position relationship in the image, and the high-frequency sub-band reflects the details and edge information of the image. The surface wave is used to process the normalized multispectral data, and the processed data is sent to the CNN network for feature learning. It can significantly improve the generalization ability of image classification.

In summary, the present invention introduces a multi-scale depth filter, which improves the classification accuracy of multi-spectral images and can be used for object classification.

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

Description of drawings

Fig. 1 is the realization flowchart of the present invention;

Fig. 2 is the manual marking diagram of the image to be classified in the present invention;

Fig. 3 is a diagram of classification results of images to be classified using the present invention.

detailed description

The present invention provides a method for classifying multispectral images based on surface wave CNN. The multispectral images to be classified are input, the multispectral data are normalized to obtain matrix D1, and the normalized matrix is divided into center pixels. Take the block to get the training data set train_data, and the data without taking the block as the test set; construct the classification model based on the multi-scale depth filter; use the training data set train_data to train the classification model; use the trained classification model to test the data set test_data classification.

Please refer to Fig. 1, the polarization SAR image classification method based on multi-scale depth filter of the present invention, comprises the following steps:

S1. Perform normalization processing on the multispectral data.

The data of each band of the multispectral remote sensing data is regarded as a matrix, and the data in each matrix are normalized. The normalization rule is:

Subtract the minimum value and divide by the maximum value minus the minimum value, and then multiply by 255 to obtain the matrix corresponding to each band after normalization, and then perform pixel-level fusion of the matrix data of each band, and the value range after normalization is [0,255 ] Obtained matrix is recorded as D1;

The matrix D1 is a matrix D1 with a size of M1×M2×10, where M1 is the length of the multispectral image, and M2 is the width of the multispectral image.

The multispectral remote sensing images to be classified are selected from the B1, B2, B3, B4, B5, B6, B7, B10, B11 band multispectral images acquired by the Landsat_8 sensor near Paris, France, and the image size is 988×1160.

The rules for normalization of spectral images in each band are as follows:

Among them, n _i is the minimum value in the band, and m _i is the maximum value in the band.

S2. Take the block of the matrix D1 with the center pixel.

Correspond the position of the marked part to D1 to take the block with the center pixel. The surrounding 32×32 block represents the original element class label value, which constitutes the training data set, which is recorded as train_data; the data without the block is used as the test data set, which is recorded as test_data.

To normalize the pixel-based feature matrix F, the method of removing the minimum value, dividing by the maximum value, and then multiplying by 255 is used, that is, the maximum value max(Di) of each dimension of D1 based on the pixel point is first obtained; and then Each element in the pixel-based feature matrix Di is divided by the maximum value max(F) and multiplied by 255 to obtain the normalized feature matrix F1.

S3. Construct a classification model based on the multi-scale depth filter.

S31. Select a layer consisting of input layer→convolution layer→pooling layer→ReLU→convolution layer→pooling layer→ReLU→convolution layer→pooling layer→ReLU→convolution layer→pooling layer→ReLU→full connection Layer→Dropout→Fully connected layer→Softmax classifier consists of a 17-layer convolutional neural network, given the feature map of each layer, and determining the filter size of the convolutional layer and randomly initializing the filter;

S32. Use Surfacelet to construct a multi-scale depth filter, and replace the randomly initialized filter in the convolutional layer of the convolutional neural network to obtain a classification model based on the multi-scale depth filter: input layer → multi-scale depth filter layer → Pooling layer→ReLU→convolution layer→pooling layer→ReLU→convolution layer→pooling layer→ReLU→convolution layer→pooling layer→ReLU→full connection layer→Dropout→full connection layer→Softmax classifier composition The 17-layer convolutional neural network structure.

The parameters of the classification model based on the multi-scale depth filter are as follows:

For the first layer input layer, set the number of feature maps to 10;

For the second layer of multi-scale depth filter layer, set the number of feature maps to 9;

For the third layer pooling layer, set the downsampling size to 2;

For the fourth convolutional layer, set the number of feature maps to 64 and set the filter size to 3;

For the 5th layer pooling layer, set the downsampling size to 2;

For the 6th convolutional layer, set the number of feature maps to 128, and set the filter size to 3;

For the 7th layer pooling layer, set the downsampling size to 2;

For the 8th convolutional layer, set the number of feature maps to 256 and set the filter size to 3;

For the 9th layer pooling layer, set the downsampling size to 2;

For the 10th fully connected layer, set the number of feature maps to 1000;

For the 11th fully connected layer, set the number of feature maps to 500;

For the 12th layer Softmax classifier, set the number of feature maps to 17.

The second layer filters the input data of the first layer. The purpose of setting the maximum pooling is to reduce the dimension without losing the main feature information. As the number of layers deepens, more features can be extracted by increasing the number of feature maps. Through multiple experiments, the experimental results of setting parameters in this way are better.

S4. Using the training data set to train the classification model to obtain a trained classification model.

The training data set train_data is used as the input of the classification model, and the category of each pixel in the training data set train_data is used as the output of the classification model. By solving the error between the above-mentioned categories and the correct category manually marked and backpropagating the error, To optimize the network parameters of the classification model, the trained classification model is obtained, and the correct class label of manual marking is shown in Figure 2.

S5. Using the trained classification model to classify the test data set.

The test data set test_data is used as the input of the trained classification model, and the output of the trained classification model is the classification category obtained by classifying each pixel in the test data set.

Example

1. Simulation conditions:

The hardware platform is: Intel(R) Xeon(R) CPU E5650@2.13GHz Multi-spectral image classification method based on multi-scale depth filter

Graphics card: Quadro K2200/PCIe/SSE2, 2.40GHz multi-spectral image classification method based on multi-scale depth filter

Memory: 8G

The software platform is: Caffe.

2. Simulation content and results:

Use the method of the present invention to carry out experiments under the above-mentioned simulation conditions, that is, randomly select 5% of the marked pixels from each category of multispectral data as training samples, and use the whole picture as test data to obtain the classification results as shown in Figure 3 .

It can be seen from Figure 3 that the regional consistency of the classification results is good, and the edges of different regions are also very clear after division, and the detailed information is maintained.

Then reduce the training samples successively, so that the training samples account for 4%, 3%, and 2% of the total number of samples, and compare the classification accuracy of the test data set of the present invention with the convolutional neural network, and the results are as shown in Table 1:

Proportion of training samples convolutional neural network this invention 5% 61.04% 62.95% 4% 60.24% 61.68% 3% 59.84% 61.34% 2% 58.37% 61.14%

It can be seen from Table 1 that when the training samples account for 5%, 4%, 3%, and 2% of the total number of samples, the classification accuracy of the test data set of the present invention is higher than that of the convolutional neural network.

In summary, the present invention effectively improves the expression ability of image features and enhances the generalization ability of the model by introducing Surfacelet transformation into the convolutional neural network, so that it can still achieve high classification accuracy.

The above content is only to illustrate the technical idea of the present invention, and cannot limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed in the present invention, all fall into the scope of the claims of the present invention. within the scope of protection.

Claims (9)

1. A method for classifying multispectral images based on surface wave CNN, characterized in that, the multispectral images to be classified are input, the multispectral data are normalized to obtain a matrix, and the normalized matrix is processed with the center pixel Take the block to get the training data set and test set; construct the classification model based on surface wave CNN; use the training data set to train the classification model; use the trained classification model to classify the test data set. 2. a kind of multispectral image classification method based on surface wave CNN according to claim 1, is characterized in that, comprises the following steps: S1. Take the data of each band of the multispectral remote sensing data as a matrix, divide the data in each matrix minus the minimum value by the maximum value minus the minimum value, and multiply by 255 to obtain the matrix corresponding to each band after normalization , and then perform pixel-level fusion of the matrix data of each band, and the numerical interval [0,255] after normalization is obtained as a matrix and recorded as D1; S2. Correspond the position of the marked part to D1 to take the block with the center pixel, remove the surrounding 32×32 block to represent the original element class value, and form the training data set, which is recorded as train_data; the data without the block is used as the test data Set, denoted as test_data; S3. Select a seventeen-layer convolutional neural network, given the feature map of each layer, determine the filter size of the convolutional layer and initialize the filter randomly, construct a multi-scale depth filter with Surfacelet, and replace the convolutional neural network Randomly initialized filters in the convolutional layer of the network to obtain a classification model based on multi-scale depth filters; S4. The training data set train_data is used as the input of the classification model, and the category of each pixel in the training data set train_data is used as the output of the classification model. By solving the error between the above-mentioned category and the correct category manually marked and reverse the error Propagate and optimize the network parameters of the classification model to obtain the trained classification model; S5. The test data set test_data is used as the input of the trained classification model, and the output of the trained classification model is the classification category obtained by classifying each pixel in the test data set. 3. a kind of multispectral image classification method based on surface wave CNN according to claim 2, is characterized in that, in step S1, the rule of each band spectral image normalization is as follows: <mrow><mi>t</mi><mi>r</mi><mi>a</mi><mi>i</mi><mi>n</mi><mo>_</mo><msub><mi>data</mi><mi>i</mi></msub><mo>=</mo><mfrac><mrow><mo>(</mo><msub><mi>B</mi><mi>i</mi></msub><mo>-</mo><msub><mi>n</mi><mi>i</mi></msub><mo>)</mo></mrow><msub><mi>m</mi><mi>i</mi></msub></mfrac><mo>*</mo><mn>255</mn></mrow> Wherein, n _i is the minimum value in the band, m _i is the maximum value in the band, B _i is the data of each band, and i=1, 2, . . . , 10 are 10 bands. 4. The multispectral image classification method based on surface wave CNN according to claim 2, wherein in step S1, the matrix D1 is a matrix D1 with a size of M1×M2×10, wherein M1 is the length of the multispectral image, and M2 is the width of the multispectral image. 5. A kind of multispectral image classification method based on surface wave CNN according to claim 2, it is characterized in that, in step S2, adopt to go minimum value, divide by maximum value, multiply by 255 methods again, to pixel-based The feature matrix F of the point is normalized, and the maximum value max(Di) of each dimension based on the pixel point of the matrix D1 is first obtained; then each element in the feature matrix Di based on the pixel point is divided by the maximum value max (F), multiplied by 255 to obtain the normalized feature matrix F1. 6. a kind of multispectral image classification method based on surface wave CNN according to claim 2, is characterized in that, in step S3, described classification model based on multi-scale depth filter comprises successively from the first layer: input Layer→Multi-scale depth filter layer→Pooling layer→ReLU→Convolutional layer→Pooling layer→ReLU→Convolutional layer→Pooling layer→ReLU→Convolutional layer→Pooling layer→ReLU→Fully connected layer→Dropout → Fully connected layer → Softmax classifier. 7. a kind of multi-spectral image classification method based on surface wave CNN according to claim 6, is characterized in that, the feature map number of the input layer described in the first layer is 10, and the multi-scale depth filtering described in the second layer The number of feature maps in the filter layer is 9, the downsampling size of the pooling layer in the third layer is 2, the number of feature maps in the convolution layer in the fourth layer is 64, and the filter size is 3. 8. a kind of multispectral image classification method based on surface wave CNN according to claim 6, is characterized in that, the downsampling size of the pooling layer described in the fifth layer is 2, the convolutional layer of the sixth layer The number of feature maps is 128, the filter size is 3, the downsampling size of the pooling layer in the seventh layer is 2, the number of feature maps in the convolution layer in the eighth layer is 256, and the filter size is 3. 9. A kind of multi-spectral image classification method based on surface wave CNN according to claim 6, is characterized in that, the downsampling size of the pooling layer described in the ninth layer is 2, the fully connected layer of the tenth layer The number of feature maps is 1000, the number of feature maps of the fully connected layer in the eleventh layer is 500, and the number of feature maps of the Softmax classifier in the twelfth layer is 17.