CN109190666B - Flower image classification method based on improved deep neural network - Google Patents

Abstract

The invention provides a flower image classification method based on an improved deep neural network. The method adopts a migration learning method, and uses the InceptionV3 network trained on a large-scale data set for the classification of flower image data sets, and activates the function to improve. Experiments on the general Oxford flower-102 dataset show that the model has higher classification accuracy than traditional methods and ordinary convolutional neural networks in flower image classification tasks, and is more accurate than unimproved convolutional neural networks. The process accuracy rate reaches 81.32%, and the fine-tuning process accuracy rate reaches 92.85%.

Description

Flower Image Classification Method Based on Improved Deep Neural Network

technical field

The invention relates to a flower image classification method, in particular to a flower image classification method based on an improved deep neural network.

Background technique

The task of classifying pictures of flowers is a difficult task. The biggest difficulty comes from variation within and between classes. For example, some images from different categories have small changes compared to the categories themselves, and some small differences determine their different classifications. In addition, as a continuous growing, non-rigid plant, flowers can be deformed in many ways, so there are also great changes in classification. Many traditional methods are used in flower identification.

Traditional methods need to build a classifier for each flower class and acquire a large number of patterns to train these classifiers. In practice, many different types of flowers make work very difficult and boring. In typical flower images, partial occlusions, lighting, multiple events, etc., there are large variations in scale and perspective.

Due to the fact that traditional flower classification methods require a large amount of manual annotation information and insufficient feature information, the classification ability is limited. On the Oxford flow-ers-102 dataset, the accuracy of traditional flower classification methods is lower than 81%.

In recent years, deep learning has made breakthroughs in many fields. Convolutional Neural Networks (CNN) are widely used in image classification tasks because they are easy to learn high-level features of images.

However, the traditional convolutional neural network has the following disadvantages when performing flower image classification tasks: 1. The entire neural network needs to undergo a large number of nonlinear transformations, which is prone to overfitting. 2. The number of layers of the traditional neural network structure is not enough, and the extraction of image feature information is not comprehensive enough. 3. In the classic BP neural network, when the error is back-propagated, if there are too many layers, the phenomenon of gradient dispersion will occur. 4. Flower pictures have the characteristics of small intra-class differences, and some flowers of similar classes may not be well differentiated by the network.

SUMMARY OF THE INVENTION

In view of the above technical problems, the present invention provides a flower image classification method based on an improved deep neural network, which can effectively improve the ability of the neural network to extract feature information, reduce the overfitting of the network, and increase the classification ability of the network.

The technical scheme adopted in the present invention is:

An embodiment of the present invention provides a method for classifying flower images based on an improved deep neural network, including: training a basic InceptionV3 network on a large-scale data set to obtain a pre-training network; improving the pre-training network to obtain a suitable An improved network based on a data set for flower recognition, the data set includes a training data set and a test data set; the improved network is migrated to the training data set for migration training to obtain a network after migration training; The activation function of the latter network is modified to the Tanh-ReLU function after correction based on the Tanh and ReLU functions, and the network after the improved activation function is obtained; fine-tune the network after the improved activation function to the training data set, perform fine-tuning training, and obtain fine-tuning Trained network; feed the test dataset into the fine-tuned trained network to classify flower images.

Optionally, the improving the pre-training network to obtain an improved network suitable for the data set of flower recognition includes: deleting the last fully connected layer of the pre-training network, adding a layer of global average pooling layer, and a first fully connected layer is added after the global average pooling layer, and a second fully connected layer is added after the first fully connected layer, so as to obtain the improved network; wherein, the first fully connected layer It contains 1024 nodes, the activation function adopts Relu, and adopts Dropout processing, and the probability is set to 0.5; the activation function of the second fully connected layer adopts Softmax, and the output nodes are 102 types.

Optionally, migrating the improved network to the training data set for migration training, and obtaining the network after migration training includes: keeping the network weight of the original InceptionV3 part unchanged, and using the training data set to train the last 4 layers The parameters of the network, so as to obtain the network after migration training;

Among them, the optimizer RMSprop is used to train the parameters. During the training process, during the gradient descent, each batch contains 32 samples, and the number of iteration rounds is set to 30 rounds.

Optionally, the expression of the Tanh-ReLU function is:

Optionally, fine-tuning the network after the improved activation function to the training data set, and performing fine-tuning training to obtain the fine-tuned trained network includes: freezing the parameters of the first two initial blocks in the network after the improved activation function, so that The values of the parameters of the two initial blocks remain unchanged during training, and the parameters of the remaining layers are retrained by using the training data set, thereby obtaining a fine-tuned trained network;

Among them, the optimizer SGD is used to train the parameters, the learning rate is set to 0.001, the momentum parameter is set to 0.9, and the loss function uses the cross-entropy loss function. During the training process, when the gradient descends, each batch contains 32 samples, and the number of iteration rounds Set to 30 rounds.

Optionally, the dataset is processed by data augmentation.

Optionally, the processing of the data set by data augmentation includes:

Tilt the image at different angles, and rotate the image horizontally and vertically to increase the number of samples;

Randomly crop the image to 80% size and randomly scale it from 80% to 120% to increase the number of samples; and

Appropriately increase the Gaussian noise of the image.

The flower image classification method based on the improved deep neural network provided by the embodiment of the present invention adopts the transfer learning method, and the InceptionV3 network trained on the large-scale data set is used for the classification of the flower image data set, and the activation function in it is used for classification. Improve. Experiments on the general Oxford flower-102 dataset show that the model has higher classification accuracy than traditional methods and ordinary convolutional neural networks in flower image classification tasks, and is more accurate than unimproved convolutional neural networks. The process accuracy rate reaches 81.32%, and the fine-tuning process accuracy rate reaches 92.85%.

Description of drawings

1 is a schematic flowchart of a flower image classification method based on an improved deep neural network provided by an embodiment of the present invention;

Figure 2 is the Tanh function and the ReLU function curve;

Figure 3 is the Tanh-ReLU function and its derivative curve;

Figure 4 shows the curve of the correct rate of different activation functions for 102 types of flower classification with the number of iterations.

Detailed ways

In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a schematic flowchart of a flower image classification method based on an improved deep neural network according to an embodiment of the present invention. As shown in FIG. 1, the flower image classification method based on the improved deep neural network provided by the embodiment of the present invention includes the following steps:

S101, train the basic InceptionV3 network on a large-scale data set to obtain a pre-training network;

S102, improving the pre-training network to obtain an improved network suitable for a data set for flower recognition, where the data set includes a training data set and a test data set;

S103, migrating the improved network to the training data set for migration training to obtain the network after migration training;

S104, modifying the activation function of the network after the transfer training to a Tanh-ReLU function after correction based on the Tanh and ReLU functions, to obtain a network with an improved activation function;

S105, fine-tuning the network after the improved activation function to the training data set, and performing fine-tuning training to obtain a fine-tuning trained network;

S106. Input the test data set into the fine-tuned trained network to classify flower images.

Hereinafter, each of the above steps will be described in detail.

S101. Train the basic InceptionV3 network on a large-scale dataset to obtain a pre-trained network

In the embodiment of the present invention, the InceptionV3 network trained on a large-scale data set is used as a flower classification network, thereby obtaining a pre-trained network.

Each Inception structure used in the embodiment of the present invention improves three Inception modules on the basis of InceptionV2, as follows:

In the first Inception structure, each 5×5 convolution is replaced by two 3×3 convolutions.

In the second Inception structure, the n×n convolution is decomposed into the form of n×1 and 1×1 two-layer convolution. For a 17×17 network, n is finally chosen to be 7.

In the third Inception structure, the output of the convolution kernel group is expanded. This architecture is used in Coarsest Grid to facilitate the representation of high-dimensional images.

On the basis of V2, the InceptionV3 network model used in the embodiment of the present invention is improved as follows: the optimizer replaces SGD with RMSProp, adds the LSR layer after the category fully connected layer, and divides the 7×7 convolution kernel into three 3×3 volumes Nuclei instead.

S102 , improving the pre-training network to obtain an improved network suitable for a data set of flower recognition.

The flower classification experiment of the embodiment of the present invention needs to classify 102 types of flowers. In order to make the network suitable for flower classification, improve the network, the network improvement includes: deleting the last fully connected layer of the pre-training network, adding a global average pooling layer, in order to expand the receptive field, and in the global average pooling layer. The first fully connected layer is added after the first fully connected layer, and the second fully connected layer is added after the first fully connected layer, so as to obtain the improved network; wherein, the first fully connected layer contains 1024 nodes, and the activation function Relu is adopted, and Dropout processing is adopted, and the probability is set to 0.5 to prevent the network from overfitting; the activation function of the second fully connected layer adopts Softmax, and the output nodes are 102 types. The improved network structure is shown in Table 1.

Table 1 Improved network structure (315 layers)

As can be seen from Table 1, the network input is 299×299×3, which is the size of the input image. The output of the network is 1×1×102, corresponding to the probability value of each type of flower.

S103: Migrate the improved network to the training data set for migration training, and obtain a network.

This step includes: keeping the network weight of the original InceptionV3 part unchanged, and using the training data set to train the parameters of the network of the last 4 layers, so as to obtain the network after migration training. Since there are fewer training parameters, the more stable optimizer RMSprop is chosen. During the training process, during gradient descent, each batch contains 32 samples, and the number of iterations is set to 30.

S104, modify the activation function of the network after the transfer training to the one after correction based on the Tanh and ReLU functions Tanh-ReLU function, the network after the improved activation function is obtained.

The improved InceptionV3 neural network structure model proposed in the embodiment of the present invention is shown in Table 1 above, except that the activation function therein is changed to the Tanh-ReLU function after the correction of the Tanh and ReLU functions. The last layer is the fully connected layer, that is, the probability of flower classification is finally obtained. The biggest feature of the present invention is to combine the characteristics of soft saturation of the non-saturated function ReLU and the Tanh function. The Tanh function can be expressed as:

The ReLU function can be expressed as:

The respective function curves are shown in Figure 2.

It can be seen from Figure 2 that the ReLU function is all set to 0 when it is less than zero, and remains unchanged when it is greater than zero, which shows that the function has a faster convergence speed when it is greater than zero, but because all negative values are set to 0. As a result, node data may die irreversibly during training, disrupting the data flow. In the task of flower image classification, since flowers have the characteristics of small inter-class differences and the ReLU output has no negative values, the accumulated bias between activation layers will affect the classification effect.

Based on this, the present invention takes the part less than zero as the left part of the Tanh function, and the part greater than zero as the right half of the ReLU function, denoted as the Tanh-ReLU function. The expression for this function is:

The Tanh-ReLU function curve and its derivative curve are shown in Figure 3.

When the network uses the BP algorithm for backtracking, the error propagates from the output layer, and the first derivative of the current activation function and the value of the current neuron must be multiplied at each layer, ie Grad=Error×(Tanh-ReLU)'(x) ×x. The derivative of this function is basically the same as the ReLU function, so inheriting the advantages of ReLU can effectively alleviate the problem of gradient disappearance, so that the deep neural network can be trained directly in a supervised way without unsupervised layer-by-layer training.

When it is greater than zero, Tanh-ReLU inherits the advantages of ReLU, because it has a linearly unsaturated form, so it has a faster convergence form during gradient descent. When it is less than zero, it overcomes the disadvantage that ReLU may destroy the data manifold, and has the characteristics of soft saturation (fixed between -1 and 0) to improve the robustness to noise. For the flower image classification task, it can better classify the characteristics of flower images with small differences between classes.

Compared with the ReLU function, Tanh-ReLU has the following two characteristics: 1. Its activation value is negative at x < 0, and the derivative is not 0. Because when the input of the ReLU function is negative, the output of its derivative will become 0, which will cause the problem of neuron death. The Tanh-ReLU function can improve this problem and make the negative input part exhibit a soft saturation characteristic, which can save noise robustness. 2. You can make the output mean 0. All outputs of ReLU are non-negative, so the output mean must be non-negative, which will cause the network to have a mean shift (Mean Shift), which may not converge when training some ultra-deep networks. The Tanh-ReLU function fixes this so that the mean may be 0.

To sum up, according to the characteristics of flower pictures, the present invention changes the activation function in the network structure in Table 1 to the Tanh-ReLU function to generate an improved InceptionV3 network structure.

S105. Fine-tune the network after the improved activation function to the training data set, perform fine-tuning training, and obtain fine-tuning The trained network.

This step includes: freezing the parameters of the first two initial blocks in the network obtained in S104 so as to keep their values unchanged during training, and retraining the parameters of the remaining layers by using the training data set. Due to the large number of training parameters, the optimizer SGD with a faster convergence speed is selected, where the learning rate is set to 0.001, the momentum parameter is set to 0.9, and the loss function uses the cross-entropy loss function. The number of iterative rounds is set to 30 rounds. During gradient descent, each batch contains the number of samples and the number of iteration rounds with the transfer training process.

Further, the data sets used in the embodiments of the present invention are processed through data augmentation. Enhanced processing can include:

Tilt the image at different angles, and rotate the image horizontally and vertically to increase the number of samples;

Randomly crop the image to 80% size and randomly scale it from 80% to 120% to increase the number of samples; and

Appropriately increase the Gaussian noise of the image.

【Example】

The advantages of the flower image classification method provided by the embodiments of the present invention are described below through experiments.

【Experiment and Analysis】

lab environment

The software and hardware experimental environment used in this experiment is shown in Table 2.

Table 2 Experimental software and hardware environment

Under the Linux system, this experiment uses the Keras deep learning framework based on TensorFlow to train and test flower pictures.

This experiment uses the Oxford flower-102 public dataset, which is from the flower image database created by the Visual Geometry Group of Oxford University. Contains 102 flower categories, each category has between 40 and 258 pictures, a total of 8189 pictures. The database also takes into account all the difficulties in the field of image recognition, such as illumination changes, visual changes, complex backgrounds, many types of flowers and complex color changes. In addition, some different flowers are highly similar, so it is of great significance to the study of flower image classification.

data augmentation

The data augmentation method can greatly increase the sample size of the training data set and improve the generalization ability of the network model. In essence, the data augmentation method is the process of artificially increasing the sample size of the dataset through data transformation methods such as affine transformation.

There are only 8189 flower pictures in this database. For 102 kinds of flower classification tasks, there are only 80 pictures per category on average, and the amount of data of each category of flower pictures is still small, so data enhancement is required to fully meet the training requirements. network needs.

1. Considering the different directions of shooting flowers, ensure the invariance of rotation and tilt during image recognition, tilt the picture at different angles, and perform horizontal and vertical image rotation. Increase the sample size.

2. Considering that a certain part of the flower picture is also the flower species under the complex background, the picture is randomly cropped by 80% size and randomly scaled by 80% to 120% to increase the number of samples.

3. Considering the changes of rain, fog, snow and some lighting, the hue, brightness and saturation of the images obtained in different seasons or at different shooting times of the day have different changes, and Gaussian noise should be appropriately increased.

Through the above three data augmentation methods, the training data is continuously generated from the original data by the training data set generator until the target number of iterations is reached. The enhanced data can effectively reduce the network overfitting during the training process and increase the ability of the convolutional network to recognize flower images.

This database has 8189 images, of which 7169 images are used as training set and 1020 images are used as test set. The data set is expanded to 30 times the original size by data augmentation technology, which effectively avoids the overfitting of the network.

InceptionV3 network structure experiment based on Tanh-ReLU activation function

After image data enhancement is performed, the image is then preprocessed. Considering that the resolutions of 102 categories of flower images are not equal, all images are scaled to 299×299 pixels to fulfill the requirement of the unified input of the network. Considering that the picture pixels are from 0 to 255, the input calculation is more complicated, and the picture pixels are compressed from 0 to 255 to -1 to 1 to simplify the input of the network.

Considering that there are too few training samples and more neurons need to be suppressed, when training on the database Oxfordflower-102, the Dropout ratio is set to 0.5 to prevent overfitting.

In order to solve the fully connected layer, the final pooling layer chooses to use global average pooling, and the feature map of the last layer is average pooled for the entire image to form a feature point, and this feature point is composed of the final feature vector. Conducive to the final Softmax calculation.

The migration process is to fix the parameters of the original InceptionV3 network part unchanged, and train the parameters of the remaining top 4 layers. The optimizer adopts the RMSProp optimizer, and the loss function adopts the multi-category logarithmic loss function (categoricalcrossentropy). The batch size is set to 32, the number of iteration rounds is set to 30, and the number of iterations is 32×30=960 times.

The process of fine-tuning the network is to fix the parameters in the first two initial blocks of InceptionV3 in the fixed network, that is, to fix the parameters of the first 127 layers, and train the remaining top parameters. The optimizer is SGD, the learning rate is set to 0.0001, the momentum parameter is set to 0.9, the loss function, Batchsize, and the number of iterations are the same as the migration process settings.

Result analysis

In order to verify whether the improved activation function can improve the classification accuracy, this paper uses three activation functions of Tanh, ReLU, and Tanh-ReLU to classify 102 types of flower images. The accuracy of the migration process and fine-tuning process is shown in Table 3. Among them, the first 30 rounds are the migration process, and the last 30 rounds are the fine-tuning process. During the migration process, since the RMSProp optimizer needs to search for the initial learning rate first, and then decrease it by orders of magnitude, the accuracy rate fluctuates, but in general the Tanh-ReLU activation function is higher than the other two activation functions. . In the fine-tuning process, a relatively stable SGD optimizer is used, and only one sample is calculated at a time. It can be seen that the classification accuracy of the improved Tanh-ReLU activation function is improved, as shown in Table 3 below.

Table 3 Classification accuracy of 102 types of flowers with different activation functions

When conducting the 102-category flower image classification experiment, the Tanh function, the ReLU function and the improved Tanh-ReLU function were directly compared. Through the experimental comparison, the number of batches in the training set of the migration process and the fine-tuning process were both 32, and the iterative round book was 30 rounds, and the total number of iteration rounds is 60 rounds. Through the above experiments, it can be concluded that the improved activation function can not only improve the convergence speed of the network, but also improve the classification and recognition rate of flower pictures. It can be seen from Table 3 that during the migration process, the accuracy of the improved Tanh-ReLU function is 2.3% and 0.48% higher than that of the Tanh and ReLU activation functions, respectively, and the accuracy of the fine-tuning process is 2.08% and 1.32% higher, respectively.

When classifying 102 types of flowers, the accuracy rate is 92.85%. In the process of migration and fine-tuning, the flower images with high accuracy rate are analyzed. Most of the flowers with high classification accuracy are significantly different from other flowers in color or shape, as shown in Figure 4.

To sum up, the present invention aims at the incompleteness and shortcomings of extracting picture feature information when the traditional network performs the flower picture classification task, using the idea of migration learning, and adopts the InceptionV3 network architecture pre-trained on the ImageNet data set. In view of the fact that too many traditional network parameters are prone to over-fitting, data enhancement techniques are used to appropriately increase the data set. On the basis of the InceptionV3 network architecture, the activation function of the network is improved, and the Tanh-ReLU activation function is used. Experiments show that the model has a better classification effect than the unimproved network model, and is better than the traditional method and other deep neural network architectures. . The accuracy of the migration process reaches 81.32%, and the accuracy of the fine-tuning process reaches 92.85%. The accuracy of the improved method for flower image classification task and the feasibility of flower recognition are verified.

It should be noted that the method provided by the present invention can also be extended to similar fields for research. It is universal in botanical classification, but some in-depth research should be done with reference to the knowledge base of botanical experts. It can also be used as a reference for animal species research.

The above-mentioned embodiments are only specific implementations of the present invention, and are used to illustrate the technical solutions of the present invention, but not to limit them. Detailed description, those of ordinary skill in the art should understand: any person skilled in the art is within the technical scope disclosed by the present invention, and it can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, Or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (5)

1. A flower image classification method based on an improved deep neural network is characterized by comprising the following steps:

training a basic Inception V3 network on a large-scale data set to obtain a pre-training network; wherein, in Inception V3, RMSProp is used as an optimizer, an LSR layer is added behind a class full-link layer, and three 3 × 3 convolution kernels are adopted;

improving the pre-training network to obtain an improved network of a data set suitable for flower recognition, wherein the data set comprises a training data set and a testing data set;

migrating the improved network to the training data set to perform migration training to obtain a network after the migration training; keeping the network weight of the original InceptitionV 3 part unchanged, and training the parameters of the network of the last 4 layers by using the training data set so as to obtain the network after the migration training; using an optimizer RMSprop training parameter, and in the training process, when the gradient is reduced, each batch comprises 32 samples, and the number of iteration rounds is set to be 30;

modifying the activation function of the network after the migration training into a Tanh-ReLU function after correction based on the Tanh and ReLU functions to obtain the network after the activation function is improved;

fine-tuning the network with the improved activation function to the training data set, and performing fine-tuning training to obtain a network after fine-tuning training; and

inputting the test data set into the network after the fine tuning training so as to classify the flower images;

wherein the improving the pre-training network to obtain the improved network of the data set suitable for flower recognition comprises: deleting the last full-link layer of the pre-training network, adding a global average pooling layer, adding a first full-link layer after the global average pooling layer, and adding a second full-link layer after the first full-link layer, thereby obtaining the improved network; the first full-connection layer comprises 1024 nodes, the activation function adopts Relu and Dropout for processing, and the probability is set to be 0.5; and the activation function of the second full connection layer adopts Softmax, and the output node is of type 102.

2. The method of claim 1, wherein the expression of the Tanh-ReLU function is:

3. the method according to claim 1 or 2, wherein the fine tuning the network after improving the activation function to the training data set for fine tuning training, and obtaining the fine tuning trained network comprises: freezing parameters of the first two initial blocks in the network after improving the activation function, keeping the values of the parameters of the two initial blocks unchanged in training, and retraining parameters of the rest layers by using the training data set so as to obtain a network after fine tuning training;

the optimization method comprises the steps of training parameters by using an SGD (generalized Gaussian distribution) optimizer, setting the learning rate to be 0.001, setting the momentum parameter to be 0.9, using a cross entropy loss function as a loss function, and setting the number of iteration rounds to be 30 when a gradient is reduced in the training process, wherein each batch contains 32 samples.

4. The method of claim 1, wherein the data set is processed by data enhancement.

5. The method of claim 4, wherein processing the data set by data enhancement comprises:

the method comprises the following steps of inclining pictures at different angles, and performing horizontal and vertical image rotation to increase the number of samples;

randomly cropping 80% of the size of the picture and randomly scaling 80% to 120% of the picture to increase the number of samples; and

the gaussian noise of the picture is increased appropriately.

Images