CN115797710A - Neural network image classification performance improving method based on hidden layer feature difference - Google Patents

Abstract

The invention discloses a neural network image classification performance improvement method based on hidden layer feature difference, relates to the technical field of image data processing, including: constructing a heterogeneous combination set composed of binary groups, each heterogeneous sample binary group includes two Heterogeneous images, and the category labels of the two heterogeneous images; construct a weight-sharing two-way network; input the heterogeneous combination set into the weight-sharing two-way network, and obtain the output feature differences of several hidden layers; according to the cross-entropy loss and hidden layer output features difference, get the combined loss function, which includes the reciprocal of the hidden layer output feature difference; according to the heterogeneous combination set and the combined loss function, the end-to-end training weights share the two-way network; take any weight from the trained weight sharing two-way network One branch, and input the test image dataset, and output the image classification result. The invention improves the classification accuracy of the model, and can effectively restrain the situation that the characteristics of the hidden layer of samples of different categories are similar due to the degradation of the network.

Description

Performance improvement method of neural network image classification based on hidden layer feature difference

technical field

The present invention relates to the technical field of image data processing, in particular to a neural network image classification performance improvement method based on hidden layer feature differences.

Background technique

As the cost of data generation, acquisition, and transmission decreases, data continues to accumulate. At the same time, due to the widespread use of mobile devices such as mobile phones and tablets, and the widespread deployment of monitoring equipment, a large amount of image data is generated. Image recognition technology has been gradually applied in more and more fields, at the same time, the accuracy, reliability and real-time requirements of image recognition are becoming more and more stringent. As an important direction of machine learning, neural network has shown significant advantages in natural language, image and other fields. Especially in the basic and important application field of image classification, great development has been achieved. And thanks to the improvement of computing power and the improvement of optimization algorithms, the development of neural networks is showing a deeper and deeper trend.

When a neural network handles classification tasks, training the network to optimize network parameters mainly depends on the difference between the final output of the network and the true category of the image. Define the loss between the network output and the real category of the image, such as cross-entropy loss or likelihood loss, and train the network with the optimization goal of minimizing this loss. Since the traditional optimization goal mainly depends on the final output of the network, the entire network is a black box model that does not make full use of the hidden layer features in the network, and as the depth of the network increases, each hidden layer shrinks the input space and the degree of folding increases. Large, resulting in images of different categories having similar features in some hidden layers, making the training of the network more difficult. [DOI:10.1109/CVPR.2016.90] shows that when the network depth exceeds a certain threshold, the network performance will be degraded. affect the accuracy of image classification.

Contents of the invention

The purpose of the present invention is to provide a neural network image classification performance improvement method based on the difference in hidden layer features to solve the problems of high training difficulty and low classification accuracy caused by similar features of different types of images in the hidden layer.

The technical scheme that the present invention takes is as follows:

A neural network image classification performance improvement method based on hidden layer feature differences, comprising the following steps:

S1. Preprocess the training image data set, combine all heterogeneous samples, and construct a heterogeneous combination set composed of all heterogeneous sample pairs, each heterogeneous sample pair includes two heterogeneous images, and the two heterogeneous images. category label;

S2. Construct a two-way weight sharing network. The two-way weight sharing network includes two juxtaposed branches with the same structure and sharing weights. Each branch uses resnet34 to remove the backbone network after the residual connection, including: 1 A convolution layer with a stride of 2, a convolution kernel size of 7*7, and an output channel of 64; a pooling layer with a stride of 2; 6 strides of 1, and a convolution kernel size of 3*3 , a convolutional layer with an output channel of 64; a convolutional layer with a stride of 2, a convolutional kernel size of 3*3, an output channel of 128, 7 strides of 1, and a convolutional kernel size of 3*3 , a convolution layer with an output channel of 128; a convolution layer with a stride of 2, a convolution kernel size of 3*3, and an output channel of 256; 11 strides of 1, and a convolution kernel size of 3*3 , a convolutional layer with an output channel of 256; 6 convolutional layers with a step size of 1, a convolution kernel size of 3*3, and an output channel of 512; an average pooling layer; and a linear layer.

S3. For each heterogeneous sample pair in the heterogeneous combination set, input the two heterogeneous images in the heterogeneous sample pair into the two branches of the weight sharing two-way network, according to the same structure and level of the two branches The corresponding features of the hidden layer output of the two branches are obtained, and the difference of the hidden layer output features of the two branches with the same structure and level is obtained;

S4. According to the cross-entropy loss of the two branches in step S3, and the hidden layer output feature differences of all levels, define a combined loss function based on the hidden layer feature difference, including the reciprocal of the hidden layer output feature difference;

S5. According to the heterogeneous combination set and the combined loss function based on the difference in hidden layer features, the end-to-end training weights share the two-way network;

S6. Take any branch from the trained weight sharing two-way network as the final neural network image classification prediction model;

S7. Input the test image data set into the neural network image classification prediction model, and output the image classification result.

In a preferred embodiment of the present invention, in step S1, the process of preprocessing the training image dataset specifically includes: combining all categories in the training image dataset, traversing all category combinations, The Cartesian product of the image sets of each category is merged into the heterogeneous combination set.

In a preferred embodiment of the present invention, in step S3, the two heterogeneous images corresponding to the two branches in the heterogeneous sample binary group are respectively input1 and input2, then the i-th hidden layer obtained in the two branches respectively The features of are recorded as Hidden ⁱ (input1) and Hidden ⁱ (input2), respectively, and the output feature difference of the i-th hidden layer is:

Dis ⁱ (input1,input2) = || Hidden ⁱ (input1)-Hidden ⁱ (input2) || ₂ .

In a preferred embodiment of the present invention, in step S4, the combined loss function based on the hidden layer feature difference includes all hidden layer feature differences and the cross-entropy loss between the final output of the network and the label, specifically:

Loss = ∑1/Dis ⁱ + cross_entropy(output1,label1) + cross_entropy(output2,label2).

Among them, Dis ⁱ is the output feature difference of the i-th hidden layer, input1 and input2 are two heterogeneous images, label1 and label2 are the labels corresponding to the two heterogeneous images, and the function cross_entropy(x,y) is the calculation result of x and y cross entropy.

In a preferred embodiment of the present invention, in step S5, the goal of training the weight sharing two-way network is to minimize the combined loss based on the difference in hidden layer features, and the basis for judging the minimum loss is that the training weights share the two-way The loss of the network has not dropped more than 1/10000 for ten consecutive rounds.

In a preferred embodiment of the present invention, in step S5, the training process of the weight sharing two-way network is: traversing the heterogeneous sample pairs in the heterogeneous combination set, and combining one of the heterogeneous images with the category label of the heterogeneous image Input to one branch of the weight-sharing two-way network, input another heterogeneous image and the category label of the heterogeneous image to the other branch of the weight-sharing two-way network, using a combined loss based on the difference in hidden layer features function as the training loss for end-to-end training.

Compared with prior art, the beneficial effect of the present invention is:

Compared with the traditional neural network that only focuses on the final output of the network, the present invention utilizes the characteristics of the hidden layer in the middle of the network. When optimizing the network, it not only minimizes the cross-entropy loss between the model output and the label, but also maximizes the difference in hidden layer features of different types of pictures. , which improves the classification accuracy;

The reciprocal of the hidden layer feature difference is used as part of the loss. When the network has similar hidden layer features of different categories of samples due to degradation, it has a large gradient, and it will jump out of this situation when the gradient drops, which can effectively curb the network. degradation.

In order to make the above-mentioned objects, features and advantages of the present invention more comprehensible, the embodiments of the present invention will be described in detail below together with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

Fig. 1 is the overall flowchart of the neural network image classification performance improvement method based on hidden layer feature difference;

Figure 2 is an example diagram of data preprocessing;

Fig. 3 is a schematic structural diagram of a weight sharing two-way network;

Fig. 4 is a schematic diagram of the definition method of the combination loss function based on the difference of hidden layer features.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments.

Please refer to Fig. 1, the present invention discloses a neural network image classification performance improvement method based on hidden layer feature difference, comprising the following steps:

S1. Preprocessing the training image data set to construct a heterogeneous combination set. Each heterogeneous sample pair includes two heterogeneous images and category labels of the two heterogeneous images.

In order to make full use of the characteristics of different types of data in the hidden layer, the difference in the characteristics of heterogeneous data in the hidden layer is increased.

The present invention starts from the original training image data set, combines all heterogeneous samples, and obtains a set composed of all heterogeneous sample binary groups, that is, a heterogeneous combination set, and the construction method is as follows:

If the training image data set contains k categories, it is recorded as: C1, C2...Ck, and the specific method of constructing a heterogeneous combination set is:

(1-1) Combining pairs of k categories to get k*(k-1)/2 category combinations;

(1-2) For each combination (Ci, Cj), do direct product of all samples of Ci category and all samples of Cj category, the direct product of all samples of Ci and all samples of Cj = { (s1,s2) | s1 ∈Ci , s2∈Cj }, merge the obtained direct product into the heterogeneous combination set.

(1-3) Traverse k*(k-1)/2 combinations, and merge all obtained direct products into heterogeneous combination sets.

Take the cifar-10 dataset as an example, as shown in Figure 2. The training set of the cifar-10 dataset includes 50,000 images divided into ten categories. The ten categories are airplane (airplane), automobile (automobile), cat (cat), bird (bird), deer (deer), dog (dog), frog (frog), horse (horse), ship (ship) ) and trucks. Each category has 5000 images. First combine the 10 categories in pairs, you can combine the aircraft (airplane) with the remaining categories, then combine the automobile (automobile) with the remaining categories, and so on, and finally get (airplane, automobile), (airplane, cat), (car, cat) and other 10*9/2 combinations. For the two categories included in each combination, combine the 5000 pictures of the first category with the 5000 pictures of the second category, for example, category combination (aircraft, car), and combine the 5000 pictures of the aircraft category { Airplane 1, Airplane 2...Aircraft 5000} are combined with 5000 pictures of the car category {Car 1, Car 2...Car 5000} to get (Aircraft 1, Car 1), (Aircraft 1, Car 2), (airplane 2, car 1), (aircraft 2, car 2) and other 5000*5000 combination results. Merge that combination set into a heterogeneous combination set. By analogy, 19*9/2 category combinations are traversed to obtain a sample combination set for each category combination, which is merged into a heterogeneous combination set to obtain the final result.

S2. Construct a weight sharing two-way network so that the neural network can use the hidden layer characteristics of heterogeneous data during the training process. The weight sharing two-way network includes two juxtaposed branches with the same structure and shared weights. Each branch Both use resnet34 to remove the backbone network after the residual connection, as shown in Figure 3.

Since the two branches share weights, when two heterogeneous images are input to the two branches respectively, the hidden layer features of the two heterogeneous images are extracted by the same neural network, so that the heterogeneous images under the same network can be obtained separately Hidden layer features.

S3. For each heterogeneous sample binary group in the heterogeneous combination set, input the two heterogeneous images in the heterogeneous sample binary group into the two branches of the weight sharing two-way network respectively, and the two heterogeneous images are in the respective branches In the forward propagation, and calculate the respective hidden layer features, according to the corresponding features of the hidden layer output of the two branches with the same structure and level, the difference of the hidden layer output features of the two branches with the same structure and level is obtained, as shown in Figure 4.

In the present invention, the difference between the hidden layer features uses the L2 distance of the same hidden layer features in the respective branches of different categories of data.

For example, if the two heterogeneous images corresponding to the two branches in the heterogeneous sample binary group are respectively input1 and input2, then the features of the i-th hidden layer obtained in the two branches are recorded as Hidden i (input1) and Hidden ⁱ (input1) and Hidden ⁱ (input2), the i-th hidden layer output feature difference is:

Dis ⁱ (input1,input2) = || Hidden ⁱ (input1)-Hidden ⁱ (input2) || ₂ .

S4, according to the cross-entropy loss of the two branches in step S3, and the hidden layer output feature difference of all levels, define the combined loss function based on the hidden layer feature difference, as shown in Figure 4;

The combined loss function based on the hidden layer feature difference includes all hidden layer feature differences and the cross-entropy loss between the final output of the network and the label, specifically:

Loss = ∑1/Dis ⁱ + cross_entropy(output1,label1) + cross_entropy(output2,label2).

This includes the inverse of the difference in the hidden layer output features.

The loss is negatively correlated with the feature difference of heterogeneous images, and positively correlated with the cross-entropy of the final output of the network and the label. Training the model to minimize this loss will make the final output of the images in the training set fit their true class labels, while increasing the difference in the hidden layer features of the network between images of different classes.

S5. According to the heterogeneous combination set and the combined loss function based on the difference in hidden layer features, the end-to-end training weights share the two-way network.

Among them, the goal of training weight sharing two-way network is to minimize the combined loss based on the difference in hidden layer features. By minimizing this loss, the network can expand the hidden layer between heterogeneous data while fitting the data category. characteristic difference.

S6. Take any branch from the trained two-way weight sharing network as the final neural network image classification prediction model.

S7. Input the test image data set into the neural network image classification prediction model, and output the image classification result.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (8)

1. A neural network image classification performance improvement method based on hidden layer feature differences, characterized in that, comprising the following steps: S1. Preprocess the training image data set, combine all heterogeneous samples, and construct a heterogeneous combination set composed of all heterogeneous sample pairs, each heterogeneous sample pair includes two heterogeneous images, and the two heterogeneous images. category label; S2. Construct a weight-sharing two-way network. The weight-sharing two-way network includes two juxtaposed branches with the same structure and shared weights. Each branch uses resnet34 to remove the backbone network after the residual connection; S3. For each heterogeneous sample pair in the heterogeneous combination set, input the two heterogeneous images in the heterogeneous sample pair into the two branches of the weight sharing two-way network, according to the same structure and level of the two branches The corresponding features of the hidden layer output of the two branches are obtained, and the difference of the hidden layer output features of the two branches with the same structure and level is obtained; S4. According to the cross-entropy loss of the two branches in step S3, and the hidden layer output feature differences of all levels, define a combined loss function based on the hidden layer feature difference, including the reciprocal of the hidden layer output feature difference; S5. According to the heterogeneous combination set and the combined loss function based on the difference in hidden layer features, the end-to-end training weights share the two-way network; S6. Take any branch from the trained weight sharing two-way network as the final neural network image classification prediction model; S7. Input the test image data set into the neural network image classification prediction model, and output the image classification result. 2. the neural network image classification performance improvement method based on hidden layer feature difference according to claim 1, it is characterized in that, in step S1, the process that the training image data set is carried out to preprocessing specifically comprises: all in the training image data set Combining categories, traversing all category combinations, performing a Cartesian product on the image sets of two categories in each combination, and merging the Cartesian product into heterogeneous combination sets. 3. The neural network image classification performance improvement method based on hidden layer feature differences according to claim 2, wherein each branch in the weight sharing two-way network includes: 1 convolution kernel in sequence A convolutional layer with a size of 7*7 and an output channel of 64; 1 pooling layer with a stride of 2; 6 convolutional layers with a kernel size of 3*3 and an output channel of 64; 8 convolutions The convolutional layer with a kernel size of 3*3 and an output channel of 128; 12 convolutional kernels with a size of 3*3 and a convolutional layer with an output channel of 256; 6 convolutional kernels with a size of 3*3 and an output channel of 512 convolutional layers; 1 average pooling layer; 1 linear layer. 4. The neural network image classification performance improvement method based on hidden layer feature difference according to claim 3, characterized in that, in step S3, the two heterogeneous images respectively corresponding to two branches in the heterogeneous sample binary group are respectively input1 and input2, then the features of the i-th hidden layer obtained in the two branches are respectively recorded as Hidden ⁱ (input1) and Hidden ⁱ (input2), and the output feature difference of the i-th hidden layer is: Dis ⁱ (input1,input2) = || Hidden ⁱ (input1)-Hidden ⁱ (input2) || ₂ . 5. The neural network image classification performance improvement method based on hidden layer feature difference according to claim 4, characterized in that, in step S4, the combined loss function based on hidden layer feature difference includes all hidden layer feature differences and the final output of the network The cross-entropy loss with labels, specifically: Loss = ∑1/Dis ⁱ + cross_entropy(output1,label1) + cross_entropy(output2,label2), Among them, Dis ⁱ is the output feature difference of the i-th hidden layer, input1 and input2 are two heterogeneous images, label1 and label2 are the labels corresponding to the two heterogeneous images, output1 and output2 are the two branches of the weight sharing two-way network respectively The final output of the function cross_entropy(x,y) is the cross entropy of x and y. 6. The neural network image classification performance improvement method based on hidden layer feature difference according to claim 5, characterized in that, in step S5, the goal of training weights sharing two-way network is to make the combination based on hidden layer feature difference The loss is minimized. 7. The neural network image classification performance improvement method based on hidden layer feature difference according to claim 6, characterized in that, the loss of the training weight sharing two-way network does not exceed 1/10000 for ten consecutive rounds of decline, then it is considered that based on The combined loss of the hidden layer feature difference is minimized. 8. The method for improving the performance of neural network image classification based on hidden layer feature differences according to claim 7, wherein in step S5, the training process of the weight sharing two-way network is: traversing the heterogeneous samples in the heterogeneous combination set A tuple, input one of the heterogeneous images and the category label of the heterogeneous image into a branch of the weight sharing two-way network, and input the other heterogeneous image and the category label of the heterogeneous image into the weight sharing two-way Another branch in the network is trained end-to-end using a combined loss function based on the difference in hidden layer features as the training loss.

Images