CN117011599A - Image detection method based on improved efficientnet model - Google Patents

Abstract

The application discloses an image detection method based on an improved efficientet model, which comprises preprocessing operation, a built neural network model and parameters required to be adjusted by the neural network model. The method comprises the following steps: the method comprises the steps of (1) preprocessing an image to be detected; (2) Building a neural network model based on an improved efficientnet model and training; (3) And (3) detecting and outputting the image to be detected by adopting the neural network model after training in the step (2). The method solves the problems of high hardware configuration, low classification accuracy, low running speed and the like of the current image classification neural network model.

Description

An image detection method based on improved efficientnet model

Technical field

The invention relates to the field of image detection, and in particular to an image detection method based on an improved efficientnet model.

Background technique

Image classification is a technology in the field of computer vision and is widely used in image recognition, target detection and behavior analysis.

At present, color method, support vector machine method, neural network method, etc. are usually used for image classification. The first two have many problems such as limited accuracy and inability to handle complex scenes. The image classification neural network model included in the final neural network method is only suitable for large-scale classification tasks, and has a large amount of calculation and number of parameters. For example, the patent application number 201810113878.4 is based on a convolutional network and recurrent neural network. The hyperspectral image classification method of the network mainly solves the problem of low accuracy of hyperspectral image classification in the existing technology. The specific steps of the present invention are as follows: (1) Construct a three-dimensional convolutional neural network; (2) Construct a recurrent neural network; (3) Preprocess the hyperspectral image matrix to be classified; (4) Generate a training data set and a test data set ; (5) Use the training data set to train the network; (6) Extract the spatial features and spectral features of the test data set; (7) Fusion of spatial features and spectral features; (8) Classify the test data set. The present invention introduces a three-dimensional convolutional neural network and a recurrent neural network to extract spatial features and spectral features of hyperspectral images, and fuses the two features for classification, which has the advantage of high accuracy for hyperspectral image classification problems.

It can be said that existing neural networks have some advantages, but when applied to various image classification tasks in real life, they are often abandoned due to disadvantages such as expensive hardware configuration, low classification accuracy, and slow running speed. Therefore, an image classification method that takes into account accuracy, running speed, and cost is needed.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings of the existing technology, provide an image detection method based on an improved efficientnet model, solve the shortcomings of image classification methods such as existing neural networks, and innovatively propose an improved efficientnet model algorithm to help Neural network technology can be applied to practical image classification tasks more efficiently and accurately.

In order to achieve the above purpose, the technical solution adopted by the present invention is: an image detection method based on an improved efficientnet model, which includes the following steps:

(1) Preprocess the image to be detected;

(2) Build a neural network model based on the improved efficientnet model and conduct training;

(3) Use the neural network model trained in step (2) to detect and output the image to be detected.

In step 2, the neural network model built based on the improved efficientnet model includes: stems used to extract shallow-level features of the image, MBConv blocks used to extract shallow-level features into deep-level features, and transform deep-level features into are the branches of the output and the classifier used to classify the output.

In step (1), preprocessing operations include image cropping, grayscale, equalization to enhance image features, median filtering to remove the influence of lighting, high-boosting filtering to separate the image from the background, and binary extraction of the visual part of the image. ization, image inpainting, tensorization into pytorch operations, and standardized operations to speed up training.

The stems used to extract shallow-level features of images include: convolution module, normalization operation module and swish function. The input image undergoes image dimensionality increase through the convolution module, and then performs batch normalization operation through the normalization operation module. , and finally output the shallow features of the image through the swish activation function.

The MBConv block used to extract shallow-level features into deep-level features includes: 1*1 convolution for dimensionality increase, k*k-dimensional depth separable convolution, SE module, and 1*1 dimensionality reduction convolution kernel. Dropout layer; the input image features first undergo 1*1 convolution to increase the dimension, then pass through the BN layer and Swish activation function, and then are sent to the k*k-dimensional depth separable convolution for processing, and then go through the BN layer and Swish activation again. The function is then sent to the SE module. After processing by the SE module, it is sent to the Dropout layer through 1*1 dimensionality reduction convolution and BN layer. After that, the Dropout output and the input of MBConv are added bitwise to obtain the output of the MBConv block.

The branches used to convert deep features into output include a 1*1 dimensionality reduction convolution kernel batch normalization operation module.

The classifier used to classify the output consists of a global average pooling layer, a random dropout layer, a fully connected layer, and a softmax activation function that sequentially connect the outputs.

The training of the neural network model based on the improved efficientnet model includes:

Establish training set, validation set and test set;

Then use the verification set for verification, import the verification set data and input it into the model obtained in this batch in turn, compare the current model output value and the real label value, and then calculate the accuracy of the current model on the verification set, as this batch The accuracy corresponding to the obtained internal parameters model is finally saved to an optimal parameter file.

Then conduct the second batch of training, use the training set again to train a model with new internal parameters, and compare and calculate the accuracy of the new model on the verification set as the corresponding internal parameters obtained in the second batch. Accuracy.

Then compare the accuracy of the second batch with the accuracy of the previous batch. If the accuracy is higher, the internal parameters of the second batch will be used to cover the optimal parameter file; if the accuracy is the same or lower, it is not correct. The optimal parameter file is updated.

Then follow the above operations and perform training updates repeatedly within the specified number of batches. Finally, the best internal parameters with the highest accuracy are obtained and the model training is completed.

Updating the internal parameters of the model based on the training set includes:

Preparatory work for parameter adjustment: import images according to the path where the images in the training set are saved and perform preprocessing operations on the images;

Internal parameter debugging: He initializes the built model to be improved, and then imports a picture of the training set into the improved neural network model to complete forward propagation;

Then select the cross-entropy loss function, and calculate the loss value by importing the output value of the model and the real label value into the cross-entropy loss function; then perform backpropagation based on the loss value to calculate the gradient of each layer, and finally update the neural network according to the gradient of each layer For the internal parameters of the model, the same operation is used to complete the images of the entire training set, thereby completing the internal parameter adjustment of the current batch.

The hyperparameters of the model are set as follows: the batch_number of the number of reads per batch is 8, the initial learning rate lr is 1e-3, the exponential decay rate betas for the gradient and gradient square are 0.9 and 0.999, and the denominator small smoothing term eps is 1e -8, weight decay weight_decay is 0, select True for the usage option of the improved version of amsgrad of Adam algorithm, and set the number of iterations of the training set num_epochs to 40.

Compared with the prior art, the advantages of the present invention are:

1) Preprocessing is performed before the picture set is imported into the neural network, which improves the quality of the picture set and helps the neural network better learn the key performance of the picture set, reducing the performance requirements of the network and improving the running speed and accuracy.

2) Measures such as early stopping strategy and improved model structure were adopted to reduce the operating parameters of the efficientnet model code and the video memory requirements during calculation, avoiding the configuration of high-performance graphics cards and other hardware conditions, saving costs and broadening the use of the model. Scenes.

3) Compared with the source code, the improved efficientnet model code has changes in branches, fully connected layers, etc., helping the model to perform 100% in small-scale actual classification tasks similar to large-scale actual classification tasks. Classification accuracy.

4) The improved efficientnet model code improves the construction principle and integrates scattered files to remove garbage time spent during runtime, helping the new model code to be applied to various practical classification tasks at a faster running speed.

Description of the drawings

The following is a brief description of the content expressed in each drawing of the specification of the present invention and the marks in the drawings:

Figure 1 is an overall flow chart of the present invention.

Figure 2 is an example diagram of an image classification task for bearing assembly quality detection according to the present invention.

Figure 3 is a diagram showing the results of the clipping operation of the example diagram of the present invention.

FIG. 4 is a result diagram of the grayscale operation of the example image of the present invention.

Figure 5 is a grayscale histogram of the grayscale operation of the example image of the present invention.

Figure 6 is a diagram showing the results of the equalization operation of the example diagram of the present invention.

Figure 7 is a grayscale histogram of the equalization operation of the example image of the present invention.

Figure 8 is a result diagram of the median filtering operation of the example diagram of the present invention.

FIG. 9 is a diagram showing the results of the high-boost filtering operation of the example diagram of the present invention.

Figure 10 is a result diagram of the binarization operation of the example diagram of the present invention.

Figure 11 is a diagram showing the results of the circle completion operation of the example diagram of the present invention.

Figure 12 is a result diagram of the standardization operation of the example diagram of the present invention.

Figure 13 is the change curve of the loss function of the training and verification sets of the efficientnet source code in the training of the present invention.

Figure 14 is the change curve of the loss function of the training and verification sets of the improved model code in the training of the present invention.

Figure 15 is the final accuracy of the efficientnet source code of the present invention.

Figure 16 The final accuracy of the improved model code of the present invention.

Figure 17 is a schematic diagram of the stem of the model;

Figure 18 is a schematic structural diagram of the MBConv block of the model;

Figure 19 is a schematic diagram of the branch structure of the model;

Figure 20 is a schematic diagram of the classifier;

Detailed ways

The specific implementation manner of the present invention will be further described in detail below by describing the preferred embodiment with reference to the accompanying drawings.

The present invention provides an image detection method based on an improved efficientnet model, which includes the following steps:

(1) Preprocess the image to be detected;

(2) Build a neural network model based on the improved efficientnet model and conduct training;

(3) Use the neural network model trained in step (2) to detect and output the image to be detected.

In step (1), preprocessing operations include image cropping, grayscale, equalization to enhance image features, median filtering to remove the influence of lighting, high-boosting filtering to separate the image from the background, and binary extraction of the visual part of the image. At least one, a combination or all of the following: image inpainting, tensorization into pytorch operation form, and normalization operations to speed up training.

In step 2, the neural network model built based on the improved efficientnet model includes: stems used to extract shallow-level features of the image, MBConv blocks used to extract shallow-level features into deep-level features, and transform deep-level features into are the branches of the output and the classifier used to classify the output.

As shown in Figure 17, the stems used to extract shallow-level features of images include: convolution module, normalization operation module and swish function. The input image goes through the convolution module to increase the image dimension, and then passes through the normalization operation module Perform batch normalization operation, and finally output the shallow features of the image through the swish activation function.

As shown in Figure 18, it is a schematic structural diagram of the MBConv block of the model; the model block consists of MBConv1 block with 2 convolution kernels of 3*3 and a stride of 1*1, and 3 convolution kernels of 3*3, MBConv6 block with a stride of 2*2, MBConv6 block with 3 convolution kernels of 5*5 and a stride of 2*2, MBConv6 with 5 convolution kernels of 3*3 and a stride of 2*2 block, an MBConv6 block consisting of 5 convolution kernels of 5*5 and a stride of 1*1, an MBConv6 block of 6 convolution kernels of 5*5 and a stride of 2*2, consisting of 2 convolution kernels The MBConv6 block is 3*3 with a stride of 1*1. The MBConv blocks used to extract shallow-level features into deep-level features include: 1*1 convolution for dimensionality increase, k*k-dimensional depth separable Convolution, SE module, 1*1 dimensionality reduction convolution and dropout layer; the input image features first undergo 1*1 convolution to increase the dimension, and then pass through the BN layer, Swish activation function and then are sent to k*k dimensions. After being processed by depth-separable convolution, it is sent to the SE module through the BN layer and Swish activation function. After the SE module is processed, it is sent to the Dropout layer after being processed by 1*1 dimensionality reduction convolution and BN layer, and then the Dropout is output. Bitwise addition to the input of MBConv results in the output of the MBConv block.

Figure 19 is a schematic diagram of the branch structure of the model. The branches used to convert deep-level features into output include 1*1 dimensionality reduction convolution and batch normalization operation modules.

Figure 20 is a schematic diagram of a classifier. The classifier used to classify the output includes a global average pooling layer, a random deactivation layer, a fully connected layer and a Softmax activation function that connect the outputs in sequence.

The training of the neural network model based on the improved efficientnet model includes:

Establish a training set, a verification set and a test set; (the training set, verification set and test set are all taken from the data set of bearing assembly quality, and the entire data set contains qualified bearings [uniformly distributed steel ball bearings - that is, the spacing of each steel ball within the error Within the range] and unqualified bearings [non-uniformly distributed steel ball bearings - that is, the distance between each steel ball is outside the error range, missing one steel ball bearing - that is, one steel ball is missing in the bearing, two steel balls are missing Bearing - that is, two situations in which two steel balls are missing in the bearing. Then, the data set is divided according to the ratio of 6:2:2, taking into account the goals of "reducing training time" and "improving test reliability")

The model is trained based on the images in the training set to complete the internal parameter adjustment of the model until the training of the entire training set is completed, and a model of the internal parameters of this batch is obtained;

Then use the verification set for verification, import the verification set data and input it into the model obtained in this batch in turn, compare the current model output value and the real label value, and then calculate the accuracy of the current model on the verification set, as this batch The accuracy corresponding to the obtained internal parameters model is finally saved to an optimal parameter file.

Then conduct the second batch of training, use the training set again to train a model with new internal parameters, and compare and calculate the accuracy of the new model on the verification set as the corresponding internal parameters obtained in the second batch. Accuracy.

Then compare the accuracy of the second batch with the accuracy of the previous batch. If the accuracy is higher, the internal parameters of the second batch will be used to cover the optimal parameter file; if the accuracy is the same or lower, it is not correct. The optimal parameter file is updated.

Then follow the above operations and perform training updates repeatedly within the specified number of batches. Finally, the best internal parameters with the highest accuracy are obtained and the model training is completed.

Updating the internal parameters of the model based on the training set includes:

Preparatory work for parameter adjustment: import images according to the path where the images in the training set are saved and perform preprocessing operations on the images;

Internal parameter debugging: He initializes the built model to be improved, and then imports a picture of the training set into the improved neural network model to complete forward propagation;

Then select the cross-entropy loss function, and calculate the loss value by importing the output value of the model and the real label value into the cross-entropy loss function; then perform backpropagation based on the loss value to calculate the gradient of each layer, and finally update the neural network according to the gradient of each layer For the internal parameters of the model, the same operation is used to complete the images of the entire training set, thereby completing the internal parameter adjustment of the current batch.

By improving the hyperparameters specified in the original paper and cooperating with internal parameter adjustments, the finally trained neural network model of the improved efficientnet model can be applied to image detection and classification. Its detection accuracy is high and the hardware cost is low. When applying, it only needs to The image to be detected is preprocessed and sent to the model, and the model can output classification detection results.

The following will take the image classification task of bearing assembly quality inspection as an example to explain in detail the specific principles and software code implementation of each module (it needs to be understood that the corresponding names indicated by the terms "Steel Ball 1" are based on the movement sequence of the steel balls). "qiu1", each term is only a simplified description to facilitate the description of the present invention, and therefore cannot be understood as a limitation of the present invention.):

An image detection method based on an improved efficientnet model, including preprocessing operations, built neural network models and parameters that need to be adjusted in the neural network model.

Preprocessing operations include cropping the image where the bearing is located, simplifying the grayscale of the image, enhancing the equalization of image features, median filtering to remove the impact of lighting, high-lift filtering to separate the steel ball from the background, and extracting the visual part of the steel ball. value, complete circles to repair complete steel ball graphics, tensorization into pytorch operation forms, and standardized operations to speed up training.

The built neural network model includes stems that extract shallow-level features of images, MBConv blocks that extract shallow-level features into deep-level features, branches that convert deep-level features into outputs, and classifiers that classify the outputs.

The parameters that need to be adjusted for the neural network model include preparatory work for parameter adjustment, internal parameters to calculate the optimal weight of the model, hyperparameters to improve the relevant performance of the model, and test set parameters to provide the basis for the final judgment.

As shown in Figure 1, the image detection method based on the improved efficientnet model includes image preprocessing, neural network model construction and neural network model parameter adjustment. Specifically, it includes the following steps:

Step 1: Load the picture sample set;

Step 2: Preprocess the sample;

Step 3: Build an improved efficientnet model;

Step 4: Debugging the internal parameters of the model;

Step 5: Hyperparameter debugging;

Step 6: Test the test set to obtain the accuracy of the model, and determine whether it is better than the accuracy of the existing technology? If so, determine the model; otherwise adjust the hyperparameters to re-obtain the model, test again, and loop until the result is yes.

As shown in Figure 2, it is an example of an image classification task for bearing assembly quality inspection.

Image preprocessing in this application includes the following steps:

Trim first, and use the mathematical relationship "modifying the coordinates of the lower left corner of the image is equal to modifying the coordinates of the center point" to call the custom function Center() through the Lambda function under the transforms class (this function moves the coordinates of the center point of the image to the specified position. The method is First get the length and width of the input image, and get the lower left corner coordinates of the new image based on "the new center point coordinates minus half the length and width of the image", and then get the new image based on the lower left corner coordinates plus the length and width of the input image. The coordinates of the upper right corner, input the coordinates of the lower left corner and upper right corner of the new image into the img.crop function [function to move the image according to the coordinates of the lower left corner and upper right corner] to realize the movement of the image, and then indirectly realize the movement of the center point) to modify the image Center point coordinates. Then use the CenterCrop function under the transforms class to start cropping the image from the new center point. Thus, the image containing only the bearing is cropped, and the cropped result is shown in Figure 3;

Then perform grayscale, and change the parameter num_output_channels of the Grayscale function under the transforms class to 1 to obtain the grayscale image of the bearing. The grayscale result is shown in Figure 4, and the grayscale histogram of the grayscale image is shown in Figure 5. shown, and then perform equalization. Call the custom function Equ() through the Lambda function under the transforms class (this function is configured to equalize the gray value, so that the image information is richer. The method is to use the ImageOps.equalize function, which uses information such as proportion to update The gray value range of each pixel point in the image achieves a uniform distribution of gray values) so that the gray values are evenly arranged and the image features are enhanced. The equalization result diagram is shown in Figure 6. The gray value of the equalization diagram The degree histogram is shown in Figure 7, and then median filtering is performed. Call the MedianFilter() function of the ImageFilter class through the Lambda function under the transforms class, and set the parameter size to 1 to remove interference factors such as lighting and shadows in the image. The results of median filtering are shown in Figure 8;

After the completion of median filtering, high boost filtering is performed. Call the custom function High() through the Lambda function under the transforms class (this function is configured to perform a convolution operation on the image, mainly to separate the bearings from the background. The method is to use the torch.ones function to create a 5*5 The convolution kernel implements a zero-filling convolution operation on the image when the convolution kernel center is 0.45), and cooperates with the final debugging parameters to obtain a highly improved filter image, which separates the steel balls and the background. The result of high-lift filtering is shown in Figure 9, and then binarized.

Binarization includes: calling the point() function through the Lambda function under the transforms class, so that the current pixel value greater than 10 is white, otherwise it is black, so that the visual part of the steel ball is extracted from the result. The result of the binarization is shown in Figure 10 As shown, then complete the circle. Call the custom function Circle() through the Lambda function under the transforms class (this function is configured to complete the complete steel ball image based on the simplified visible part of the steel ball. It mainly uses the cv2.findContours function to find the contours of each part, and based on The contour obtains the radius and center of the circle corresponding to the complete pattern of the steel ball here, and uses this information to draw a circle based on the original pattern to achieve only the image information of the complete steel ball). Through the combination of detecting the contour and drawing the circle function under the cv library, it is made possible by The visualization part of the steel ball can obtain the complete pattern of the steel ball. The completed circular result is shown in Figure 11, and then tensorization is performed. The image pixel values are normalized to floating point values of [0.0, 1.0], and the shape and format of the image are converted into a form that can be processed under the pytorch framework, and finally standardized. Through the Normalize function under the transforms class, combined with the mean and standard deviation calculated using the mean function and std function under each data set, the data becomes a standard normal distribution. The standardized result chart is shown in Figure 12.

This application needs to build a neural network model based on efficient NET. The neural network model includes: stems that extract shallow-level features of images, MBConv blocks that extract shallow-level features into deep-level features, and branches that convert deep-level features into outputs. Stem, a classifier that classifies the output. The specific structure diagram of each part is shown in Figure 17-20.

To build a neural network model, first build the stem of the model. By first using the custom Conv2d() function (this function creates the form of the image that can be used in the neural network operation, and then performs the convolution operation. Specifically, it includes obtaining the Conv2d form of the same size as the input image. According to this form, according to the convolution The mathematical formula of the product operation is used with the function of the math library to implement the convolution operation with specified parameters. This corresponds to the 3*3 convolution operation with a step size of 1) and the relevant parameters are used to implement the 3*3 convolution operation to increase the image dimension, and then The batch normalization operation of data is realized through the BatchNorm2d tool in the nn tool library under the torch library. Finally, the customized MemoryEfficientSwish() function is used (to avoid the Swish activation function compatibility issues and low memory caused by the old and new versions of pycharm to implement Swish efficiently. function. Specifically, it determines the version status by accessing the "presence or absence of SiLU function" and calls the swish function corresponding to the version. Low memory and high efficiency mainly save the final result of propagation separately in forward and reverse propagation. To avoid recalculation when the result value is needed later, and save running time) implement the swish activation operation and complete the construction of the stem of the model. See code attachment 1 for specific parameters; (the stem extracts the preliminary features of the image, such as edges, texture, etc.)

Then build the MBConv block of the model. It is necessary to build the SE module in advance. First, use the adaptive_avg_pool2d function of the functional function library in the nn tool library under the torch library to implement global average pooling. Use the custom Conv2d() function (this custom function creates images that can be The form of the neural network operation, and then perform the convolution operation. Specifically, it includes obtaining the Conv2d form of the same size as the input image. According to this form, according to the mathematical formula of the convolution operation and the function of the math library to implement the convolution operation of the specified parameters, This corresponds to the 1*1 convolution operation) (the purpose of the 1*1 convolution operation here is to reduce the dimension, which can greatly reduce the amount of calculation and better fit the complex relationship between channels) and implement it with related parameters For 1*1 convolution for dimensionality reduction, the Swish function is implemented efficiently through the custom MemoryEfficientSwish() function (to avoid the Swish activation function compatibility issues caused by old and new versions of pycharm and low memory. Specifically, by accessing the "SiLU function" ", thereby determining the version situation and calling the swish function corresponding to the version. Low memory and high efficiency mainly save the final result of propagation separately in forward and reverse propagation, avoiding recalculation when the result value is needed later, saving money Running time) (The activation function here introduces nonlinear transformation to help the SE module adaptively learn the importance of each pixel in the input feature map. The result after passing the activation function has important information (pixels in the area) point) has a higher weight value, which improves the accuracy of the later results) implement the swish activation function, and use the custom Conv2d() function with relevant parameters to implement 1*1 convolution for dimensionality enhancement (this custom function creates The image can be in the form of a neural network operation, and then perform a convolution operation. Specifically, it includes obtaining a Conv2d form of the same size as the input image. According to this form, the mathematical formula of the convolution operation is used with the function of the math library to implement the convolution of the specified parameters. Product operation, which corresponds here to the 1*1 convolution operation) (The purpose of the 1*1 convolution operation here is to increase the dimension. Since the dimensionality has just been reduced to reduce the amount of calculation, the image form has changed, but it needs to be added later. The weight result here is multiplied with the original input image. In order to be successfully multiplied with the original image, the weight result must be dimensionally increased according to the original dimensionality reduction form, so this 1*1 convolution operation is performed), use the torch library The sigmoid function implements sigmoid activation, obtains the final weight and multiplies it with the original image to obtain the final processing result of the SE module. The entire MBConv block uses the custom Conv2d() function with relevant parameters to implement k*k depth separable convolution to increase the dimension of the image (Conv2d() function). The custom function creates the form of the image that can be used in the neural network operation, and then Perform a convolution operation. Specifically, it includes obtaining a Conv2d form of the same size as the input image. According to this form, the convolution operation with specified parameters is implemented according to the mathematical formula of the convolution operation and the function of the math library. This corresponds to k*k convolution. Product operation, k refers to the size of the convolution kernel in MBConv, corresponding to the parameters in Figure 18) (The purpose of the k*k convolution operation here is to increase the dimension, which can increase image channels, improve the expression ability of the model, and Reduce the loss of information), and then implement the batch normalization operation of the data through the BatchNorm2d function in the nn tool library under the torch library, and then implement the swish activation operation through the custom MemoryEfficientSwish() function, and call the built SE module after activation , and then use the custom Conv2d() function with relevant parameters to implement 1*1 point-by-point ordinary convolution to reduce the dimension of the image (Conv2d() custom function creates the form of the image that can be used in the neural network operation, and then proceeds Convolution operation. Specifically, it includes obtaining a Conv2d form of the same size as the input image. According to this form, the convolution operation with specified parameters is implemented according to the mathematical formula of the convolution operation and the function of the math library. This corresponds to 1*1 convolution. operation) (the purpose of the 1*1 convolution operation here is to increase the dimension of the image and restore it to the image shape just before the dimension increase), and then The batch normalization operation of the data is implemented through the BatchNorm2d function in the nn tool library under the torch library, and finally through the customized drop_connect() (this function will randomly change the value of the pixel on the input feature map to 0, the purpose is to prevent overfitting [performing well on the training set, performing poorly on the unknown test set]. The deactivation probability comes from the respective index of each MVBConv block, divided by the index value by the number of MBConv blocks The divisor of (26) is obtained by multiplying it by 0.2 (hyperparameter), so that the probability of dropout is greater for the later blocks, again avoiding overfitting) to implement connection skipping to avoid overfitting, and complete the production of the entire MBConv , see code attachment 2 for specific parameters; (this step is used to extract advanced features of the image)

Then build the branches of the model. As shown in Figure 19, by first using the custom Conv2d() function with relevant parameters to implement a 1*1 convolution operation for image dimensionality reduction (the custom function creates the form in which the image can be used in the neural network operation, and then performs the convolution operation . Specifically, it includes obtaining a Conv2d form of the same size as the input image. According to this form, the convolution operation with specified parameters is implemented according to the mathematical formula of the convolution operation and the function of the math library. This corresponds to the 1*1 convolution operation) ( Here, the image dimension is reduced to 10*10*1536 (length, width, number of channels). The purpose is to reduce the subsequent calculation amount, control the model complexity, and improve the feature expression ability), and then use the nn tool library under the torch library The BatchNorm2d inside implements the batch normalization operation of the data and completes the construction of the branches of the model. See code attachment 3 for specific parameters; (the branches perform the final prediction and output of the high-level features of the image extracted from the block)

Finally, build the classifier of the model. As shown in Figure 20, global average pooling is first implemented through the AdaptiveAvgPool2d() function in the nn tool library under the torch library, and then random deactivation is implemented through the Dropout() function in the nn tool library under the torch library. The Linear() function and Softmax() function in the nn tool library under the library realize the classification output of the final probability distribution. See code attachment 4 for specific parameters. After completing the above operations, the construction of the improved efficientnet neural network model is finally completed and named as the model variable. (The classifier performs final prediction and output on the high-level features of the image extracted from the block)

Neural network model parameter adjustment (before adjusting parameters, you need to prepare the training set and verification set as required. The training set, verification set and test set are all taken from the bearing assembly quality data set. The entire data set contains qualified bearings [uniformly distributed steel ball bearings] - That is, the distance between each steel ball is within the error range] and unqualified bearings [Non-uniformly distributed steel ball bearings - that is, the distance between each steel ball is outside the error range, one steel ball bearing is missing - that is, one ball bearing is missing in the bearing. Two steel balls are missing, and two steel ball bearings are missing - that is, two steel balls are missing in the bearing. Then, the data set is divided according to the ratio of 6:2:2, taking into account "reduce training time" and " To improve test reliability"), first carry out the preparatory work for parameter adjustment. You need to use the ImageFolder class under the datasets library to import and preprocess the images according to the path where the images are saved, and use DataLoader to process the overall images in batches. At the same time, the computing device is set to GPU through the device() function under the Torch library to speed up training, thereby completing the preparatory work and then debugging internal parameters. The training set is imported during training and the validation set is imported during verification.

The built improved model model needs to be passed through the custom he_init() function (the custom function here uses init.kaiming_normal_, init.normal_ and init.constant_ functions for the convolution layer, batch normalization layer, and fully connected layer Initialize. This is because the internal parameters during initial model training need to be set to present a normal distribution to prevent gradient disappearance and explosion problems in later training], and the mean and standard deviation corresponding to the normal distribution are hyperparameters. (for manual adjustment) is set to He initialization to help better solve the problems of gradient explosion and gradient disappearance. Then use a for statement to loop through all the batches of the training set and input them into the improved neural network model in turn to complete forward propagation. Then select CrossEntropyLoss() in the nn tool library under the torch library to select the cross-entropy loss function. The loss value is calculated by importing the output value and the real label value into the cross-entropy loss function. Then use backward() after the loss value to perform back propagation to calculate the gradient of each layer. Finally, use step() through the Adam optimizer under the Optim class to complete the parameter operation of updating the neural network model according to the gradient of each layer, thereby completing an internal parameter adjustment. , and then perform hyperparameter debugging. After the training of the entire training set is completed, a verification set is used for verification. That is, use the for statement to loop through all the batches of the verification set and input them into the improved neural network model model in sequence to complete forward propagation. Then import the output value and the real label value into the CrossEntropyLoss() cross-entropy loss function in the nn tool library under the torch library to calculate the loss value and complete the debugging. The set of internal parameters of the neural network with the smallest loss value on the validation set is finally saved as the final parameters of the model. At the same time, there are a variety of hyperparameters in this model. After many experimental comparisons, it was found that the batch_number of the number of reads in each batch is 8, the initial learning rate lr is 1e-3, and the exponential decay of the gradient and the square of the gradient The rate betas are 0.9 and 0.999, the denominator small smoothing term eps is 1e-8, the weight decay weight_decay is 0, the use option of the improved version of Adam algorithm amsgrad is selected True, and the number of iterations of the training set num_epochs is set to 40 to help the model compare The hyperparameters of the Yuyuan model showed better performance, and finally the accuracy test was performed. This time, use the for statement to loop through all the batches of the test set and input them into the improved neural network model model in turn, through, predicted=torch.max(outputs.data,1) (because the ImageFolder function is used when importing images , the image and its corresponding category label are saved in the form of tuples. After neural network training, because the cross-entropy loss function is used, the final output result is also "the possible probability value of this label" + "the identification of this label" The tuple form of "value". Therefore, the torch.max function is used to obtain a tuple with the largest possible probability value, and the second bit of its index ["identification value of this label"] is output to predicted, thereby obtaining the most likely category considered by the model. ) Extract the predicted labels of the model output results and import the real labels of this batch for comparison. If correct, add one to the variable correct. Finally, divide the variable correct by the total number of samples in the test set to get the final result. (See the code in Figures 14 and 16)

After completing the above operations, the change curves of the loss function of the training set and verification set of the efficientnet source code during the training process are shown in Figure 13. The change curves of the loss function of the training set and verification set of the improved model code during the training process are shown in Figure 14. The final accuracy of the efficientnet source code is shown in Figure 15, and the final accuracy of the improved model code is shown in Figure 16. The hyperparameters of this model are adjusted according to the corresponding debugging results, so that the entire model shows the best performance. The initialization is changed to he initialization, and the optimizer is changed to the adam optimizer. Through these improvements, the entire error function curve of the model is close to the standard error function curve, and the accuracy during testing is 100%.

Compared with the original model, this model adds additional image preprocessing before importing the image into the model. The operation of these simplified features simplifies the picture of the bearing assembly into a picture containing only bearing steel balls, reducing the difficulty of the neural network processing task. Overall, image preprocessing improves the image quality before importing the model, laying the foundation for "the neural network to show good performance." This model uses Python to improve the model code when building it. First, the EfficientNet-B3 model parameters are calculated in advance based on the corresponding proportions and the EfficientNet-B0 model, and the EfficientNet-B3 model is directly built based on the parameters using container functions such as nn.Sequential(). , which saves avoidable waste time such as building B0 model, calculating B3 model parameters, and calling multiple py files during the construction process, reduces hardware requirements, and is more suitable for the task of bearing assembly quality inspection.

Code attachment 1:

Conv2d＝get_same_padding_conv2d(image_size＝(300,300))

self.stem_block1=nn.Sequential(Conv2d(in_channels=1,out_channels=40,kernel_size=3,stride=2,padding=1,groups=1,bias=False),nn.BatchNorm2d(num_features=40,momentum=self ._bn_mom,eps=self._bn_eps))

Code attachment 2:

self.block2=nn.Sequential(*[MBConv(in_channels=40, out_channels=16, kernel_size=3, stride=1, expand_ratio=1, squeeze=0.25, id_skip=False, image_size=(150,150))]

+[MBConv(in_channels=16, out_channels=16, kernel_size=3, stride=1, expand_ratio=1, squeeze=0.25, id_skip=True, idx=2, image_size=(150,150))])

self.block3=nn.Sequential(*[MBConv(in_channels=16, out_channels=32, kernel_size=3, stride=2, expand_ratio=6, squeeze=0.25, id_skip=False, image_size=(150,150))]

+[MBConv(in_channels=32, out_channels=32, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=4, image_size=(75,75))]

+[MBConv(in_channels=32, out_channels=32, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=5, image_size=(75,75))])

self.block4=nn.Sequential(*[MBConv(in_channels=32, out_channels=48, kernel_size=5, stride=2, expand_ratio=6, squeeze=0.25, id_skip=False, image_size=(75,75))]+ [MBConv(in_channels=48, out_channels=48, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=7, image_size=(38,38))]

+[MBConv(in_channels=48, out_channels=48, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=8, image_size=(38,38))])

self.block5=nn.Sequential(*[MBConv(in_channels=48, out_channels=96, kernel_size=3, stride=2, expand_ratio=6, squeeze=0.25, id_skip=False, image_size=(38,38))]

+[MBConv(in_channels=96, out_channels=96, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=10, image_size=(19,19))]

+[MBConv(in_channels=96, out_channels=96, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=11, image_size=(19,19))]

+[MBConv(in_channels=96, out_channels=96, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=12, image_size=(19,19))]

+[MBConv(in_channels=96, out_channels=96, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=13, image_size=(19,19))])

self.block6=nn.Sequential(*[MBConv(in_channels=96, out_channels=136, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=False, image_size=(19,19))]+ [MBConv(in_channels=136, out_channels=136, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=15, image_size=(19,19))]

+[MBConv(in_channels=136, out_channels=136, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=16, image_size=(19,19))]

+[MBConv(in_channels=136, out_channels=136, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=17, image_size=(19,19))]

+[MBConv(in_channels=136, out_channels=136, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=18, image_size=(19,19))])

self.block7=nn.Sequential(*[MBConv(in_channels=136, out_channels=232, kernel_size=5, stride=2, expand_ratio=6, squeeze=0.25, id_skip=False, image_size=(19,19))]+ [MBConv(in_channels=232, out_channels=232, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=20, image_size=(10,10))]

+[MBConv(in_channels=232, out_channels=232, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=21, image_size=(10,10))]

+[MBConv(in_channels=232, out_channels=232, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=22, image_size=(10,10))]

+[MBConv(in_channels=232, out_channels=232, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=23, image_size=(10,10))]

+[MBConv(in_channels=232, out_channels=232, kernel_size=5, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=24, image_size=(10,10))])

self.block8=nn.Sequential(*[MBConv(in_channels=232, out_channels=384, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=False, image_size=(10,10))]+ [MBConv(in_channels=384, out_channels=384, kernel_size=3, stride=1, expand_ratio=6, squeeze=0.25, id_skip=True, idx=26, image_size=(10,10))])

Code attachment 3:

Conv2d＝get_same_padding_conv2d(image_size＝(10,10))

self.head_conv9=nn.Sequential(Conv2d(in_channels=384, out_channels=1536, kernel_size=1, stride=1, padding=1, groups=1, bias=False), nn.BatchNorm2d(num_features=1536, momentum=self ._bn_mom,eps=self._bn_eps))

Code attachment 4:

self._avg_pooling=nn.AdaptiveAvgPool2d(1)

self.classifier=nn.Sequential(nn.Dropout(0.3),

nn.Linear(1536,self.num_classes),

nn.Softmax(dim=1))

Obviously, the specific implementation of the present invention is not limited by the above-mentioned manner. As long as various non-substantive improvements are made using the method concepts and technical solutions of the present invention, they are all within the protection scope of the present invention.

Claims (10)

1. An image detection method based on an improved efficientnet model, which is characterized by including the following steps: (1) Preprocess the image to be detected; (2) Build a neural network model based on the improved efficientnet model and conduct training; (3) Use the neural network model trained in step (2) to detect and output the image to be detected. 2. An image detection method based on an improved efficientnet model as claimed in claim 1, characterized in that: In step 2, the neural network model built based on the improved efficientnet model includes: stems used to extract shallow-level features of the image, MBConv blocks used to extract shallow-level features into deep-level features, and transform deep-level features into are the branches of the output and the classifier used to classify the output. 3. A kind of image detection method based on improved efficientnet model as claimed in claim 1, characterized in that: In step (1), preprocessing operations include image cropping, grayscale, equalization to enhance image features, median filtering to remove the influence of lighting, high-boosting filtering to separate the image from the background, and binary extraction of the visual part of the image. ization, image inpainting, tensorization into pytorch operations, and standardized operations to speed up training. 4. An image detection method based on an improved efficientnet model as claimed in claim 1 or 2, characterized in that: The stems used to extract shallow-level features of images include: convolution module, normalization operation module and swish function. The input image undergoes image dimensionality increase through the convolution module, and then performs batch normalization operation through the normalization operation module. , and finally output the shallow features of the image through the swish activation function. 5. An image detection method based on an improved efficientnet model as claimed in claim 1 or 2, characterized in that: The MBConv block used to extract shallow-level features into deep-level features includes: 1*1 convolution for dimensionality increase, k*k-dimensional depth separable convolution, SE module, 1*1 dimensionality reduction convolution and Dropout layer; the input image features first undergo 1*1 convolution to increase the dimension, then pass through the BN layer and Swish activation function, and then are sent to the k*k-dimensional depth separable convolution for processing, and then go through the BN layer and Swish activation again. The function is then sent to the SE module. After processing by the SE module, it is sent to the Dropout layer after 1*1 dimensionality reduction convolution and BN layer. The Dropout output and the initial input of MBConv are added bitwise to obtain the final output of the MBConv block. 6. An image detection method based on an improved efficientnet model as claimed in claim 1 or 2, characterized in that: The branches used to convert deep features into output include 1*1 dimensionality reduction convolution and batch normalization operation modules. 7. An image detection method based on an improved efficientnet model as claimed in claim 2, characterized in that: The classifier used to classify the output consists of a global average pooling layer that sequentially connects the outputs, a random dropout layer, a fully connected layer, and a softmax activation function. 8. An image detection method based on an improved efficientnet model as claimed in claim 1 or 2, characterized in that: The training of the neural network model based on the improved efficientnet model includes: Establish training set, validation set and test set; The model is trained based on the images in the training set to complete the internal parameter adjustment of the model until the training of the entire training set is completed, and a model of the internal parameters of this batch is obtained; Then use the verification set for verification, import the verification set data and input it into the model obtained in this batch in turn, compare the current model output value and the real label value, and then calculate the accuracy of the current model on the verification set, as this batch The accuracy rate corresponding to the model of the obtained internal parameters is obtained, and finally the internal parameters are saved to an optimal parameter file; Then conduct the second batch of training, use the training set again to train a model with new internal parameters, and compare and calculate the accuracy of the new model on the verification set as the corresponding internal parameters obtained in the second batch. Accuracy. Then compare the accuracy of the second batch with the accuracy of the previous batch. If the accuracy is higher, the internal parameters of the second batch will be used to cover the optimal parameter file; if the accuracy is the same or lower, it is not correct. The optimal parameter file is updated. Then within the specified number of batches, training and updating are performed cyclically, and finally the best internal parameters with the highest accuracy are obtained to complete the training of the model. 9. An image detection method based on an improved efficientnet model as claimed in claim 8, characterized in that: Updating the internal parameters of the model based on the training set includes: Preparatory work for parameter adjustment: import images according to the path where the images in the training set are saved and perform preprocessing operations on the images; Internal parameter debugging: He initializes the built model to be improved, and then imports a picture of the training set into the improved neural network model to complete forward propagation; Then select the cross-entropy loss function, and calculate the loss value by importing the output value of the model and the real label value into the cross-entropy loss function; then perform backpropagation based on the loss value to calculate the gradient of each layer, and finally update the neural network according to the gradient of each layer For the internal parameters of the model, the same operation is used to complete the images of the entire training set, thereby completing the internal parameter adjustment of the current batch. 10. An image detection method based on an improved efficientnet model as claimed in claim 8, characterized in that: The hyperparameters of the model are set as follows: the batch_number of the number of reads per batch is 8, the initial learning rate lr is 1e-3, the exponential decay rate betas for the gradient and gradient square are 0.9 and 0.999, and the denominator small smoothing term eps is 1e -8, weight decay weight_decay is 0, select True for the usage option of the improved version of amsgrad of Adam algorithm, and set the number of iterations of the training set num_epochs to 40.