TWI813522B - Classification Model Building Method - Google Patents

Abstract

A method for establishing a classification model, implemented by a computer device, includes the following steps: (A) Based on multiple training images and multiple category tags of multiple training data, use a neural network algorithm to create a model for classifying an image Classification model of the category; (B) According to the classification model, a neural network visualization algorithm is used to generate multiple heat maps corresponding to the training images. Each heat map has a model of interest related to the classification model. area; (C) generate multiple correction data, each correction data includes a heat map corresponding to one of the training images, and a training area of interest position annotation corresponding to one of the training images; and (D) ) train the classification model based on the correction data to adjust the model area of interest of the classification model.

Description

Classification model building method

The present invention relates to a method for establishing a model, in particular to a method for establishing a classification model.

During the manufacturing process of electronic products such as semiconductors, panels (LEDs), and mobile phones, components will inevitably have varying degrees of defects, such as holes, cuts, cracks, and misalignments on circuit boards and LCD panels, which will seriously affect the products. performance.

In the past, in order to judge defects, the traditional inspection method was manual visual inspection. However, personnel needed to use a microscope under strong light illumination, which not only caused great harm to the eyes of the inspectors, but also often caused inspection personnel to be unable to perform their work due to aging eyesight, resulting in personnel turnover. The rate is high, and there are disadvantages such as the subjectivity of different inspectors in judging defects and low efficiency, making it difficult to meet the high-speed and high-resolution inspection requirements.

In order to solve the above problems, the Republic of China Patent Announcement No. I694250 provides a surface defect detection method. Its main technology is to execute a deep learning algorithm to select multiple bounding boxes from an image and output multiple bounding boxes associated with these bounding boxes. characteristic parameters, where each bounding box contains a possible defect of the surface, and then a classification determination algorithm is executed based on the bounding boxes and the characteristic parameters to determine whether the surface meets a specification, wherein The deep learning algorithm uses a defect detection model that has been trained in advance using a Region-based Convolutional Neural Network (R-CNN).

However, the defect detection model used to classify defect types is only trained with the existing supervised learning algorithm based on the training images and corresponding training parameters. The training parameters include sample bounding boxes annotated with the training images with defects. and corresponding defect type labels, whose accuracy is limited.

Therefore, the purpose of the present invention is to provide a method for establishing a classification model that improves model accuracy.

Therefore, the classification model establishment method of the present invention is implemented by a computer device that stores a plurality of training data. Each training data includes a training image, a pair of training category annotations of the training image, and a pair of training data. The method includes a step (A), a step (B), a step (C), and (D) for position labeling of training regions of interest in images.

In the step (A), the computer device uses a neural network algorithm to establish a classification model for classifying an image category based on the training images and category labels of the training data.

In step (B), the computer device uses a neural network visualization algorithm to generate a plurality of heat maps corresponding to the training images based on the classification model. Each heat map has a heat map related to the classification model. Model area of interest.

In this step (C), the computer device generates a plurality of correction data. Each correction data includes a heat map corresponding to one of the training images, and a training area of interest corresponding to the one of the training images. Location annotation.

In the step (D), the computer device trains the classification model based on the training data and the correction data to adjust the model area of interest of the classification model.

The effect of the present invention is to: after establishing the classification model through the computer device, the correction data including the heat maps are generated, and the model interest area of the classification model is adjusted according to the correction data to improve the classification. The accuracy of the model.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are designated with the same numbering.

Referring to FIG. 1 , a computer device 1 used to implement an embodiment of the classification model building method of the present invention includes a storage unit 11 and a processing unit 12 electrically connected to the storage unit 11 .

The storage unit 11 stores a plurality of training data and a plurality of test data. Each training data includes a training image related to an object, a pair of training category annotations of the training image, and a pair of training regions of interest of the training image. Location annotation. The training/test category annotations include a normal annotation, a damaged annotation, a scratch annotation, a crack annotation, a foreign matter annotation, and a contamination annotation, and the training/testing area of interest is annotated as the defective area of the object, If the object is normal, the training/testing region of interest position label will be empty, but it is not limited to this.

Referring to Figures 1 and 2, it is described how the computer device 1 executes this embodiment of the classification model building method of the present invention. This embodiment includes the following steps.

In step 21 , the processing unit 12 uses a neural network algorithm to establish a classification model for classifying an image category based on the training images and category labels of the training data. It is worth noting that in this embodiment, the neural network algorithm is, for example, a convolutional neural network (CNN), but it is not limited to this.

With reference to Figure 3, the sub-steps 211~215 included in step 21 are described below.

In sub-step 211, for each training image, the processing unit 12 uses a preset classification model to generate a first classification result based on the training image. In this embodiment, the preset classification model is, for example, Inception-ResNet-V2 in which the softmax function of the last layer is changed to a sigmoid function, but it is not limited to this.

In sub-step 212, the processing unit 12 uses a first loss function to calculate a first category error value based on the first classification results and training category annotations corresponding to the training images.

It is worth noting that in this embodiment, the first loss function can be any loss function, such as mean square error (Mean square error, MSE), mean absolute error MAE, and other loss functions.

In sub-step 213, the processing unit 12 determines whether the preset classification model converges according to the first category error value. When the processing unit 12 determines that the preset classification model does not converge, the process proceeds to sub-step 214; and when the processing unit 12 determines that the preset classification model converges, the process proceeds to sub-step 215.

In sub-step 214, the processing unit 12 adjusts the preset classification model and repeats steps 211 to 213 until convergence.

In sub-step 215, the processing unit 12 uses the preset classification model as the classification model.

In step 22, the processing unit 12 uses a neural network visualization algorithm to generate a plurality of heat maps corresponding to the training images according to the classification model. Each heat map has a model sense related to the classification model. area of interest.

It is worth noting that in this embodiment, the neural network visualization algorithm is, for example, the Gradient-weighted Class Activation Mapping++ (Grad-CAM++) algorithm. According to the volumes of the last few layers of the classification model, The heat map is obtained by using a feature map of convolution, but is not limited to this.

In step 23 , the processing unit 12 generates a plurality of correction data. Each correction data includes a heat map corresponding to one of the training images, and a training region of interest position annotation corresponding to the one of the training images. .

In step 24 , the processing unit 12 trains the classification model according to the correction data to adjust the model region of interest of the classification model.

With reference to Figure 4, the sub-steps 241~242 included in step 24 are described below.

In sub-step 241, the processing unit 12 uses a second loss function to calculate a position error value and a second category error value based on the correction data.

It is worth noting that in this embodiment, the second loss function can be any loss function different from the first loss function. The processing unit 12 uses the neural network visualization algorithm to be differentiable and capable of backpropagation. The characteristic calculates the position error value and the second type error value.

In sub-step 242, the processing unit 12 uses a transfer learning algorithm to train the classification model according to the second category error value and the position error.

In step 25, the processing unit 12 determines whether the classification model reaches a preset standard based on the test data. When the processing unit 12 determines that the classification model does not reach the preset standard, the process ends; when the processing unit 12 determines that the classification model does not reach the preset standard, the process proceeds to step 26.

It is worth noting that in this embodiment, the preset standard is, for example, that the accuracy of the classification model is greater than a preset threshold. In other embodiments, the preset standard can also be the accuracy of the classification model, The recall rate (Recall), F1 score (F1 Score), and/or area under the Curve of ROC (AUC ROC) are greater than their corresponding preset thresholds, but are not limited to this.

With reference to Figure 5, the sub-steps 251~253 included in step 25 are described below.

In sub-step 251, for each test image of the test data, the processing unit 12 uses the classification model to generate a second classification result based on the test image.

In sub-step 252, the processing unit 12 calculates an accuracy rate based on the second classification results and test category annotations corresponding to the test images.

In sub-step 253, the processing unit 12 determines whether the accuracy rate is greater than a preset threshold to determine whether the classification model reaches the preset standard. When the processing unit 12 determines that the accuracy rate is greater than the preset threshold, it is determined that the classification model reaches the preset standard.

In step 26, the processing unit 12 adjusts the first loss function and the second loss function, and calculates a correlation between the position error value and the second type error based on the position error value and the second type error value. Parameter threshold for value.

In step 27 , the processing unit 12 adjusts the classification model according to the first loss function, the second loss function, and the parameter threshold, and repeats step 25 until the classification model reaches the preset standard.

It is worth noting that in this embodiment, in step 25, it will first be determined whether the classification model reaches the preset standard. If it does not reach the preset standard, the processing unit 12 will adjust the first loss function and the second loss function. The loss function is calculated, and the parameter threshold is calculated based on the position error value and the second category error value, and then the classification model is re-adjusted to ensure the accuracy of the classification model. In other implementations, only steps 21~ 24, not limited to this.

It is important to note that after the classification model is adjusted, when using the classification model to determine a test image related to an object to be tested, the neural network visualization algorithm can also be used to obtain a pair of images to be tested. The heat map allows users to not only know the category of the object to be tested based on the classification results output by the classification model, but also to quickly know the defective areas of the object to be tested based on the heat map.

In summary, the classification model establishment method of the present invention uses the processing unit 12 to establish the classification model, then generates the correction data including the heat maps, and adjusts the model sense of the classification model based on the correction data. The region of interest is used to improve the accuracy of the classification model. In addition, the classification model does not require the position annotation of the training region of interest of the training data for training and learning. The processing unit 12 uses the neural network visualization algorithm to directly obtain Including the heat map of the model's area of interest, compared to the previous case that required training and learning based on the bounding box, the classification model structure of this case is relatively simple, and the prediction speed is relatively fast, so it can indeed achieve the purpose of the present invention.

However, the above are only examples of the present invention and should not be used to limit the scope of the present invention. All simple equivalent changes and modifications made based on the patent scope of the present invention and the content of the patent specification are still within the scope of the present invention. within the scope covered by the patent of this invention.

1: Computer device 11:Storage unit 12: Processing unit 21~27: Steps 211~215: Sub-steps 241~242: Sub-steps 251~253: Sub-steps

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, in which: Figure 1 is a block diagram illustrating a computer device used to implement an embodiment of the classification model building method of the present invention; Figure 2 is a flow chart illustrating this embodiment of the classification model establishment method of the present invention; Figure 3 is a flow chart to assist in explaining the sub-steps of step 21 in Figure 2; Figure 4 is a flow chart to assist in explaining the sub-steps of step 24 in Figure 2; and FIG. 5 is a flow chart to assist in explaining the sub-steps of step 25 in FIG. 2 .

21~27: Steps

Claims (4)

A method for establishing a classification model is implemented by a computer device. The computer device stores a plurality of training data. Each training data includes a training image, a pair of training category annotations corresponding to the training image, and a pair of training data corresponding to the training image. Region of interest position annotation, the method includes the following steps: (A) Based on the training images and category labels of the training data, use a neural network algorithm to establish a classification model for classifying the category of an image; (B) According to the classification model, a neural network visualization algorithm is used to generate multiple heat maps corresponding to the training images. Each heat map has a model area of interest related to the classification model; (C) Generating multiple strokes Correction data, each correction data including a heat map corresponding to one of the training images, and a training area of interest position annotation corresponding to one of the training images; and (D) training the correction data based on the correction data Classification model to adjust the model area of interest of the classification model. Step (D) includes the following sub-steps: (D-1) Based on the correction data, use a second loss function to calculate a position error value and a first two categories of error values, and (D-2) using a transfer learning algorithm to train the classification model based on the second category error values and the position error. The method for establishing a classification model as described in claim 1, wherein step (A) includes the following sub-steps: (A-1) for each training image, use a preset classification model to generate a first classification result based on the training image ; (A-2) Use a first loss function to calculate a first category error value based on the first classification results and training category annotations corresponding to the training images; (A-3) Determine the first category error value based on the first category error value Whether the preset classification model converges; (A-4) When it is determined that the preset classification model does not converge, adjust the preset classification model and repeat sub-steps (A-1) ~ (A-3) until convergence; and ( A-5) When it is determined that the preset classification model has converged, the preset classification model is used as the classification model. According to the classification model building method described in claim 1, the computer device also stores a plurality of test data. Each test data includes a test image, a pair of test category annotations of the test image, and a pair of tests of the test image. The location labeling of the region of interest also includes the following steps after step (D): (E) Determining whether the classification model reaches a preset standard based on the test data; (F) When it is determined that the classification model does not reach the preset standard When standard, adjust the first loss function and the second loss function, and calculate a parameter threshold related to the position error value and the second type error value based on the position error value and the second type error value; and ( G) Adjust the classification model according to the first loss function, the second loss function, and the parameter threshold, and repeat step (E) until the classification model reaches the preset standard. The method for establishing a classification model as described in claim 3, wherein step (E) includes the following sub-steps: (E-1) For each test image of the test data, use the classification model to generate a second classification result based on the test image; (E-2) Calculate based on the second classification result and test category label corresponding to the test image Determine an accuracy rate; and (E-3) determine whether the accuracy rate is greater than a preset threshold value to determine whether the classification model reaches the preset standard. When it is determined that the accuracy rate is greater than the preset threshold value, determine the classification The model reaches this preset standard.

Images