WO2025139322A1 - Chip defect detection method and system - Google Patents

Abstract

The present invention relates to a chip defect detection method and system. The method comprises: acquiring a one-dimensional vibration signal of a chip and converting same into two-dimensional image data; training an algorithm NAS by means of the two-dimensional image data, and optimizing the algorithm NAS to obtain a neural architecture search algorithm model ASNDARTS; constructing a knowledge distillation algorithm model DPSKD on the basis of the neural architecture search algorithm model ASNDARTS; and acquiring a one-dimensional vibration signal of a chip under test and converting sane into two-dimensional image data, and detecting the two-dimensional image data of the chip under test by means of the knowledge distillation algorithm model DPSKD, so as to determine whether the chip under test has a defect. In the present invention, the algorithm NAS is optimized, the knowledge distillation transfer method is optimized, and the obtained model DPSKD effectively improves the accuracy of chip defect detection while having reduced model redundancy.

Description

Chip defect detection method and system Technical Field

The present invention relates to the technical field of chip quality detection, and in particular to a chip defect detection method and system.

Background Art

The rapid development of electronic products has prompted electronic devices to develop in the direction of miniaturization and high integration, and traditional wire bonding technology is difficult to meet the requirements. Flip chip packaging technology is widely used in the microelectronic packaging industry due to its good manufacturability and high reliability. However, due to the mismatch of thermal expansion coefficients between the chip and the substrate, it is easy to cause internal defects of the chip, such as cracks, missing solder joints and cold solder joints. Therefore, in order to ensure the high performance and high reliability of the product, chip defect detection technology has received more and more attention in the field of electronic packaging.

Recently, deep learning technology has gradually entered the field of engineering. There are many reasons for its successful application in the field of fault detection. First, deep learning can extract useful information from a large amount of data and has strong big data processing capabilities. Second, industrial data collection and storage technology continues to develop, and the scale of industrial data continues to expand. These massive data have laid the foundation for the application of deep learning. Third, deep learning has powerful feature extraction and fitting capabilities, which can realize end-to-end fault diagnosis and save a lot of manual data processing and feature extraction steps. Fourth, deep learning can solve a variety of problems such as classification, prediction, and decision-making.

Existing fault diagnosis deep models require a lot of trial and error debugging by professionals when dealing with different application scenarios, resulting in high development costs and long cycles. Furthermore, in order to achieve the final improvement in prediction accuracy, many existing deep application models with poor performance will be added with other modules, thus becoming more and more bloated. This will undoubtedly slow down the prediction speed of the model in practical applications. The design and adjustment of deep models requires staff to have a certain level of professional knowledge. Because in different engineering application scenarios, Different scenarios, different research objects, and different signal distributions require different model structures for specific tasks to accurately identify fault patterns. Designing algorithm models that are suitable for various fault diagnosis tasks and have excellent performance is a great challenge. Therefore, research on automatic lightweight deep models for fault diagnosis is in urgent need of development.

Summary of the invention

To this end, the technical problem to be solved by the present invention is to overcome the problems in the prior art of deep technology, such as high degree of manual intervention, vague engineering application directionality, redundant and bloated models, and inconvenience in edge device deployment.

In order to solve the above technical problems, the present invention provides a chip defect detection method, comprising:

Acquire the one-dimensional vibration signal of the chip and convert it into two-dimensional image data;

Construct a neural architecture search algorithm model ASNDARTS, and construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS;

The method for constructing the neural architecture search algorithm model ASNDARTS is as follows: inputting the two-dimensional image data of the chip into the neural architecture search algorithm NAS for training, and optimizing the neural architecture search algorithm NAS during the training process, and obtaining the neural architecture search algorithm model ASNDARTS when the neural architecture search algorithm NAS reaches Nash equilibrium;

The construction method of the knowledge distillation algorithm model DPSKD is: using the neural architecture search algorithm model ASNDARTS as a teacher network for knowledge distillation, and taking a single cell in the neural architecture search algorithm model ASNDARTS as a student network for knowledge distillation, wherein the neural architecture search algorithm model ASNDARTS includes a plurality of cells connected in series, and each of the cells is a result obtained by searching the neural architecture search algorithm model ASNDARTS in the search space;

Based on the probability space, the knowledge transfer method between the teacher network and the student network is decoupled, and the hyperparameters of the student network are optimized. When the student network converges, the knowledge distillation algorithm model DPSKD is obtained;

The one-dimensional vibration signal of the chip to be detected is obtained and converted into two-dimensional image data. The two-dimensional image data of the chip to be detected is detected by the knowledge distillation algorithm model DPSKD to determine the Is the chip defective?

In one embodiment of the present invention, the optimization of the neural architecture search algorithm NAS during the training process includes a first optimization method, specifically:

During the training process, the probability of the original skip_connect operation of the neural architecture search algorithm NAS being selected is band-limited.

In one embodiment of the present invention, the optimization of the neural architecture search algorithm NAS during the training process includes a second optimization method, specifically:

During the training process, several operation modes OP are randomly selected from the search space of the algorithm NAS to form an initial search space Q1, and the remaining operation modes OP in the search space of the algorithm NAS are then combined to form a spare search space Q2. According to a preset criterion, it is determined whether adaptive adjustment is required between the operation modes OP in the initial search space Q1 and the operation modes OP in the spare search space Q2. If the operation mode OP requires adaptive adjustment, the operation mode OP with the lowest probability of being selected in the initial search space Q1 is replaced by the first operation mode OP in the spare search space Q2, and the operation mode OP with the lowest probability of being selected is used as the last operation mode OP in the spare search space Q2.

In one embodiment of the present invention, the optimization of the neural architecture search algorithm NAS during the training process includes a third optimization method, specifically:

During the training process, the number of operation modes OP of the node is adaptively expanded according to the model contribution rate of each operation mode OP.

In one embodiment of the present invention, the method of performing band-limited correction on the probability of selection of the original skip_connect operation of the neural architecture search algorithm NAS during the training process includes:

The selected probability value α _skip of the skip_connect operation during the training of the neural architecture search algorithm NAS is recorded in real time. The selected probability distribution of α _skip is P _skip . The point when P _skip shows a gradient rise phenomenon is defined as the algorithm NAS crash starting point epoch_break. The parameter k _skip is introduced to correct the α _skip of the skip_connect operation to obtain the corrected value α _{correct_skip} = α _skip *k _skip . The difference between α _skip and α _{correct_skip} at the algorithm NAS crash starting point is defined as △. The band-limited corrected probability is obtained according to α _skip , α _{correct_skip} and △. The formula is as follows:

Among them, * represents the multiplication sign, k _skip represents the restriction on the probability of skip_connect operation in the algorithm NAS search process; the first half of k _skip , k _skip(tanh) , is the limit on the number of early skip_connect operations at the epoch level based on the tanh activation function, and the second half of k _skip, k _{skip(sigmoid),} is the limit on the number of skip_connect operations at the edge level based on the sigmoid activation function; Indicates the probability of α _skip taking value before the crash starting point epoch_break; Indicates the probability of α _skip taking the value after the crash starting point epoch_break.

In one embodiment of the present invention, the preset criterion includes:

The preset criteria include, in order of judgment, the algorithm accuracy trend under equal training epoch intervals, whether there is an abnormal accuracy value under equal training epoch intervals, the OP contribution value trend of each operation mode under equal training epoch intervals, and the OP frequency of each node selected operation mode under equal training epoch intervals, wherein:

The algorithm accuracy trend under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, the algorithm accuracy value of each equal training epoch interval is counted, and the algorithm accuracy values of all equal training epoch intervals are linearly fitted to obtain a first linear fitting value. If the first linear fitting value is greater than 0, it indicates that the algorithm accuracy is on an upward trend; if the first linear fitting value is less than 0, This indicates that the accuracy of the algorithm is on a downward trend;

Whether there is an abnormal accuracy value under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, and the algorithm accuracy value of each equal training epoch interval is counted. If the algorithm accuracy value does not increase with the increase of the equal training epoch interval, it indicates that the algorithm accuracy value of the equal training epoch interval is abnormal, otherwise it is normal;

The contribution value trend of each operation mode OP under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, and at the same time, the selection probability of each operation mode OP in each equal training epoch interval is obtained, and the selection probability of OP in all equal training epoch intervals is linearly fitted to obtain a second linear fitting value, and if the second linear fitting value is greater than 0, it indicates that the contribution value of the operation mode OP is in an upward trend; if the second linear fitting value is less than 0, it indicates that the contribution value of the operation mode OP is in a downward trend;

The frequency of the operation mode OP selected by each node under the equal training epoch interval is: obtain a preset number of equal training epoch intervals, and simultaneously obtain the number of times each operation mode OP is selected in all equal training epoch intervals, and according to the preset number of selections a, determine the relationship between the number of times each operation mode OP is selected and the preset number of selections a.

In one embodiment of the present invention, the method for adaptively expanding the number of operation modes OP of a node according to the model contribution rate of each operation mode OP includes:

In the algorithm NAS, the operation mode OP contribution discrimination mechanism is introduced into the predecessor connection edge of each node, and a preset number of equal training epoch intervals are obtained. At the same time, the selection probability of each operation mode OP in each equal training epoch interval is obtained, and the selection probability of OP in all equal training epoch intervals is linearly fitted to obtain the second linear fitting value. If the second linear fitting value is greater than 0, it means that the contribution value of the operation mode OP is on an upward trend; if the second linear fitting value is less than 0, it means that the contribution value of the operation mode OP is on a downward trend;

During the training process, the current node selects the operation mode OP whose contribution value is an upward trend, ignores the operation mode OP whose contribution value is a downward trend, and completes the adaptive expansion of the number of operation mode OPs of the current node.

In one embodiment of the present invention, the search space of the algorithm NAS includes several optional operation modes OP, and the search space of the algorithm NAS includes several optional operation modes OP, including several max_pool_nxn, several avg_pool_nxn, skip_connect, several sep_conv_nxn, several dil_conv_nxn, and none, wherein max_pool represents maximum pooling, avg_pool represents average pooling, skip_connect represents skip connection, sep_conv represents depthwise separable convolution, dil_conv represents hole convolution, none represents no operation, and n takes an odd number in the value range ∈ [1,9].

In one embodiment of the present invention, the method for decoupling the knowledge transfer between the teacher network and the student network based on the probability space includes:

The model ASNDARTS is probability reassigned and divided into three hierarchical category probability spaces, namely target space t, target class space O and overall space C. To separate the predictions of the three hierarchical category probability spaces, the symbol P = [p _t , p _O\t , p _\O ] is defined, where p _t represents the probability of target space t, p _O\t represents the probability of excluding target t in the target class space O, and p _\O represents the probability of excluding O in the overall space C.

Among them, _zt represents the soft probability of the target class, _zj represents the soft probability of all classes, _zk represents the soft profile of the non-target class, k represents the in-class index of the non-target class, and j represents the out-class index of the non-target class;

Define the various probabilities in O\t space and \O as and Among them, ∧ represents the correction symbol of the final skip_connect, It means that in the target class space O, other independent similar probabilities of the target space t are excluded; similarly, represents the probability of other independent similar objects in the overall space C after excluding the target class space O, where

The knowledge distillation KD is decoupled into three-level category probability spaces, and the formula is:

make ⊙ represents dot product;

in, Represents the KL divergence TKD of the teacher-student model in the three defined spaces; Represents the KL divergence CKD of the teacher-student model on the non-target; It represents the KL divergence OKD of the teacher-student model among other classes;

The KL divergence TKD, KL divergence CKD and KL divergence OKD of the three-level category probability space are reorganized, and the knowledge transfer between the teacher network and the student network is realized based on the reorganization results.

In order to solve the above technical problems, the present invention provides a chip defect detection system, comprising:

Acquisition module: used to acquire the one-dimensional vibration signal of the chip and convert it into two-dimensional image data;

Construction module: used to construct a neural architecture search algorithm model ASNDARTS, and to construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS;

The method for constructing the neural architecture search algorithm model ASNDARTS is as follows: inputting the two-dimensional image data of the chip into the neural architecture search algorithm NAS for training, and optimizing the neural architecture search algorithm NAS during the training process, and obtaining the neural architecture search algorithm model ASNDARTS when the neural architecture search algorithm NAS reaches Nash equilibrium;

The construction method of the knowledge distillation algorithm model DPSKD is: using the neural architecture search algorithm model ASNDARTS as a teacher network for knowledge distillation, and taking a single cell in the neural architecture search algorithm model ASNDARTS as a student network for knowledge distillation, wherein the neural architecture search algorithm model ASNDARTS includes a plurality of cells connected in series, and each of the cells is a result obtained by searching the neural architecture search algorithm model ASNDARTS in the search space;

Based on the probability space, the knowledge transfer method between the teacher network and the student network is decoupled, and the hyperparameters of the student network are optimized. When the student network converges, the knowledge distillation algorithm model DPSKD is obtained;

Detection module: used to obtain the one-dimensional vibration signal of the chip to be detected and convert it into two-dimensional image data, and detect the two-dimensional image data of the chip to be detected through the knowledge distillation algorithm model DPSKD to determine whether the chip to be detected has defects.

The above technical solution of the present invention has the following advantages compared with the prior art:

The present invention selects the neural architecture search algorithm NAS as the basic network model, effectively solves the time-consuming problem of manually designing models, and designs a lightweight network model with excellent performance that can effectively target specific engineering application backgrounds;

The present invention performs band-limited correction on the skip_connect operation in the NAS algorithm to solve the NAS algorithm The present invention proposes the concept of alternative search space, which solves the problem of excessive GPU usage of NAS algorithm on the basis of expanding the diversity of search space; the present invention introduces a contribution discrimination mechanism to the predecessor connection edge of each node in the NAS algorithm, obtains a variable number of node structures, and constitutes diversified cells; the present invention greatly improves the training stability and detection accuracy of the neural architecture search algorithm NAS, and reduces the consumption of hardware resources;

The present invention uses the network model ASNDARTS obtained by neural architecture search as the basic teacher model, further lightweights the network structure through knowledge distillation technology to obtain the student model, and decouples the knowledge transfer logic between the teacher and student models, effectively improving the ability to obtain information from the teacher model, and reducing the computing resource loss of the model without reducing the prediction accuracy;

The present invention can adaptively design deep neural networks under different industrial backgrounds, improve the prediction accuracy of model data while lightweighting the model structure, making it easy to deploy on edge devices to realize chip defect detection.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

FIG1 is a flow chart of a chip defect detection method according to the present invention;

FIG2 is a flow chart of a skip_connect operation band limit correction in an embodiment of the present invention;

FIG3 is a logic diagram of a skip_connect operation band limit correction in an embodiment of the present invention;

4 is a logic diagram of the decoupling of the teacher-student knowledge transfer logic in the DPSKD model according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a result structure (ie, a cell) searched by the model ASNDARTS in an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

Embodiment 1

Referring to FIG. 1 , the present invention relates to a chip defect detection method, comprising:

Acquire the one-dimensional vibration signal of the chip and convert it into two-dimensional image data;

Construct a neural architecture search algorithm model ASNDARTS, and construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS;

The method for constructing the neural architecture search algorithm model ASNDARTS is as follows: inputting the two-dimensional image data of the chip into the neural architecture search algorithm NAS for training, and optimizing the neural architecture search algorithm NAS during the training process, and obtaining the neural architecture search algorithm model ASNDARTS when the neural architecture search algorithm NAS reaches Nash equilibrium;

The construction method of the knowledge distillation algorithm model DPSKD is: using the neural architecture search algorithm model ASNDARTS as a teacher network for knowledge distillation, and taking a single cell in the neural architecture search algorithm model ASNDARTS as a student network for knowledge distillation, wherein the neural architecture search algorithm model ASNDARTS includes a plurality of cells connected in series/parallel, each of which is a result obtained by the neural architecture search algorithm model ASNDARTS in a search space, and the search space is used to store the operation mode OP;

Based on the probability space, the knowledge transfer method between the teacher network and the student network is decoupled, and the hyperparameters of the student network are optimized. When the student network converges, the knowledge distillation algorithm model DPSKD is obtained;

The one-dimensional vibration signal of the chip to be detected is obtained and converted into two-dimensional image data, and the two-dimensional image data of the chip to be detected is detected by the knowledge distillation algorithm model DPSKD to determine whether the chip to be detected has defects.

Furthermore, the optimization of the neural architecture search algorithm NAS during the training process includes a first optimization method, specifically: performing a band-limited correction on the original skip_connect operation selection probability of the neural architecture search algorithm NAS during the training process.

Specifically, during the training process, the original skip_connect operation of the neural architecture search algorithm NAS is subjected to a band-limited correction, and the method includes:

The selected probability value α _skip of the skip_connect operation during the training of the neural architecture search algorithm NAS is recorded in real time. The selected probability distribution of α _skip is P _skip . The point when P _skip shows a gradient rise phenomenon is defined as the algorithm NAS crash starting point epoch_break. The parameter k _skip is introduced to correct the α _skip of the skip_connect operation to obtain the corrected value α _{correct_skip} = α _skip *k _skip . The difference between α _skip and α _{correct_skip} at the algorithm NAS crash starting point is defined as △. The band-limited corrected probability is obtained according to α _skip , α _{correct_skip} and △. The formula is as follows:

Among them, * represents the multiplication sign, k _skip represents the restriction on the probability of skip_connect operation in the algorithm NAS search process; in the early stage of the algorithm search, data features can be fully extracted without using skip_connect operation for skip connection; the first half of k _skip k _skip(tanh) is the limit on the number of early skip_connect operations at the epoch level based on the tanh activation function, and the second half of k _skip k _{skip(sigmoid)} is the limit on the number of skip_connect operations at the edge level (edge is built-in to the NAS algorithm, such as the line between 0 and 2 in Figure 5 is an edge) based on the sigmoid activation function (the first/second half: the operation that has a greater impact on the epoch/edge in the first/second half); Indicates the probability of α _skip taking value before the crash starting point epoch_break; Indicates the probability of α _skip taking the value after the crash starting point epoch_break.

This embodiment corrects the probability of the skip_connect operation being selected, which can effectively solve the algorithm crash phenomenon.

Furthermore, the optimization of the neural architecture search algorithm NAS during the training process includes a second optimization method, specifically: during the training process, a number of operation modes OP are randomly selected from the search space of the algorithm NAS to form an initial search space Q1, and then the remaining operation modes OP in the search space of the algorithm NAS form a spare search space Q2, and whether the operation modes OP in the initial search space Q1 and the operation modes OP in the spare search space Q2 need to be adaptively adjusted according to a preset criterion, wherein, if the operation mode OP needs to be adaptively adjusted, the operation mode OP with the lowest probability of being selected in the initial search space Q1 is replaced by the first operation mode OP in the spare search space Q2, and the operation mode OP with the lowest probability of being selected is used as the last operation mode OP in the spare search space Q2. It should be noted that the initial order of the operation modes OP in the spare search space Q2 is random.

In this embodiment, the search space of the algorithm NAS includes several optional operation modes OP, and the operation modes OP include several max_pool_nxn, several avg_pool_nxn, skip_connect, several sep_conv_nxn, several dil_conv_nxn, and none, where max_pool represents maximum pooling, avg_pool represents average pooling, skip_connect represents skip connection, sep_conv represents depthwise separable convolution, dil_conv represents dilated convolution, none represents no operation, and n takes an odd number in the range ∈ [1,9]. In this embodiment, the search strategy of the algorithm NAS adopts the gradient descent method.

Specifically, the preset criteria include, in order of judgment, (1) the algorithm accuracy trend under equal training epoch intervals, (2) whether there is an abnormal accuracy value under equal training epoch intervals, (3) the OP contribution value trend of each operation mode under equal training epoch intervals, and (4) the OP frequency of each node selected under equal training epoch intervals, wherein:

(1) The algorithm accuracy trend under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, the algorithm accuracy value of each equal training epoch interval is counted, and the algorithm accuracy values of all equal training epoch intervals are linearly fitted to obtain a first linear fitting value. If the first linear fitting value is greater than 0, it indicates that the algorithm accuracy is on an upward trend; if the first linear fitting value is less than 0, it indicates that the algorithm accuracy is on a downward trend.

(2) Whether there is an abnormal accuracy value under the equal training epoch interval is: Get the preset The algorithm accuracy value of each equal training epoch interval is counted. If the algorithm accuracy value does not increase with the increase of the equal training epoch interval, it indicates that the algorithm accuracy value of the equal training epoch interval is abnormal, otherwise it is normal. For example, the algorithm accuracy values of four equal training epoch intervals are obtained: 0.3, 0.7, 0.4, and 0.8. Then 0.4 is an abnormal accuracy value, because the accuracy of general model training will increase, and a sudden drop of 0.4 is judged as an abnormal accuracy value.

(3) The contribution value trend of each operation mode OP under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, and at the same time, the selection probability of each operation mode OP in each equal training epoch interval is obtained, and the selection probability of OP in all equal training epoch intervals is linearly fitted to obtain a second linear fitting value. If the second linear fitting value is greater than 0, it indicates that the contribution value of the operation mode OP is on an upward trend; if the second linear fitting value is less than 0, it indicates that the contribution value of the operation mode OP is on a downward trend.

(4) The frequency of the operation mode OP selected by each node under the equal training epoch interval is as follows: a preset number of equal training epoch intervals is obtained, and the number of times each operation mode OP is selected in all equal training epoch intervals is obtained, and according to the preset number of times a, the relationship between the number of times each operation mode OP is selected and the preset number of times a is determined.

Furthermore, the neural architecture search algorithm NAS is optimized during the training process, including a third optimization method, specifically: during the training process, the number of operation modes OP of the node is adaptively expanded according to the model contribution rate of each operation mode OP.

Specifically, the method for adaptively expanding the number of operation modes OP of a node according to the model contribution rate of each operation mode OP includes:

In the algorithm NAS, the operation mode OP contribution discrimination mechanism is introduced into the predecessor connection edge of each node, and a preset number of equal training epoch intervals are obtained. At the same time, the selection probability of each operation mode OP in each equal training epoch interval is obtained, and the selection probability of OP in all equal training epoch intervals is linearly fitted to obtain the second linear fitting value. If the second linear fitting value is greater than 0, it means that the contribution value of the operation mode OP is on an upward trend (equivalent to having a positive meaning); if the second linear fitting value is less than 0, it means that the contribution value of the operation mode OP is on a downward trend;

During the training process, the current node selects the operation mode OP whose contribution value is an upward trend, ignores the operation mode OP whose contribution value is a downward trend, and completes the adaptive expansion of the number of operation mode OPs of the current node.

The method of knowledge transfer between the teacher network and the student network based on probability space decoupling includes:

The deep learning soft probability model ASNDARTS is probability reassigned and divided into three hierarchical category probability spaces, namely target space t, target class space O and overall space C. To separate the predictions of the three hierarchical category probability spaces, the symbol P = [p _t , p _O\t , p _\O ] is defined, where p _t represents the probability of target space t, p _O\t represents the probability of excluding target t in the target class space O, and p _\O represents the probability of excluding O in the overall space C.

Among them, _zt represents the soft probability of the target class, _zj represents the soft probability of all classes, _zk represents the soft profile of the non-target class, k represents the in-class index of the non-target class, and j represents the out-class index of the non-target class;

Define the various probabilities in O\t space and \O as and Among them, ∧ represents the correction symbol of the final skip_connect, It means that in the target class space O, other independent similar probabilities of the target space t are excluded; similarly, represents the probability of other independent similar defects in the overall space C after excluding the target class space O (other independent similar defects represent different defect degrees of the same type of defects), where

The knowledge distillation KD is decoupled into three-level category probability spaces, and the formula is:

make ⊙ represents dot product;

in, Represents the KL divergence TKD of the teacher-student model in the three defined spaces; Represents the KL divergence CKD of the teacher-student model on the non-target; represents the KL divergence between the teacher-student model and other classes Degree OKD;

The KL divergence TKD, KL divergence CKD and KL divergence OKD of the three-level category probability space are reorganized (DPSKD=α*TKD+β*CKD+γ*OKD), and the knowledge transfer between the teacher network and the student network is realized based on the reorganization results.

Figure 4 shows the decoupling logic of knowledge transfer logic using three-level category probability space. A1, A2, A3, A4, and A5 in Figure 4 are as follows:

A1:

A2:

A3:

A4:

A5:

The technical solution of the present invention is further described below in conjunction with specific embodiments:

Step S1: Obtain a one-dimensional vibration signal from the faulty chip, sample the data with equal steps using a window width of n*n length; reconstruct the sampled signal into an n*n matrix after Fourier transformation, and concatenate three layers of n*n matrices to form a three-channel image data. Each sample is of uniform size n*n and is A one-hot type label of the corresponding category is produced for each sample (since it is a prior art, it will not be described in detail in this embodiment). All samples are divided into a training set and a test set according to a certain ratio.

Step S2: construct a neural architecture search algorithm model ASNDARTS, and construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS.

Build an improved neural architecture search algorithm ASNDARTS and obtain the lightweight model model_V1.0.

Step S3: Based on the original model NAS, a skip_connect operation correction value is introduced, which is combined with the original skip_connect operation selection probability value. Based on the accuracy convergence trend and model structure convergence trend during the algorithm training process, the skip_connect operation selection probability is corrected to solve the algorithm crash. The specific workflow and usage logic are shown in Figure 2.

The selected probability value α _skip and the correction value α _{correct_skip} of the skip_connect operation in the search space are recorded in real time. The distribution of α _skip is P _skip ; the point where the gradient rise phenomenon occurs in P _skip is defined as the algorithm crash starting point epoch_break. The parameter k _skip is introduced to correct the a value of the skip operation to obtain α _{correct_skip} = α _skip *k _skip . The corrected coefficient α _{correct_skip} will tend to be stable. The difference between the algorithm crash point and is defined as △. The probability of α _skip taking values after the crash point is taken as Get the final band-limited corrected probability

Among them, k _skip represents the control of the probability of skip_connect operation during the algorithm search process. In the early stage of algorithm search, data features can be fully extracted without using skip_connect for skip connection. The first half of k _skip is based on the tanh activation function to limit the number of early skip_connect operations at the epoch level. The second half is based on the sigmoid activation function to limit the number of skip_connect operations at the edge level.

Step S4: For example, define the initialization neural architecture search algorithm search space: Q1 = {max_pool_3x3, sep_conv_3x3, sep_conv_7x7, dil_conv_3x3, dil_conv_7x7, skip_connect, none}, spare search space: Q2 = {avg_pool_3x3, avg_pool_5x5, sep_conv_5x5, dil_conv_5x5, max_pool_5x5}. According to the set criteria, the optional operations in the search space are replaced and adjusted. The criteria include: (1) the trend of model accuracy under equal training epoch intervals, (2) the abnormal situation of model accuracy value under equal training epoch intervals, (3) the trend of contribution value of each operation mode under equal training epoch intervals, and (4) the frequency of operation mode selected by each node under equal training epoch intervals. The specific judgment logic is shown in Table 1 below.

Table 1 Search space adaptation criteria

Step S5: Introduce the operation mode op contribution discrimination mechanism for each node predecessor connection edge, and adaptively expand the number of node operations according to the op model contribution rate to obtain a variable number of node nodes. structure;

Step S6: loop steps S3 to S5 with equal epoch as the step length. When the model accuracy converges to the best and the model structure searched by the neural architecture converges stably after about 150 iterations in this embodiment, the neural architecture search algorithm model ASNDARTS is obtained. Input the test set, and the chip defect detection lightweight model model_V1.0 is built.

Step S7: Use model_V1.0 (i.e., model ASNDARTS) as the teacher network in the knowledge distillation model, and take a single cell as the student network.

Figure 5 shows the result structure of the search by the model ASNDARTS, i.e., a cell. Blocks 1 to 4 in Figure 5 represent four nodes, and the input predecessor edge of each node corresponds to an operation mode OP. model_V1.0 is formed by multiple cells in series/parallel, and is also the structure of model_V2.0 (i.e., model DPSKD).

Step S8: Construct a new information transfer logic of the teacher-student network in the knowledge distillation algorithm, combine the chip fault category and fault degree to achieve the decoupling of the target class, target family class and all classes into three probability spaces, and obtain the knowledge transfer logic formula under different probability spaces. The specific workflow and usage logic are shown in Figure 3.

The model probability is reassigned and divided into three levels of class probability space, namely target space t, target class space O and overall space C. In order to separate the prediction situations at three levels, the following symbol P = [p _t , p _O\t , p _\O ] is defined. _{p t} represents the probability of target t, p _O\t represents the probability of excluding target t in O space, and p _\O represents the probability of excluding O in the overall space C.

Among them, _zt represents the soft probability of the target class, _zj represents the soft probability of all classes, _zk represents the soft profile of the non-target class, k represents the in-class index of the non-target class, and j represents the out-class index of the non-target class;

Define the various probabilities in O\t space and \O as and Among them, ∧ represents Finally, the corrected symbol of skip_connect, It means that in the class O space, other independent similar probabilities of t are excluded. Similarly, Represents other independent probabilities in C space after excluding O space.

Decouple the traditional KD into three probability spaces:

according to and ⊙ represents dot product:

Among them, ① represents the KL divergence TKD of the teacher-student model in the three defined spaces; ② represents the KL divergence CKD of the teacher-student model in non-target; ③ represents the KL divergence OKD of the teacher-student model between other classes.

The KL divergence TKD, KL divergence CKD and KL divergence OKD of the three-level category probability space are analyzed. The network is reorganized and the knowledge transfer between the teacher network and the student network is realized based on the reorganization results.

Step S9: Optimize the hyperparameters of the student model using a hyperparameter optimization algorithm;

Step S10: loop steps S8 to S9, and after about 50 iterations, the student model accuracy converges to the best, and the knowledge distillation algorithm model DPSKD is obtained. Input the test set, and the minimalist self-search model model_V2.0 is built. Model model_V2.0 can be deployed on the edge machine to implement the chip defect detection application of the edge device.

Embodiment 2

This embodiment provides a chip defect detection system, including:

Acquisition module: used to acquire the one-dimensional vibration signal of the chip and convert it into two-dimensional image data;

Construction module: used to construct a neural architecture search algorithm model ASNDARTS, and to construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS;

The method for constructing the neural architecture search algorithm model ASNDARTS is as follows: inputting the two-dimensional image data of the chip into the neural architecture search algorithm NAS for training, and optimizing the neural architecture search algorithm NAS during the training process, and obtaining the neural architecture search algorithm model ASNDARTS when the neural architecture search algorithm NAS reaches Nash equilibrium;

The construction method of the knowledge distillation algorithm model DPSKD is: using the neural architecture search algorithm model ASNDARTS as a teacher network for knowledge distillation, and taking a single cell in the neural architecture search algorithm model ASNDARTS as a student network for knowledge distillation, wherein the neural architecture search algorithm model ASNDARTS includes a plurality of cells connected in series, and each of the cells is a result obtained by searching the neural architecture search algorithm model ASNDARTS in the search space;

Based on the probability space, the knowledge transfer method between the teacher network and the student network is decoupled, and the hyperparameters of the student network are optimized. When the student network converges, the knowledge distillation algorithm model DPSKD is obtained;

Detection module: used to obtain the one-dimensional vibration signal of the chip to be detected and convert it into two-dimensional image data, and detect the two-dimensional image data of the chip to be detected through the knowledge distillation algorithm model DPSKD to determine whether the chip to be detected has defects.

Embodiment 3

This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the chip defect detection method described in the first embodiment are implemented.

Embodiment 4

This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the chip defect detection method described in the first embodiment are implemented.

Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

The present application is described with reference to the flowchart and/or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and/or box in the flowchart and/or block diagram, and the combination of the process and/or box in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions can also be loaded into a computer or other programmable data processing device. A series of operational steps are performed on a computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows in the flowchart and/or one or more blocks in the block diagram.

Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims (10)

A chip defect detection method, characterized in that it includes: Acquire the one-dimensional vibration signal of the chip and convert it into two-dimensional image data; Construct a neural architecture search algorithm model ASNDARTS, and construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS; The method for constructing the neural architecture search algorithm model ASNDARTS is as follows: inputting the two-dimensional image data of the chip into the neural architecture search algorithm NAS for training, and optimizing the neural architecture search algorithm NAS during the training process, and obtaining the neural architecture search algorithm model ASNDARTS when the neural architecture search algorithm NAS reaches Nash equilibrium; The construction method of the knowledge distillation algorithm model DPSKD is: using the neural architecture search algorithm model ASNDARTS as a teacher network for knowledge distillation, and taking a single cell in the neural architecture search algorithm model ASNDARTS as a student network for knowledge distillation, wherein the neural architecture search algorithm model ASNDARTS includes a plurality of cells connected in series, and each of the cells is a result obtained by searching the neural architecture search algorithm model ASNDARTS in the search space; Based on the probability space, the knowledge transfer method between the teacher network and the student network is decoupled, and the hyperparameters of the student network are optimized. When the student network converges, the knowledge distillation algorithm model DPSKD is obtained; The one-dimensional vibration signal of the chip to be detected is obtained and converted into two-dimensional image data, and the two-dimensional image data of the chip to be detected is detected by the knowledge distillation algorithm model DPSKD to determine whether the chip to be detected has defects. The chip defect detection method according to claim 1 is characterized in that: the optimization of the neural architecture search algorithm NAS during the training process includes a first optimization method, specifically: During the training process, the probability of the original skip_connect operation of the neural architecture search algorithm NAS being selected is band-limited. The chip defect detection method according to claim 1 is characterized in that: the optimization of the neural architecture search algorithm NAS during the training process includes a second optimization method, specifically: During the training process, several operation modes OP are randomly selected from the search space of the algorithm NAS to form an initial search space Q1, and the remaining operation modes OP in the search space of the algorithm NAS are then combined to form a spare search space Q2. According to a preset criterion, it is determined whether adaptive adjustment is required between the operation modes OP in the initial search space Q1 and the operation modes OP in the spare search space Q2. If the operation mode OP requires adaptive adjustment, the operation mode OP with the lowest probability of being selected in the initial search space Q1 is replaced by the first operation mode OP in the spare search space Q2, and the operation mode OP with the lowest probability of being selected is used as the last operation mode OP in the spare search space Q2. The chip defect detection method according to claim 1 is characterized in that: the optimization of the neural architecture search algorithm NAS during the training process includes a third optimization method, specifically: During the training process, the number of operation modes OP of the node is adaptively expanded according to the model contribution rate of each operation mode OP. The chip defect detection method according to claim 2 is characterized in that: during the training process, the original skip_connect operation selection probability of the neural architecture search algorithm NAS is subjected to band-limited correction, and the method comprises: The selected probability value α _skip of the skip_connect operation during the training of the neural architecture search algorithm NAS is recorded in real time. The selected probability distribution of α _skip is P _skip . The point when P _skip shows a gradient rise phenomenon is defined as the algorithm NAS crash starting point epoch_break. The parameter k _skip is introduced to correct the α _skip of the skip_connect operation to obtain the corrected value α _{correct_skip} = α _skip *k _skip . The difference between α _skip and α _{correct_skip} at the algorithm NAS crash starting point is defined as △. The band-limited corrected probability is obtained according to α _skip , α _{correct_skip} and △. The formula is as follows:



Among them, * represents the multiplication sign, k _skip represents the restriction on the probability of skip_connect operation in the algorithm NAS search process; the first half of k _skip , k _skip(tanh) , is the limit on the number of early skip_connect operations at the epoch level based on the tanh activation function, and the second half of k _skip, k _{skip(sigmoid),} is the limit on the number of skip_connect operations at the edge level based on the sigmoid activation function; Indicates the probability of α _skip taking value before the crash starting point epoch_break; Indicates the probability of α _skip taking the value after the crash starting point epoch_break. The chip defect detection method according to claim 3 is characterized in that: the preset criterion includes: The preset criteria include, in order of judgment, the algorithm accuracy trend under equal training epoch intervals, whether there is an abnormal accuracy value under equal training epoch intervals, the OP contribution value trend of each operation mode under equal training epoch intervals, and the OP frequency of each node selected operation mode under equal training epoch intervals, wherein: The algorithm accuracy trend under the equal training epoch interval is as follows: obtain a preset number of equal training epoch intervals, count the algorithm accuracy value of each equal training epoch interval, and perform linear fitting on the algorithm accuracy values of all equal training epoch intervals to obtain a first linear fitting value. If the first linear fitting value is greater than 0, it indicates that the algorithm accuracy is on an upward trend; if the first linear fitting value is less than 0, then Indicates that the accuracy of the algorithm is on a downward trend; Whether there is an abnormal accuracy value under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, and the algorithm accuracy value of each equal training epoch interval is counted. If the algorithm accuracy value does not increase with the increase of the equal training epoch interval, it indicates that the algorithm accuracy value of the equal training epoch interval is abnormal, otherwise it is normal; The contribution value trend of each operation mode OP under the equal training epoch interval is as follows: a preset number of equal training epoch intervals are obtained, and at the same time, the selection probability of each operation mode OP in each equal training epoch interval is obtained, and the selection probability of OP in all equal training epoch intervals is linearly fitted to obtain a second linear fitting value, and if the second linear fitting value is greater than 0, it indicates that the contribution value of the operation mode OP is in an upward trend; if the second linear fitting value is less than 0, it indicates that the contribution value of the operation mode OP is in a downward trend; The frequency of the operation mode OP selected by each node under the equal training epoch interval is: obtain a preset number of equal training epoch intervals, and simultaneously obtain the number of times each operation mode OP is selected in all equal training epoch intervals, and according to the preset number of selections a, determine the relationship between the number of times each operation mode OP is selected and the preset number of selections a. The chip defect detection method according to claim 4 is characterized in that: the adaptive expansion of the number of operation modes OP of the node according to the model contribution rate of each operation mode OP comprises: In the algorithm NAS, the operation mode OP contribution discrimination mechanism is introduced into the predecessor connection edge of each node, and a preset number of equal training epoch intervals are obtained. At the same time, the selection probability of each operation mode OP in each equal training epoch interval is obtained, and the selection probability of OP in all equal training epoch intervals is linearly fitted to obtain the second linear fitting value. If the second linear fitting value is greater than 0, it means that the contribution value of the operation mode OP is on an upward trend; if the second linear fitting value is less than 0, it means that the contribution value of the operation mode OP is on a downward trend; During the training process, the current node selects the operation mode OP whose contribution value is an upward trend, ignores the operation mode OP whose contribution value is a downward trend, and completes the adaptive expansion of the number of operation mode OPs of the current node. The chip defect detection method according to claim 3 is characterized in that: the search space of the algorithm NAS includes several optional operation modes OP, and the operation modes OP include several max_pool_nxn, several avg_pool_nxn, skip_connect, several sep_conv_nxn, several dil_conv_nxn, and none, wherein max_pool represents maximum pooling, avg_pool represents average pooling, skip_connect represents skip connection, sep_conv represents depthwise separable convolution, dil_conv represents void convolution, none represents no operation, and n takes an odd number in the value range ∈ [1,9]. The chip defect detection method according to claim 1 is characterized in that: the knowledge transfer method between the teacher network and the student network based on the probability space decoupling method includes: The model ASNDARTS is probability reassigned and divided into three hierarchical category probability spaces, namely target space t, target class space O and overall space C. To separate the predictions of the three hierarchical category probability spaces, the symbol P = [p _t , p _O\t , p _\O ] is defined, where p _t represents the probability of target space t, p _O\t represents the probability of excluding target t in the target class space O, and p _\O represents the probability of excluding O in the overall space C.
Among them, _zt represents the soft probability of the target class, _zj represents the soft probability of all classes, _zk represents the soft profile of the non-target class, k represents the in-class index of the non-target class, and j represents the out-class index of the non-target class; Define the various probabilities in O\t space and \O as and Among them, ∧ represents the correction symbol of the final skip_connect, It means that in the target class space O, other independent similar probabilities of the target space t are excluded; similarly, represents the probability of other independent similar objects in the overall space C after excluding the target class space O, where
The knowledge distillation KD is decoupled into three-level category probability spaces, and the formula is:
make ⊙ represents dot product;
in, Represents the KL divergence TKD of the teacher-student model in the three defined spaces; Represents the KL divergence CKD of the teacher-student model on the non-target; It represents the KL divergence OKD of the teacher-student model among other classes; The KL divergence TKD, KL divergence CKD and KL divergence OKD of the three-level category probability space are reorganized, and the knowledge transfer between the teacher network and the student network is realized based on the reorganization results. A chip defect detection system, characterized in that it includes: Acquisition module: used to acquire the one-dimensional vibration signal of the chip and convert it into two-dimensional image data; Construction module: used to construct a neural architecture search algorithm model ASNDARTS, and to construct a knowledge distillation algorithm model DPSKD based on the neural architecture search algorithm model ASNDARTS; The method for constructing the neural architecture search algorithm model ASNDARTS is as follows: inputting the two-dimensional image data of the chip into the neural architecture search algorithm NAS for training, and optimizing the neural architecture search algorithm NAS during the training process, and obtaining the neural architecture search algorithm model ASNDARTS when the neural architecture search algorithm NAS reaches Nash equilibrium; The construction method of the knowledge distillation algorithm model DPSKD is: using the neural architecture search algorithm model ASNDARTS as a teacher network for knowledge distillation, and taking a single cell in the neural architecture search algorithm model ASNDARTS as a student network for knowledge distillation, wherein the neural architecture search algorithm model ASNDARTS includes a plurality of cells connected in series, and each of the cells is a result obtained by searching the neural architecture search algorithm model ASNDARTS in the search space; Based on the probability space, the knowledge transfer method between the teacher network and the student network is decoupled, and the hyperparameters of the student network are optimized. When the student network converges, the knowledge distillation algorithm model DPSKD is obtained; Detection module: used to obtain the one-dimensional vibration signal of the chip to be detected and convert it into two-dimensional image data, and detect the two-dimensional image data of the chip to be detected through the knowledge distillation algorithm model DPSKD to determine whether the chip to be detected has defects.