TWI866351B - Generative adversarial networks-based method for component anomaly detection and device thereof - Google Patents

Abstract

A generative adversarial networks-based method for component anomaly detection is provided. The method includes: deriving a first abnormal score of input data by a denoising auto-encoder, comprising: extracting a manifold coordinate of the input data, and calculating a distance between the manifold coordinate and an average coordinate of normal data as the first abnormal score; deriving a second abnormal score of the input data by a discriminator; and adding the first abnormal score and the second abnormal score to calculate an abnormal degree of the input data.

Description

Part abnormality detection method and device based on generative adversarial network (GAN)

The present disclosure relates to a method and device for detecting anomalies, and more particularly to a method and device for detecting anomalies of parts based on a generative adversarial network (GAN).

In the process of detecting whether machine parts are abnormal, the following problems are encountered:

(1) Since most of the data that can be collected in industrial applications is generated under normal operation, the amount of positive and abnormal data is uneven.

(2) Due to changes in production factors such as work orders, molds, and materials, a continuous production process will include multiple working conditions. Different working conditions will cause the AI model to have certain difficulties in distinguishing normal/abnormal conditions.

(3) The training data of supervised algorithms needs to have clear labels (e.g., the type of factor causing the anomaly). In industrial applications, many anomalies rarely occur or have never occurred. Unsupervised algorithms have better adaptability, but some unsupervised algorithms cannot adapt to problems under various working conditions.

When the amount of positive/abnormal data is unbalanced, the AI model training performance will be poor due to the small amount of abnormal data available. In addition, since machine parts have a variety of working conditions, most algorithms cannot effectively identify the actual abnormal state.

Therefore, a method and device for detecting abnormal parts based on generating an adversarial network are needed to improve the above problems.

Therefore, the main purpose of the present disclosure is to provide a component abnormality detection method and device based on generating a countermeasure network.

The present disclosure provides a method for detecting abnormal parts in an industrial automation system, including: inferring a first abnormal score of input data by a noise reduction self-encoder, including: extracting a manifold coordinate of the above-mentioned input data, and calculating a distance between the above-mentioned manifold coordinate and an average coordinate of normal data as the above-mentioned first abnormal score; generating a second abnormal score of the above-mentioned input data by a discriminator; and adding the above-mentioned first abnormal score and the above-mentioned second abnormal score to calculate an abnormal degree of the above-mentioned input data.

The present disclosure provides a component abnormality detection device, comprising: a processor; and a computer storage medium, coupled to the processor and configured to store computer-readable instructions for instructing the processor to execute the component abnormality detection method.

Based on the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP), the present invention uses a denoising autoencoder to replace the generator of WGAN-GP and retains the discriminator of WGAN-GP to form an AI model architecture. The disclosed method and device for detecting abnormal parts based on the generative adversarial network have the following advantages:

(1) The noise reduction self-encoder can extract the manifold coordinates of the input data. When the dimension of the input data is higher and different, the overlap probability of the manifold coordinates is lower, so the identification degree is higher. Therefore, in industrial automation applications, when machine parts face complex working conditions, the part abnormality detection method and device based on the generated adversarial network disclosed in the present invention have a higher identification degree.

(2) The AI model architecture disclosed herein utilizes both a denoising autoencoder and a discriminator for inference, which can improve the discrimination accuracy.

The disclosed embodiment provides a method and device for detecting abnormal parts based on a generative adversarial network. In industrial automation applications, when it is difficult to obtain abnormal data, an AI model architecture based on a generative adversarial network (GAN) is trained with normal training data to solve the problem of difficulty in distinguishing normal/abnormal states in various working conditions.

FIG. 1 is a schematic diagram of a component abnormality detection system 10 according to an embodiment of the present disclosure. The component abnormality detection system 10 includes a belt 110, a plurality of drive wheels 112, a variable frequency drive (VFD) 12, and a controller 13, wherein the controller 13 can be equipped with an inference system 30.

As shown in FIG. 1 , the belt 110 is wound around a plurality of drive wheels 112, which are connected to and controlled by a motor (not shown), and the motor is connected to and controlled by a variable frequency drive 12. When the motor continues to operate and drives the belt 110, the controller 13 collects part parameters (e.g., motor current, motor torque, motor speed, and motor operation cycle) through the variable frequency drive 12 to generate a corresponding data set. The inference system 30 is configured to perform online inference based on the input real-time data to predict the abnormality of the part.

Generally speaking, the inference system 30 must be trained before it can infer the abnormality of the part. In this embodiment, the inference system 30 is trained by the controller 13, wherein the inference system 30 receives the part parameters to generate a data set and performs training based on the data set. In one embodiment, the controller 13 outputs the data set to an external device, and the inference system 30 is trained by the external device (e.g., the electronic device 70 of FIG. 7), and then the external device outputs the trained inference system 30 to the controller 13 for online inference. In another embodiment, the controller 13 outputs the data set to an external device, and the inference system 30 is trained by the external device, and performs online inference in the external device. The advantage of using an external device to train the inference system 30 is that a more powerful computer or processor can be used to speed up the training process. The inference system 30 can be stored in a memory built into or connected to the controller 13 or an external device.

In one embodiment, the controller 13 may be, for example, a central processing unit (CPU), a micro control unit (MCU), a system on chip (SoC), or a programmable logic controller (PLC), but is not limited thereto.

FIG. 2 is a schematic diagram of a model training system 20 according to an embodiment of the present disclosure. The model training system 20 includes a data collection module 28, a data preprocessing module 21, and a model training module 22. The model training system 20 can be installed in the controller 13 or an external training device (e.g., the electronic device 70 in FIG. 7).

In the data collection module 28, relevant historical data for model training must be collected before model training is performed. Taking motor operation as an example, motor current, motor torque, motor speed, and operation cycle can be collected as historical data. In one embodiment, the relevant historical data is numerical data or pictures. A portion of the historical data can be used as training data, and another portion can be used as test data.

The data preprocessing module 21 is configured to perform statistical analysis on the training data and the test data to obtain statistical values. Then, the data preprocessing module 21 is configured to select multiple influential features from the statistical values to generate a feature set for subsequent model training and testing. Specifically, when performing model training and online inference, the data preprocessing module 21 is configured to delete parameters and data that do not match the feature set, and retain parameters and data that match the feature set, so that this data is used as input data in the model training module 22 and the inference system 30.

The model training module 22 includes a denoising auto-encoder (DAE) 23, a discriminator (Discriminator) 24, a noise generator 25, a first target function 26 and a second target function 27, wherein the denoising auto-encoder 23 and the discriminator 24 conform to a deep neural network (DNN) model architecture.

In operation, the noise generator 25 adds noise to normal data to generate a noisy data, and then outputs the noisy data to the noise reduction self-encoder 23. The noise reduction self-encoder 23 is an initial noise reduction self-encoder configured to convert the noisy data into a generated data, and input the generated data to the discriminator 24.

The discriminator 24 is an initial discriminator, which is configured to generate a discriminant value Dg of generated data and a discriminant value Dn of real data, respectively, and feed back to the noise reduction self-encoder 23, wherein the discriminant values Dg and Dn are arbitrary real numbers, indicating the degree to which the input data is real. Our experimental results show that this real number range falls between 1 and -1. The closer this real number is to 1, the higher the degree to which the input data is regarded as real data; the closer this real number is to -1, the higher the degree to which the input data is regarded as generated data. The discriminator 24 calculates the first expected value based on the discriminant values Dg and Dn, and adjusts the parameters to achieve the maximized first objective function 26. Through multiple trainings, the performance of the discriminator 24 can be gradually optimized. The noise reduction self-encoder 23 calculates the second expected value according to the feedback identification values Dg and Dn, adjusts the parameters, and then performs the next training to achieve the minimized second objective function 27. Through multiple trainings, the performance of the noise reduction self-encoder 23 can be gradually optimized.

Furthermore, the generated data and the real data (i.e., normal data) are input to the discriminator 24 to generate the discriminant values Dg and Dn, and the first and second expected values are calculated, so that the noise reduction self-encoder 23 and the discriminator 24 adjust the parameters according to their respective target functions, wherein the formula of the first target function 26 of the discriminator 24 is as follows: The formula of the second objective function 27 of the noise reduction self-encoder 23 is as follows: P _data and P _DAE are the probability distributions of real data (i.e. normal data) and generated data, respectively. , The first objective function 26 aims to maximize the difference between the discriminant values Dg and Dn, that is, the discriminator 24 has a better ability to distinguish between real data and synthetic data, while the second objective function 27 aims to minimize the difference between the discriminant values Dg and Dn, that is, the generated data is closer to the real data, resulting in the discriminator 24 being unable to effectively distinguish. In addition, a penalty term (Penalty) is added to improve the problem that GAN often fails to converge during training.

Specifically, the discriminator 24 refers to the first target function To adjust the parameters, after multiple parameter iterations, the discriminator 24 can be gradually optimized; the noise reduction self-encoder 23 refers to the second objective function The parameters are adjusted, and after multiple parameter iterations, the noise reduction self-encoder 23 can be gradually optimized. The parameter adjustment will continuously repeat the following steps, including: (1) fixing the parameters of the noise reduction self-encoder 23 and adjusting the parameters of the discriminator 24 until the discrimination difference is maximized; (2) fixing the parameters of the discriminator 24 and adjusting the parameters of the noise reduction self-encoder 23 until the discrimination difference is minimized.

FIG. 3 is a schematic diagram of an inference system 30 according to an embodiment of the present disclosure. The inference system 30 includes a data preprocessing module 31 and an AI model 32, wherein the AI model 32 includes a noise reduction self-encoder 33, a discriminator 34, a digital converter 35, a subtractor 36, and an adder 37. The inference system 30 can be installed in the controller 13 or an external detection device (e.g., the electronic device 70 in FIG. 7).

In operation, the inference system 30 obtains real-time data of the operation of the machine parts through the controller 13; taking the operation of the motor as an example, the motor current, motor torque, motor speed and operation cycle can be collected as real-time data. The data pre-processing module 31 pre-processes the real-time data 310 to generate input data to the noise reduction self-encoder 33 and the discriminator 34. The noise reduction self-encoder 33 and the discriminator 34 are both noise reduction self-encoders and discriminators trained by the second figure. The noise reduction self-encoder 33 is configured to extract the manifold coordinates (Manifold) of the input data, and then the subtractor 36 calculates the distance between the manifold coordinates and the average coordinates of the normal data as the first abnormal score. In one embodiment, the dimension of the manifold coordinates corresponds to the number of nodes in the hidden layer of the denoising self-encoder. The discriminator 34 infers the discriminator value D(x) of the real-time data, where the discriminator value D(x) is an arbitrary real number. The larger the discriminator value D(x), the greater the similarity between the input data and the normal data; and the smaller the discriminator value D(x), the smaller the similarity between the input data and the normal data. The discriminator value D(x) can be expressed by the following formula: D( x ) = W× x + b, where D( x ) represents the judgment result of the discriminator 34 on the input data x , W refers to the weight matrix of the last layer of the neural network, and b is the bias. The role of the bias weight is to introduce an additional parameter into the discriminator 34 so that the discriminator 34 can better fit the input data to improve performance, wherein the bias weight b is usually initialized to a small constant. The discriminator 34 will finally convert the judgment result through an activation function and obtain the identification value, wherein the activation function can be a linear function.

The digital converter 35 is configured to convert the identification value D(x) into a second abnormal score. In order to make the size of the identification value D(x) intuitively represent the similarity between the input data and the normal data, in one embodiment, the digital converter 35 multiplies the identification value D(x) by -1 to convert it into the second abnormal score. In another embodiment, the digital converter 35 subtracts the identification value D(x) from a preset value to convert it into the second abnormal score. In another embodiment, when the identification value D(x) is greater than zero and less than 1, the digital converter 35 takes the reciprocal of the identification value D(x) as the second abnormal score. Finally, the adder 37 adds the first abnormal score and the second abnormal score to calculate the abnormality of the input data. The larger the value of the abnormality, the greater the abnormality between the input data and the normal data; conversely, the smaller the value of the abnormality, the smaller the abnormality between the input data and the normal data.

It is worth noting that although the object of abnormal detection in Figures 1 to 3 is a motor as an example, the present disclosure should not be limited to this. As an example, the present disclosure can also be applied to a computer numerical control (CNC) milling machine process. The tool condition, feed rate, clamping pressure and other values of the parts collected from the CNC machine can be used to determine whether the wear state of the parts is normal or abnormal.

FIG. 4 is a flow chart of a method 40 for training an AI model according to an embodiment of the present disclosure. The method 40 for training an AI model may be executed by the controller 13 of FIG. 1 or the electronic device 70 of FIG. 7 and includes the following steps.

Step S41: inferring identification values of multiple sets of input data by the discriminator, wherein the multiple sets of input data include real data and generated data.

Step S42: Using the identification values of the real data and the generated data, a first expected value is obtained, and a first objective function is calculated.

Step S43: Determine whether the first objective function is maximized, if so, proceed to step S45; if not, proceed to step S44.

Step S44: fix the noise reduction self-encoder parameters and adjust the parameters of the discriminator. Return to step S41.

Step S45: The discriminator is used to infer the discriminant values of the real data and the generated data, and the discriminant values are fed back to the noise reduction self-encoder to calculate the second expected value.

Step S46: Obtain a second expected value and calculate a second objective function.

Step S47: Determine whether the second objective function is minimized. If yes, end; if no, proceed to step S48.

Step S48: Fix the discriminator parameters and adjust the noise reduction self-encoder parameters. Return to step S45.

Steps S41, S42, S43, S44, and S45 may be performed by the discriminator 24, steps S45, S46, S47, and S48 may be performed by the noise reduction self-encoder 23, and steps S42 and S46 may be performed by the first and second objective functions. In one embodiment, after step S48, the controller 13 or the processor may determine whether to return to step S41 for training again according to the situation. By repeatedly executing the training AI model method 40, the performance of the noise reduction self-encoder 23 and the discriminator 24 of FIG. 2 may be gradually optimized to be applied to the noise reduction self-encoder 33 and the discriminator 34 in the AI model 32 of FIG. 3.

FIG. 5 is a flow chart of a component abnormality detection method 50 based on generating a countermeasure network according to an embodiment of the present disclosure. The component abnormality detection method 50 can be executed by the controller 13 of FIG. 1 or the electronic device 70 of FIG. 7 and includes the following steps.

Step S51: inferring a first anomaly score of the input data by using the noise reduction self-encoder. In this embodiment, step S51 includes step S11: extracting manifold coordinates of the input data; and step S12: calculating the distance between the manifold coordinates and the average coordinates of the normal data as the first anomaly score.

Step S52: Inferring a second abnormality score of the input data by the discriminator. In this embodiment, step S52 includes step S13: Inferring a discriminator value of the input data by the discriminator; and step S14: Converting the discriminator value into the second abnormality score.

Step S53: Add the first abnormal score and the second abnormal score to calculate the abnormality level of the input data. In one embodiment, when the abnormality level is higher than a custom threshold value, the controller 13 or the electronic device 70 determines that the input data is abnormal and sends a warning signal to notify the user.

Steps S51 and S11 can be executed by the noise reduction encoder 33, step S12 can be executed by the subtractor 36, steps S52 and S13 can be executed by the discriminator 34, step S14 can be executed by the digital converter 35, and step S53 can be executed by the adder 37.

Before the part abnormality detection method 50 is started, real-time data can be obtained through the controller 13, and the real-time data can be pre-processed (including statistical analysis, deleting non-matching feature set data and retaining matching feature set data) to generate input data. By executing the part abnormality detection method 50, the controller 13 or the electronic device 70 can perform online inference to generate the abnormality degree of the part and then determine whether the input data is abnormal.

FIG. 6 is a data characteristic coordinate diagram 60 of manifold coordinates according to an embodiment of the present disclosure. The historical data, training data, test data, and input data used in the embodiment of the present disclosure may be numerical data or images. FIG. 6 uses images as examples to illustrate that different types of images are each clustered in a low-dimensional manifold space in a high-dimensional space, and the manifold coordinates of different types of images are almost non-overlapping. In other words, the higher the dissimilarity and the higher the dimension, the lower the probability of overlap, and therefore the higher the discrimination.

As described above, the disclosed method and device for detecting abnormal parts only need to train the noise reduction self-encoder and the discriminator with normal training data to solve the problem that the amount of abnormal data is too small and cannot be distinguished under various working conditions.

It should be noted that the noise reduction self-encoders 23, 33 and the discriminators 24, 34 in FIG. 2 to FIG. 3 are configured to be implemented in hardware, software, firmware or any combination thereof. For example, the noise reduction self-encoders 23, 33 and the discriminators 24, 34 can be implemented as computer program codes configured to be executed in one or more processors. Alternatively, the noise reduction self-encoders 23, 33 and the discriminators 24, 34 can be implemented in hardware logic/circuitry.

FIG. 7 is a schematic diagram of an electronic device 70 according to an embodiment of the present invention.

The present invention may be implemented in computer program code or machine-usable instructions to perform the present invention, which may be computer-executable instructions of a program module, which is executed by a computer or other machine, such as a personal digital assistant or other portable device. Generally speaking, a program module includes routines, programs, objects, components, data structures, etc. A program module refers to a program code that performs a specific task or implements a specific abstract data type. The present invention can be implemented in a variety of system configurations, including portable devices, consumer electronic products, general-purpose computers, more professional computing devices, etc. The present invention can also be implemented in a distributed computing environment, processing devices connected by a communication network.

7. Electronic device 70 includes bus 77 that directly or indirectly couples the following devices, memory 71, one or more processors 72, one or more display elements 73, input/output (I/O) ports 74, peripheral components 75, and illustrative power supply 76. Bus 77 represents a component that can be one or more buses (e.g., an address bus, a data bus, or a combination thereof).

The memory 71 includes but is not limited to random access memory (RAM), read-only memory (ROM), electrically-erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage devices, magnetic disks, magnetic disks, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store the required information and can be accessed by the electronic device 70.

The electronic device 70 includes a processor that reads data from various entities such as a memory 71 or a peripheral component 75. The display component 73 displays data or indications to a user or other device, such as a display device, a speaker, a printing component, a vibration component, etc.

I/O port 74 allows electronic device 70 to be logically connected to other devices including peripheral components 75, some of which are built-in devices. Exemplary components include microphones, joysticks, game consoles, satellite dish signal receivers, scanners, printers, wireless devices, etc. Peripheral components 75 can provide a user interface for processing user-generated gestures, sounds, or other physiological inputs. In some examples, these inputs can be transmitted to a suitable network component for further processing. Electronic device 70 can be equipped with a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, and a combination of these systems to detect and identify objects.

In addition, the processor 72 in the electronic device 70 may also execute the program code 78 and instructions in the memory 71 to present the actions and steps of the above-mentioned embodiments, or other descriptions of the contents in the specification.

This disclosure has been disclosed as an implementation example as above. Anyone familiar with this technology can make some changes and modifications without departing from the spirit and scope of this disclosure. Therefore, the protection scope of this case shall be defined by the scope of the attached patent application.

10: Parts abnormality detection system 110: Belt 112: Drive wheel 12: Variable frequency drive 13: Controller 20: Model training system 21: Data preprocessing module 22: Model training module 23: Noise reduction self-encoder 24: Discriminator 25: Noise generator 26: First objective function 27: Second objective function 28: Data collection module 30: Inference system 31: Data preprocessing module 32: AI model 33: Noise reduction self-encoder 34: Discriminator 35: Digital converter 36: Subtractor 37: Adder 40: AI model training method S41~S48: Steps 50: Parts abnormality detection method S51, S52, S53, S11, S12, S13, S14: Steps 60: Data characteristic coordinate diagram 70: Electronic device 71: Memory 72: Processor 73: Display element 74: I/O port 75: Peripheral components 76: Power supply 77: Bus 78: Program code

FIG. 1 is a schematic diagram of a part anomaly detection system according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram of a model training system according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram of an inference system according to an embodiment of the present disclosure. FIG. 4 is a flow chart of a method for training an AI model according to an embodiment of the present disclosure. FIG. 5 is a flow chart of a method for detecting anomalies according to an embodiment of the present disclosure. FIG. 6 is a data characteristic coordinate diagram of a manifold coordinate according to an embodiment of the present disclosure. FIG. 7 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.

50: Methods

S51,S52,S53,S11,S12,S13,S14: Steps

Claims (8)

A component anomaly detection method based on a generative adversarial network comprises: 1) inferring a first anomaly score of an input data by a noise reduction self-encoder, comprising: 11) extracting a manifold coordinate of the input data; and 12) calculating a distance between the manifold coordinate and an average coordinate of normal data as the first anomaly score; 2) inferring a second anomaly score of the input data by a discriminator, comprising: 21) inferring a first identification value of the input data by the discriminator; and 22) converting the first identification value The identification value is the second abnormal score, including one of the following steps: multiplying the first identification value by -1 to convert it into the second abnormal score, wherein the identification value is an arbitrary real number; subtracting a preset value from the first identification value to convert it into the second abnormal score; and when the first identification value is greater than zero and less than 1, taking the reciprocal of the first identification value as the second abnormal score; and 3) adding the first abnormal score and the second abnormal score to calculate an abnormal degree of the input data. The method for detecting abnormal parts of claim 1, wherein before the above step 1), it further includes: Preprocessing a real-time data to generate processed data; deleting the processed data that does not match a feature set; and retaining the processed data that matches the feature set as the input data. The method for detecting abnormal parts of claim 1, wherein before the above step 1), a training process is further included, including: A1) inferring a second identification value of multiple sets of input data by an initial identifier, wherein the multiple sets of input data include at least one real data and at least one generated data; A2) calculating a first expected value by the initial identifier according to the second identification value; A3) inputting the first expected value A4) when the first target function is not maximized, the noise reduction self-encoder parameters are fixed, and the parameters of the initial discriminator are adjusted according to the first expected value; and A5) steps A1) to A4) are repeated to train the initial discriminator until the first target function is maximized to generate the discriminator. As in claim 3, the part abnormality detection method, wherein the training process further includes: A6) inferring a third identification value of the generated data and the at least one real data by the initial discriminator, and feeding it back to the denoising self-encoder; A7) calculating a second expected value by the denoising self-encoder according to the third identification value; A8) inputting the second expected value into a second target function to determine whether the second target function is minimized; A9) when the second target function has not been minimized, fixing the parameters of the discriminator, and adjusting the parameters of the initial denoising self-encoder according to the second expected value; and A10) repeating the steps A6) to A9) to train the initial denoising self-encoder until the second target function is minimized to generate the denoising self-encoder. As in claim 3, the method for detecting abnormal parts, wherein the above step A1) includes: adding noise to the above normal data to generate noisy data; and converting the above noisy data into the above generated data by the above initial discriminator, and inputting the above generated data into the above discriminator. The method for detecting abnormal parts of claim 3, wherein before the above training process, a pre-processing process is further performed, including: B1) selecting multiple influential features from the statistical values of a training data set through a feature selection method to generate a feature set; and B2) selecting data and parameters matching the above feature set as the training data set to provide to the model training stage. The part abnormality detection method of claim 1 further includes: 4) when the abnormality level is higher than a custom valve value, the input data is judged to be abnormal. A device for detecting abnormal parts, comprising: a processor; and a computer storage medium coupled to the processor and configured to store computer-readable instructions for instructing the processor to execute the abnormal parts detection method as described in claim 1.

Images