CN107505837A - A kind of semi-supervised neural network model and the soft-measuring modeling method based on the model - Google Patents

Abstract

The invention discloses a kind of soft-measuring modeling method based on semi-supervised neural network model, the model is divided into three layers, first layer is input layer, the second layer is hidden layer, third layer is output layer, output layer is divided into self-encoding encoder output layer and neural network model output layer, self-encoding encoder and neural network model share input layer and hidden layer, the modeling method is made up of self-encoding encoder and neutral net, it is few can effectively to have solved exemplar, caused by unlabeled exemplars are more the problem of soft sensor modeling inaccuracy, so as to establish more accurate semi-supervised soft-sensing model, the monitoring of implementation process and corresponding control.

Description

A semi-supervised neural network model and a soft sensor modeling method based on the model

technical field

The invention belongs to the field of industrial process prediction and control, and relates to a semi-supervised neural network model and a soft sensor modeling method based on the model.

Background technique

In the actual industrial production process, there are often more or less key process variables that cannot be detected online. In order to solve this problem, a variable that is easier to detect in the collection process is constructed according to an optimal standard. With these variables as input and key process variables as output mathematical model, the online estimation of key process variables is realized. This is the soft sensor modeling commonly used in industrial processes.

The development of statistical process soft-sensing modeling has an extremely significant demand for large-scale industrial data. However, there are still many problems in soft sensor modeling. The complexity of the system in the industrial process is also increasing day by day, and the nonlinear relationship in the process data is becoming more and more prominent. If you still use the traditional linear method to establish a soft sensor model, it is undoubtedly not up to the task of accurately predicting variables. For nonlinear process characteristics, There are models such as the neural network kernel method. Among many models, the adaptability of the neural network model and the fitting ability of the nonlinear process are extremely strong, and the task of variable prediction of the industrial process can be accurately completed.

At the same time, in many cases, labeled samples in machine learning problems are extremely rare and rare, and unlabeled samples are easy to obtain but the manual labeling process is difficult. How to fully extract the useful information in the unlabeled data to improve the performance of the model, so the semi-supervised field has attracted more and more attention and attention.

Contents of the invention

Aiming at the problems of few labeled samples, many unlabeled samples and serious process nonlinearity in the current industrial process, the present invention proposes a soft sensor modeling method based on semi-supervised neural network, which combines autoencoder and neural network model Combined with semi-supervised soft sensor modeling for industrial processes, accurate online estimation of key process variables is realized. The specific technical solutions are as follows:

A kind of semi-supervised neural network model, described model is made up of autoencoder and neural network, is divided into three layers, the first layer is input layer, the second layer is hidden layer, the third layer is output layer, autoencoder The input layer and the hidden layer are shared with the neural network model, and the output layer is divided into the output layer of the autoencoder and the output layer of the neural network model. The input variable of the input layer is x, and the weight and bias from the input layer to the hidden layer are ω ₁ and b ₁ , the weights and biases from the hidden layer to the output layer of the neural network are ω _y and b _y , the weights and biases from the hidden layer to the output layer of the autoencoder are ω ₂ and b ₂ , and the weights and biases output from the output layer of the autoencoder configuration value The predicted value of the output layer output of the neural network model is

A soft sensor modeling method based on the above-mentioned semi-supervised neural network model, the steps are as follows:

Step 1: Collect historical industrial process data to form a training data set for modeling. The training data set includes a labeled data set L containing both leading variables and auxiliary variables, L∈R ^n×d , and only An unlabeled data set U containing auxiliary variables, U∈R ^N×M , n represents the number of data samples in the labeled data set, d represents the number of process variables, R is a real number set, and N represents the data samples in the unlabeled data set The number, M represents the number of auxiliary variables in the unlabeled data set;

Step 2: Standardize the collected training data set, and transform the process variable into a new data set with a mean value of 0 and a variance of 1;

Step 3: Use the auxiliary variables x _l and x _u in the standardized labeled data set and unlabeled data set as the input variable x of the model, and use the leading variable in the standardized labeled data set as the output variable y for semi-supervised Neural network model training, so as to obtain the predicted value of the leading variable output by the output layer of the semi-supervised neural network model and the reconstructed value corresponding to the input variable x output by the output layer of the autoencoder model Further get the entire prediction error of the semi-supervised neural network model:

E＝σ*E _ae +(1-σ)*E _nn +λE _weight

where E _ae represents the reconstruction error of the autoencoder,

E _nn represents the prediction error of the neural network,

Represents the regularization constraint on the weight; σ to control the balance between E _ae and E _nn , λ is the regularization coefficient, which is an empirical value;

Step 4: Calculate the gradient of relevant weights and offsets using the backpropagation algorithm;

Step 5: Continuously train the semi-supervised neural network model according to the gradient descent method, calculate the optimal parameters of the model, and complete the semi-supervised neural network model modeling process;

Step 6: Collect new industrial process data, repeat steps 1 to 2, and substitute the processed industrial process data into the optimized semi-supervised neural network model to obtain the predicted value of the leading variable In order to realize the monitoring and control of the process.

Description of drawings

Fig. 1 is a semi-supervised neural network model structure diagram;

Fig. 2 is the process structure of debutanizer;

Fig. 3 shows the sample real value and semi-supervised neural network model prediction value effect diagram when the label ratio is 5%;

Fig. 4 shows the effect diagram of the actual value of the sample and the predicted value of the traditional neural network model when the label ratio is 5%;

detailed description

The present invention will be further described in detail below in combination with specific embodiments.

A kind of semi-supervised neural network model, described model is made up of autoencoder and neural network, is divided into three layers, the first layer is input layer, the second layer is hidden layer, the third layer is output layer, autoencoder The input layer and the hidden layer are shared with the neural network model, and the output layer is divided into the output layer of the autoencoder and the output layer of the neural network model. The input variable of the input layer is x, and the weight and bias from the input layer to the hidden layer are ω ₁ and b ₁ , the weights and biases from the hidden layer to the output layer of the neural network are ω _y and b _y , the weights and biases from the hidden layer to the output layer of the autoencoder are ω ₂ and b ₂ , and the weights and biases output from the output layer of the autoencoder configuration value The predicted value of the output layer output of the neural network model is

A soft sensor modeling method based on the above-mentioned semi-supervised neural network model, the steps are as follows:

Step 1: Collect historical industrial process data to form a training data set for modeling. The training data set includes a labeled data set L containing both leading variables and auxiliary variables, L∈R ^n×d , and only An unlabeled data set U containing auxiliary variables, U∈R ^N×M , n represents the number of data samples in the labeled data set, d represents the number of process variables, R is a real number set, and N represents the data samples in the unlabeled data set The number, M represents the number of auxiliary variables in the unlabeled data set;

Step 2: Standardize the collected training data set, and transform the process variable into a new data set with a mean value of 0 and a variance of 1;

Step 3: Use the auxiliary variables x _l and x _u in the standardized labeled data set and unlabeled data set as the input variable x of the model, and use the leading variable in the standardized labeled data set as the output variable y for semi-supervised Neural network model training, so as to obtain the predicted value of the leading variable output by the output layer of the semi-supervised neural network model and the reconstructed value corresponding to the input variable x output by the output layer of the autoencoder model Further get the entire prediction error of the semi-supervised neural network model:

E＝σ*E _ae +(1-σ)*E _nn +λE _weight

where E _ae represents the reconstruction error of the autoencoder,

E _nn represents the prediction error of the neural network,

Represents the regularization constraint on the weight; σ to control the balance between E _ae and E _nn , λ is the regularization coefficient, which is an empirical value;

Step 4: Calculate the gradient of relevant weights and offsets using the backpropagation algorithm;

(1) Introduce the error parameters of the output layer of the autoencoder where the superscript 3 represents the output layer, the subscript j represents the jth neuron of the output layer, and z _j represents the weighted input value of the jth neuron of the hidden layer, in, Indicates the weight on the connection from the kth neuron of the input layer to the jth neuron of the hidden layer, Indicates the bias of the jth neuron in the hidden layer, and x _k represents the input of the kth neuron in the input layer;

(2) Calculate the weight and bias gradient from the hidden layer to the output layer of the autoencoder according to the backpropagation algorithm in Indicates the weight on the connection from the kth neuron in the hidden layer to the jth neuron in the output layer of the autoencoder, Represents the bias of the jth neuron in the output layer of the autoencoder, and a _k represents the output of the kth neuron in the hidden layer;

(3) Introduce the error of the output layer of the neural network Among them, the superscript 3 indicates the output layer, and the subscript j indicates the jth neuron of the output layer;

(4) Calculate the gradient from the hidden layer to the neural network output layer weight and bias according to the backpropagation algorithm in Represents the weight on the connection from the kth neuron in the hidden layer to the jth neuron in the output layer of the neural network, Represents the bias of the jth neuron in the output layer of the neural network, and a _k represents the output of the kth neuron in the hidden layer;

(5) Calculate the error from the input layer to the hidden layer:

For the hidden layer, the loss function used when calculating the error is the overall prediction error. When calculating the error of the hidden layer, there are two cases, one is labeled data, and the other is unlabeled data.

a) The error of labeled data comes from both the prediction error of the neural network and the reconstruction error of the autoencoder, calculated as follows:

Indicates the error of the labeled data of the jth neuron of the hidden layer, Indicates the error of the kth neuron in the output layer of the autoencoder, Indicates the error of the kth neuron in the output layer of the neural network, Indicates the weight on the connection from the jth neuron in the hidden layer to the kth neuron in the output layer of the autoencoder, Represents the weight on the connection between the jth neuron in the hidden layer and the kth neuron in the output layer of the neural network, and f'(z _j ) represents the derivative of the activation function of the hidden layer neuron;

b) The error of unlabeled data is only from the reconstruction error of the autoencoder:

Indicates the error of the unlabeled data of the jth neuron of the hidden layer;

c) Calculate the gradient from the input layer to the hidden layer weights and biases

Among them, x _{k, u} represent the input value of the kth neuron in the input layer of unlabeled data, and x _{k, l} represent the input value of the kth neuron in the input layer of labeled data;

Step 5: Continuously train the semi-supervised neural network model according to the gradient descent method, calculate the optimal parameters of the model, and complete the modeling process of the semi-supervised neural network model.

Step 6: Collect new industrial process data, repeat steps 1 to 2, and substitute the processed industrial process data into the optimized semi-supervised neural network model to obtain the predicted value of the leading variable In order to realize the monitoring and control of the process.

In order to better illustrate the structure of the semi-supervised neural network model, suppose the input variable is x, the number of neurons in the input layer is 3, and the number of neurons in the hidden layer is 4, because the autoencoder reconstructs the input variable x, so The number of output neurons of the autoencoder is the same as that of the input, and the number of output neurons of the neural network model is 2. The structure of the semi-supervised neural network model at this time is shown in Figure 1.

The performance of the semi-supervised neural network model is illustrated below with a specific example of a debutanizer. The debutanizer is a commonly used standard industrial process platform for the validation of soft-sensing modeling algorithms. The debutanizer is an important device in the refining process. The structure is shown in Figure 2. The purpose of this device is to remove propane and butane from naphtha gas. The butane content at the bottom of the tower is A very important key indicator, in order to improve the control quality of the debutanizer, it is necessary to establish a soft-sensing model for the butane content in the bottom of the tower.

Table 1 shows the seven auxiliary variables selected for the key quality variable butane content, which are tower top temperature, tower top pressure, reflux flow rate, next stage flow rate, temperature of sensitive plate, tower bottom temperature and tower bottom pressure . For this process, 2394 process data were collected continuously at equal time intervals, 1197 of which were modeled as training samples, and their corresponding butane content values were analyzed and marked offline. In addition, 1197 data samples collected are used as test samples to verify the effectiveness of the semi-supervised neural network model of the present invention. In the process of selecting the training set and the test set, the interval sampling method of including two adjacent sample points in each space into the training set and the test set is adopted. Randomly select a certain proportion of data in the training set as labeled samples, and remove the labeled samples from the training set as unlabeled samples.

Table 1: Input variable description

input variable variable description _x1 Top temperature _x2 Top pressure _x3 return flow _x4 next level traffic _x5 Sixth tray temperature X ₆ Bottom temperature 1 X ₇ Bottom temperature 2

In order to evaluate the prediction accuracy of the semi-supervised neural network model, the error standard root mean square error (RMSE) is defined in the traditional way, and the calculation formula is as follows:

Where M is the number of test samples, y _j is the true value of the leading variable, is the predicted value of the semi-supervised neural network model of the leading variable.

In Figure 3-Figure 4, Figure 3 shows the curve of the predicted value and the real value of the semi-supervised neural network model, Figure 4 shows the curve of the predicted value and the real value of the traditional neural network model, through Figure 3-Figure 4, you can see The fitting effect of semi-supervised neural network model of the present invention is better, and the RMSE=0.16261 of model semi-supervised neural network model of the present invention simultaneously, and the RMSE=0.24076 of traditional neural network, the predictive precision of semi-supervised neural network model will be high than traditional neural network models. The model of the invention is superior to the traditional neural network model, and the accuracy is further improved.

Claims (2)

1. A semi-supervised neural network model is composed of self-encoder and neural network, and is divided into three layers, the first layer is input layer, the second layer is hidden layer, the third layer is output layer, the self-encoder and neural network model share input layer and hidden layer, the output layer is divided into self-encoder output layer and neural network model output layer, the input variable of input layer is x, and the weight and bias from input layer to hidden layer are omega respectively₁And b₁The weight and bias of the hidden layer to the output layer of the neural network is omega_yAnd b_yHidden layer to self codingThe weight and offset of the output layer of the filter are omega₂And b₂The reconstructed value of x output from the encoder output layer isThe predicted value output by the output layer of the neural network model is

2. A soft measurement modeling method based on the semi-supervised neural network model of claim 1, comprising the following steps:

collecting data of historical industrial processes to form a training data set for modeling, wherein the training data set comprises labeled data sets L, L ∈ R containing main variables and auxiliary variables^n×dAlso included are unlabeled datasets U, U ∈ R containing only auxiliary variables^N×MN represents the number of data samples of the labeled data set, d represents the number of process variables, R is a real number set, N represents the number of data samples of the unlabeled data set, and M represents the number of auxiliary variables of the unlabeled data set;

step two: standardizing the collected training data set, and quantizing the process variables into a new data set with a mean value of 0 and a variance of 1;

step three: normalizing the auxiliary variable x in the labeled dataset and the unlabeled dataset_lAnd x_uTaking the main variable in the standardized labeled data set as an output variable y as an input variable x of the model, and performing semi-supervised neural network model training to obtain a predicted value of the main variable output by a neural network model output layer in the semi-supervised neural network modelAnd a reconstructed value corresponding to the input variable x output from the encoder model output layerFurther obtaining the whole prediction error of the semi-supervised neural network model:

E＝σ*E_ae+(1-σ)*E_nn+λE_weight

wherein E is_aeRepresenting the reconstruction error from the encoder,

E_nnrepresenting the prediction error of the neural network,

representing regularization constraints for the weights; σ to control E_aeAnd E_nnThe balance between the weight of the two parts,λ is the regularization coefficient, taking the empirical value.

Step four: calculating the related weight and the gradient of the bias by adopting a back propagation algorithm;

step five: continuously training a semi-supervised neural network model according to a gradient descent method, calculating the optimal parameters of the model, and completing the modeling process of the semi-supervised neural network model;

step six: collecting new industrial process data, repeating the first step to the second step, substituting the processed industrial process data into the optimized semi-supervised neural network model to obtain the predicted value of the dominant variableThereby realizing the monitoring and control of the process.