WO2022160902A1 - Anomaly detection method for large-scale multivariate time series data in cloud environment - Google Patents

Abstract

An anomaly detection method for large-scale multivariate time series data in a cloud environment. The method comprises: establishing an anomaly detection model for multivariate time series data by means of offline training, and performing anomaly detection on online monitored data by means of the offline-trained anomaly detection model. According to the method, the feedforward neural network of a native variational autoencoder is improved in the stage of offline model training to construct the dependency of multivariate time series; a loss function calculation method is improved, so that during model training, the data in a normal mode can be paid attention to and the data in an abnormal mode can be ignored, so that when an anomaly occurs during online anomaly detection, the probability of model reconstruction is low, and it is easier to detect the anomaly.

Description

Anomaly detection method for large-scale multivariate time series data in cloud environment technical field

The invention belongs to the field of computer technology, and more particularly, relates to a large-scale multivariate time series data anomaly detection method in a cloud environment.

Background technique

With the development of cloud computing technology, virtualization technology and container technology, more and more enterprises build container cloud environments and apply them to actual production. In a complex and changeable cloud environment, in order to ensure that various applications and services deployed in the cloud are online 24/7, operation and maintenance engineers need to simultaneously monitor multiple indicators time series of entities (cluster machines, containers, applications, etc.) (such as CPU utilization, memory utilization, number of online users, request response delay, etc.) in order to detect abnormalities in time and locate the causes of abnormalities to ensure service quality and reliability.

In recent years, many studies have used deep learning and other algorithms for anomaly detection of time series, but most of them are index-level anomaly detection, that is, for different indicators, it is necessary to re-select an algorithm to train an anomaly detection model, but the monitoring in the cloud environment There are many types of entity multi-indicators. If anomaly detection is performed on each index, it will consume a lot of manpower and time, and it is impossible to detect anomalies in time and quickly locate the cause of anomalies. However, a small number of existing researches on the entity level (all indicator sequences of monitoring entities such as applications, servers, containers, etc. are used for abnormal judgment together, that is, multivariate time series anomaly detection), either need a large amount of label data; Consistent assumptions; or it is difficult to capture the time-dependent, high-dimensional and random characteristics of the index sequence, and it is difficult to meet the anomaly detection of large-scale time series in the cloud environment.

SUMMARY OF THE INVENTION

In view of the above defects or improvement needs of the prior art, the present invention provides a large-scale multivariate time series data anomaly detection method in a cloud environment, anomaly detection based on a semi-supervised variational autoencoder based on a long short-term memory network, the purpose of which is It is to realize anomaly detection of multivariate time series. Aiming at the problem that multivariate time series labels are difficult to obtain, LSTM is introduced to improve the feedforward neural network of native VAE, and an improved loss function is proposed to improve the abnormal detection algorithm of VAE and its training variant, so that the input data of training can be improved. It can include abnormal data, and focus on normal mode data during training to improve the accuracy of anomaly detection.

To achieve the above object, the present invention provides a large-scale multivariate time series data anomaly detection method in a cloud environment, comprising the following steps:

(1) Offline module training multivariate time series anomaly detection model: take a small part of the labeled data and most of the unlabeled data collected by the detection system as the data set for offline training, and preprocess the data, and the preprocessed data is used for Train the multivariate time series anomaly detection model; in the model training, first learn the dependencies of the multivariate time series through the long short-term memory network (LSTM: Long Short-Term Memory), and then pass the input multivariate time series through the variational autoencoder ( VAE: Variational Auto-Encoder) to reduce the dimension and map it to the random variable Z space and obtain hidden variables, and then splicing the data labels obtained from the classifier with the random variable z extracted from the prior distribution of the random variable Z space, and finally splicing The obtained data reconstructs the input sequence after the decoder; among them, the parameter training goal of the multivariate time series anomaly detection model is to maximize the improved loss function, and stop training when it converges;

而非窗口本身，则可以使用概率来表示异常分数；在线模块对在线检测采集的数据进行预处理，预处理后的数据经与离线模块相同的变分自编码器处理，然后使用多元时间序列异常检测模型得到的参数计算随机变量Z空间的先验对角高斯分布的参数，将从分类器获取的数据标签与从随机变量Z空间先验分布中抽取的随机变量z进行拼接，最后将拼接得到的数据用于重建；在线模块计算所有点的重建概率分 数，并根据窗口内最后一点的概率分数与阈值进行判断实体状态。 (2) The online module calculates the reconstruction probability score to determine the entity status: the online monitoring data can be used to determine whether the input monitoring value x ^(t) at time t is normal by calculating the reconstruction probability through the offline training model. The multivariate sub-time series x ^{( t-w+1:t)} as the input data to reconstruct x ^(t) , since it is reconstructed on the distribution parameters μ, σ and π of x ^(t-w+1:t)

Instead of the window itself, the probability can be used to represent the anomaly score; the online module preprocesses the data collected by the online detection, the preprocessed data is processed by the same variational autoencoder as the offline module, and then the multivariate time series anomaly is used. The parameters obtained by the detection model are used to calculate the parameters of the prior diagonal Gaussian distribution of the random variable Z space. The data is used for reconstruction; the online module calculates the reconstruction probability scores of all points, and judges the entity state according to the probability scores and thresholds of the last point in the window.

Compared with the prior art, in the large-scale cloud environment, the present invention improves the native VAE for the difficulty in obtaining multivariate time series labels, uses LSTM to replace the feedforward neural network of the native VAE, and uses the gating mechanism of LSTM to improve the native VAE. The assumption that the data of VAE is independent in time, the reconstruction value can only depend on the current input, and it is not suitable for time series data, etc., uses LSTM to capture multivariate time series dependencies; Normal sequence fragments are used to train the anomaly detection model. However, due to the problem that the randomness of anomalies is difficult to apply in practice, a new loss function calculation method is proposed, so that the training data can contain abnormal data, but pay attention to the model training process. Normal mode, ignoring abnormal mode, in order to learn the complex distribution of multivariate time series, so as to achieve good reconstruction effect and improve the accuracy of abnormal detection.

Description of drawings

1 is a schematic diagram of a model of a method for detecting anomalies in large-scale multivariate time series data in a cloud-oriented environment according to an embodiment of the present invention;

2 is an overall framework diagram of a method for detecting anomalies in large-scale multivariate time series data in a cloud-oriented environment according to an embodiment of the present invention;

FIG. 3 is a network structure diagram of a large-scale multivariate time series data anomaly detection method in a cloud environment according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

Anomaly detection is a common technology in the field of intelligent operation and maintenance. Due to the complex and changeable cloud environment, the occurrence of anomalies is often unpredictable. The automatic operation and maintenance mode based on traditional manual operation and maintenance or based on automatically triggered scripts with predefined rules to perform common and repetitive tasks can no longer be satisfied. timeliness requirements. With the development of artificial intelligence and machine learning technology, the intelligent operation and maintenance model emerges as the times require, and its goal is based on the existing operation and maintenance data (logs, monitoring indicators, application information, abnormal events, and manual processing logs of operation and maintenance engineers, etc.) , to further solve the problems that cannot be solved by automatic operation and maintenance through machine learning. They do not rely on artificially specified rules. They advocate that machine learning algorithms automatically learn from massive operation and maintenance data and continuously refine and summarize rules. In this way, the abnormality can be detected quickly and accurately, and the cost loss caused by the abnormality can be reduced.

1 is a schematic diagram of a model of a method for detecting anomalies in large-scale multivariate time series data in a cloud environment in an embodiment of the present invention; FIG. 2 is a method for detecting anomalies in large-scale multivariate time series data in a cloud environment in an embodiment of the present invention Overall frame diagram.

In order to achieve the abnormal detection of multivariate time series at the entity level, as shown in Figure 1 and Figure 2, the present invention provides a large-scale multivariate time series data abnormality detection method in a cloud environment, including:

(1) Offline module training multivariate time series anomaly detection model: take a small part of the labeled data and most of the unlabeled data collected by the detection system as the data set for offline training, and preprocess the data, and the preprocessed data is used for Train the multivariate time series anomaly detection model; in the model training, first learn the dependencies of the multivariate time series through the long short-term memory network (LSTM: Long Short-Term Memory), and then pass the input multivariate time series through the variational autoencoder ( VAE: Variational Auto-Encoder) to reduce the dimension and map to the random variable Z space and obtain hidden variables, and then obtain the data label through the classifier and splicing the random variable z extracted from the prior distribution of the random variable Z space, and finally spliced to get The input sequence is reconstructed after the data is passed through the decoder; among them, the parameter training goal of the multivariate time series anomaly detection model is to maximize the improved loss function, and stop training when it converges;

由于是在x ^(t-w+1:t)的分布参数μ、σ和π上重建出

而非窗口本身，则可以使用概率来表示 异常分数，其中t表示多元时间序列的监控时间；在线模块对在线检测采集的数据进行预处理，预处理后的数据经与离线模块相同的变分自编码器处理，然后使用多元时间序列异常检测模型得到的参数计算随机变量Z空间的先验对角高斯分布的参数，将从分类器获取的数据标签与从随机变量Z空间先验分布中抽取的随机变量z进行拼接，最后将拼接得到的数据用于重建；在线模块计算所有点的重概率分数，并根据窗口内最后一点的概率分数与阈值进行判断实体状态。 (2) The online module calculates the reconstruction probability score to determine the entity status: the online monitoring data can be used to determine whether the input monitoring value x ^(t) at time t is normal by calculating the reconstruction probability through the offline training model. The multivariate sub-time series x ^{( t-w+1:t)} as input data to reconstruct

Since it is reconstructed on the distribution parameters μ, σ and π of x ^(t-w+1:t)

Instead of the window itself, the probability can be used to represent the abnormal score, where t represents the monitoring time of the multivariate time series; the online module preprocesses the data collected by the online detection, and the preprocessed data is processed by the same variational automatic as the offline module. The encoder processes, and then uses the parameters obtained from the multivariate time series anomaly detection model to calculate the parameters of the prior diagonal Gaussian distribution of the random variable Z space. The random variable z is spliced, and finally the spliced data is used for reconstruction; the online module calculates the multiple probability scores of all points, and judges the entity state according to the probability scores and thresholds of the last point in the window.

FIG. 3 is a network structure diagram of a large-scale multivariate time series data anomaly detection method in a cloud environment according to an embodiment of the present invention. For step (1), the offline module training multivariate time series anomaly detection model specifically includes:

(1.1) Preprocessing of training data: First, the training data shown in formula (8) is processed

Z-Normalization is standardized so that each indicator conforms to the standard normal distribution; x is divided into subsequences with a sliding window, the moving step is 1 unit, and the optimal window length of the sequence is selected, and the label of the defined window is defined by It is determined whether there is abnormal data in this window; among them, N is the monitoring time of the multivariate time series x, and x ^(t) ∈ R ^M as shown in formula (9) is an M-dimensional vector, which represents the monitoring index value at time t, M represents the number of indicators monitored;

长度为w的子时间序列和相应的窗口标签，输入的训练数据可以包含异常数据； Among them, y _l represents the data label, 0 represents normal, 1 represents abnormal, and if it is NULL, it represents unlabeled data; after preprocessing, the result is as follows

A sub-time series of length w and the corresponding window label, the input training data can contain abnormal data;

来获取t时刻更深层次的表达，即先通过计算当前时刻的候选状态

(1.2) Encoding dimension reduction and obtaining hidden variables: The multivariate sub-time series dimension reduction is mapped to the random variable Z space through the encoder, and the distribution p _θ (Z) on the Z space is constrained to a multivariate normal distribution N(0,I ), the prior distribution q _φ (Z|X) of the random variable Z space is a diagonal Gaussian distribution N(μ,σ ² I); in the encoding process, the gating mechanism of LSTM is used to improve the feedforward neural network of the native VAE, The preprocessed data is encoded by LSTM-Encoder. When the monitoring value x ^(t) at time t is input, LSTM is used to combine the hidden state at time t-1.

To obtain a deeper expression at time t, that is, first calculate the candidate state at the current time

控制当前时刻候选状态

有需要保存的信息： Then, through the input gate

Control the candidate state at the current moment

There is information to save:

通过式(12)控制上一个时刻的内部状态

需要遗忘的信息： Forgotten Gate

Control the internal state of the previous moment by formula (12)

Information to be forgotten:

控制当前时刻的内部状态

需要输出给外部的信息，即： output gate

Control the internal state of the current moment

Information that needs to be output to the outside, namely:

可以由式(14)计算得到： Hidden state after LSTM encoding

It can be calculated by formula (14):

其中

是需要学习的参数，*∈{i,f,o},·是元素积； Then, batch normalization (Batch Norm: Batch Normalization) is used to process the hidden state obtained by LSTM encoding, so that the training data and the hidden state obtained by online data encoding have the same distribution, ensuring that the model obtained from the training data can be used for online data. And make the distribution more uniform, increase the model convergence speed, and then use the tanh activation function such as formula (15) to nonlinearly transform the features of each data point to obtain the final encoding state

in

is the parameter to be learned, *∈{i,f,o},·is the element product;

Introduce the gating mechanism of LSTM in the coding stage to improve the feedforward neural network of the native VAE, assuming that the data is independent in time, the reconstruction value can only depend on the current input, it is not suitable for time series data, and the gradient explosion or gradient disappears. Use LSTM instead The feedforward neural network of the native VAE captures the multivariate time series dependencies; the input multivariate sub-time series is dimensionally mapped to the random variable Z space, and the distribution p _θ (z) on the random variable Z space is constrained to a multivariate normal distribution N (0,I), where the prior distribution q _φ (z|x) of the random variable Z space is a diagonal Gaussian distribution N(μ,σ ² I);

如果输入数据x带有标签，则不需要通过分类网络进行训练，直接将标签数据y _l与随机变量z进行拼接，即(z,y _l)，如果输入数据x是无标签的，则要经过分类网络对标签进行预测，然后将预测的标签

与随机变量z进行拼接得到

之后再用于解码器进行重建；在分类过程中，将y视为一个未知变量，而q _φ(y|x)可近似为Cat(π _φ(x)),即q _φ(y|x)＝Cat(π _φ(x))，服从级联多项分布，而π _φ(x)的计算由带参数

的神经网络定义； (1.3) Training the classifier: Semi-supervised learning is used for training, and a small amount of labeled data is used to drive most of the unlabeled data to train the classifier; in order to improve the accuracy of reconstruction, labeled data is introduced during decoding, and a named The classification network Classifier of q _φ (y|x) consists of a linear layer, a tanh activation layer, and a Softmax layer. The output is a probability vector, that is, the predicted label.

If the input data x has a label, it does not need to be trained by the classification network, and the label data y _l is directly spliced with the random variable z, that is (z, y _l ). If the input data x is unlabeled, it needs to go through The classification network predicts the label, and then converts the predicted label

Concatenated with random variable z to get

It is then used for reconstruction in the decoder; in the classification process, y is regarded as an unknown variable, and q _φ (y|x) can be approximated as Cat(π _φ (x)), that is, q _φ (y|x) =Cat(π _φ (x)), subject to a cascaded multinomial distribution, and π _φ (x) is calculated by the parameter

The definition of neural network;

Because a small number of labeled data is used to drive most of the unlabeled data to train the classifier, two situations are considered to optimize the training objective function when training the classifier, namely, optimizing the training evidence lower bound (ELBO: Evidence Lower Bound) loss function. The first case is for labeled data, and the improved ELBO is shown in formula (16):

表示x中正常点的比例，当接触到异常点时，p _θ(x ^(t)|y,z)的作用可以用a _t直接排除，p _θ(z)和p _θ(y)的贡献可k的乘积计算得到，而q _φ(z|x,y)仅是(x,y)到z的映射，不用考虑是否为正常数据点，因此不用进行修改； Among them, a _t =0,t∈{1,2,...,w} indicates that x ^(t) is abnormal at time t, otherwise a _t =1,

Represents the proportion of normal points in x. When encountering abnormal points, the role of p _θ (x ^(t) |y,z) can be directly excluded by a _t , and the contributions of p _θ (z) and p _θ (y) can be The product of k is calculated, and q _φ (z|x, y) is only the mapping from (x, y) to z, regardless of whether it is a normal data point, so there is no need to modify it;

The second case is that for unlabeled input data, the above method of reducing the interference caused by abnormal points is still available, and the lower bound of the evidence for unlabeled data can be expressed by formula (17):

Then the ELBO that can satisfy the above two conditions at the same time can be expressed as:

相关，为了让分类器能够在有标签情况下学习，则在目标函数中添加一个分类损失，扩展的ELBO如下： In ELBO at this time, the label prediction distribution q _φ (y|x) is only similar to the unlabeled

Related, in order to allow the classifier to learn with labels, a classification loss is added to the objective function, and the extended ELBO is as follows:

Among them, the hyperparameter λ is used to balance the use of direct label data and predicted label data. Using this objective function, labeled and unlabeled data can be correctly evaluated. Finally, gradient descent is used to update the encoding network and decoding network. parameter;

输入LSTM-Decoder进行解码，得到隐藏状态

最后，经过线性层处理可以将隐藏状态转为输入状态，得到重建的

其中先验对角高斯分布的参数μ和logσ的计算公式如式(20)所示： (1.4) Decoding and reconstructing the input sequence: In the stage of decoding and reconstructing the input sequence using LSTM-Decoder, the random variable z extracted from the prior diagonal Gaussian distribution q _φ (z|x) needs to be spliced with the label or predicted label; Then stitch the obtained (z, y _l ) or

Input LSTM-Decoder to decode to get the hidden state

Finally, after linear layer processing, the hidden state can be converted to the input state, and the reconstructed

The calculation formulas of the parameters μ and logσ of the prior diagonal Gaussian distribution are shown in formula (20):

The training goal of step (1) is to maximize the improved loss function, stop training when it converges, and then save the trained model, that is, save the classifier parameters, encoding grid parameters, and decoding grid parameters in the training model.

For the data collected by the online module monitoring in step (2), using the anomaly detection model trained by the offline module to detect entities specifically includes:

而非窗口本身，则可以使用概率来表示异常分数。在线模块对在线检测采集的数据进行预处理，预处理后 的数据经与离线模块相同的编码器处理，然后使用异常检测模型得到的参数如式(20)计算随机变量Z空间先验对角高斯分布参数，最后经分类器获取的数据标签与从先验对角高斯分布随机抽取的随机变量z拼接的数据用于解码重建。在线模块计算所有点的重概率分数，并根据窗口内最后一点的概率分数与阈值进行判断实体状态。 (2.1) For the online monitoring data, by calculating the reconstruction probability score through the offline training anomaly detection model, it is possible to judge whether the monitoring value at a certain time (such as x ^(t) at time t) is normal, using a multivariate sub-time series of length w as the Input data, that is, input x ^(t-w+1:t) to reconstruct x ^(t) , since it is reconstructed on the distribution parameters μ, σ and π of x ^(t-w+1:t)

Instead of the window itself, probabilities can be used to represent anomaly scores. The online module preprocesses the data collected by the online detection. The preprocessed data is processed by the same encoder as the offline module, and then the parameters obtained by the anomaly detection model are used to calculate the random variable Z space prior diagonal Gaussian. Distribution parameters, and finally the data concatenated by the data labels obtained by the classifier and the random variable z randomly drawn from the prior diagonal Gaussian distribution is used for decoding and reconstruction. The online module calculates the multiple probability scores of all points, and judges the entity state according to the probability scores and thresholds of the last point in the window.

作为异常检测器，然后使用蒙特卡洛方法近似求解，如下式： (2.2) Judging the entity state by the reconstruction probability score: using the reconstruction probability

As an anomaly detector, it is then approximately solved using the Monte Carlo method as follows:

其中f(x)＝1/(1+e ^-x)。若r ^(t)越高，表示重建的效果越好，x ^(t)越有可能判定为正常；最后，当模型计算出检测序列的重建概率分数后，根据设定的阈值来决定实体的状态，如式(22)所示： Since the reconstruction probability is a negative number, Sigmoid is used to transform it into the range of [0,1], then the reconstruction score r ^(t) at time t can be expressed as

where f(x)=1/(1+e ^-x ). If r ^{(t) is} higher, it means that the reconstruction effect is better, and x ^(t) is more likely to be judged to be normal; finally, after the model calculates the reconstruction probability score of the detection sequence, the state of the entity is determined according to the set threshold , as shown in formula (22):

If r ^{(t) is} higher than the set threshold, it is judged as normal, represented by 0, otherwise it is abnormal, represented by 1.

Those skilled in the art can easily understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (4)

A method for anomaly detection of large-scale multivariate time series data in a cloud environment, characterized in that it includes the following steps: (1) Offline module training multivariate time series anomaly detection model: take a small part of the labeled data and most of the unlabeled data collected by the detection system as the data set for offline training, and preprocess the data, and the preprocessed data is used for Train the multivariate time series anomaly detection model; in the model training, first learn the dependencies of the multivariate time series through the long short-term memory network (LSTM: Long Short-Term Memory), and then pass the input multivariate time series through the variational autoencoder ( VAE: Variational Auto-Encoder) to reduce the dimension and map to the random variable Z space and obtain hidden variables, and then obtain the data label through the classifier and splicing the random variable z extracted from the prior distribution of the random variable Z space, and finally spliced to get The input sequence is reconstructed after the data is passed through the decoder; among them, the parameter training goal of the multivariate time series anomaly detection model is to maximize the improved loss function, and stop training when it converges; (2) The online module calculates the reconstruction probability score to determine the entity status: the online monitoring data can be used to determine whether the input monitoring value x ^(t) at time t is normal by calculating the reconstruction probability through the offline training model. The multivariate sub-time series x ^{( t-w+1:t)} as input data to reconstruct

Since it is reconstructed on the distribution parameters μ, σ and π of x ^(t-w+1:t)

Instead of the window itself, the probability can be used to represent the abnormal score, where t represents the monitoring time of the multivariate time series; the online module preprocesses the data collected by the online detection, and the preprocessed data is processed by the same variational automatic as the offline module. The encoder processes, and then uses the parameters obtained from the multivariate time series anomaly detection model to calculate the parameters of the prior diagonal Gaussian distribution of the random variable Z space. The random variable z is spliced, and finally the spliced data is used for reconstruction; the online module calculates the multiple probability scores of all points, and judges the entity state according to the probability scores and thresholds of the last point in the window. The large-scale multivariate time series data anomaly detection method in a cloud-oriented environment as claimed in claim 1, wherein the step (1) specifically comprises: (1.1) Data preprocessing: First, the training data shown in formula (1) is processed

Z-Normalization is standardized so that each indicator conforms to the standard normal distribution; x is divided into subsequences with a sliding window, the moving step is 1 unit, and the optimal window length of the sequence is selected, and the label of the defined window is defined by Whether there is abnormal data in this window is determined; among them, N is the monitoring time of the multivariate time series x and the data label y _l ; x ^(t) ∈ R ^M is an M-dimensional vector whose value range is R, and M represents the monitoring Number of indicators; y _l represents the label of the data, 0 means normal, 1 means abnormal, if it is NULL, it means unlabeled data; after preprocessing, the form is as follows

A sub-time series of length w and the corresponding window label, the input training data can contain abnormal data; (1.2) Encoding dimensionality reduction and obtaining hidden state: The multivariate sub-time series dimensionality reduction is mapped to the random variable Z space through the encoder, and the distribution p _θ (Z) on the Z space is constrained to a multivariate normal distribution N(0,I ), the prior distribution q _φ (Z|X) of the random variable Z space is a diagonal Gaussian distribution N(μ,σ ² I); in the encoding process, the gating mechanism of LSTM is used to improve the feedforward neural network of the native VAE, The preprocessed data is encoded by LSTM-Encoder. When the monitoring value x ^(t) at time t is input, LSTM is used in combination with the hidden state at time t-1 to obtain a deeper expression at time t; then, batch normalization ( Batch Norm: Batch Normalization) processes the hidden state obtained by LSTM encoding, so that the training data and the hidden state obtained by online data encoding have the same distribution, ensuring that the model obtained from the training data can be used for online data, and make its distribution more uniform , increase the convergence speed of the model, and then use the tanh activation function such as formula (2) to perform nonlinear transformation on the features of each data point to obtain the final encoding state;

(1.3) Training the classifier: In order to improve the accuracy of reconstruction, label data is introduced during decoding, and a classification network Classifier named q _φ (y|x) is designed, which consists of a linear layer, a tanh activation layer, and a Softmax layer. The output is a probability vector, the predicted data labels

If the input data has a label, it does not need to be trained by the classification network, and the label y _l is directly spliced with the extracted random variable z, that is (z, y _l ). If the input data x is unlabeled, it needs to go through The classification network predicts the label, and then converts the predicted label

It is obtained by splicing with the extracted random variable z

It is then used for reconstruction in the decoder; in the classification process, y is regarded as an unknown variable, and q _φ (y|x) can be approximated as Cat(π _φ (x)), that is, q _φ (y|x) =Cat(π _φ (x)), subject to a cascaded multinomial distribution, and π _φ (x) is calculated by the parameter

The definition of neural network; Because a small number of labeled data is used to drive most of the unlabeled data to train the classifier, two situations are considered to optimize the training objective function when training the classifier, namely, optimizing the training evidence lower bound (ELBO: Evidence Lower Bound) loss function; (1.4) Decoding and reconstructing the input sequence: the obtained (z, y _l ) or

Input LSTM-Decoder to decode to get the hidden state

After linear layer processing, the hidden state can be converted into the input state, and the reconstructed

Then update the parameters. The anomaly detection method for large-scale multivariate time series data in a cloud-oriented environment according to claim 2, characterized in that, in the step (1.3): The first case is for labeled data, and the improved ELBO is shown in formula (3):

Among them, at = 0, t∈{1,2,...,w} indicates that the monitoring value x ^(t) _is abnormal at time _t , otherwise at =1,

represents the proportion of normal points in x; the contributions of p _θ (z) and p _θ (y) can be calculated as a product of k, while q _φ (z|x,y) is just a mapping of (x,y) to z ; In the second case, for unlabeled input data, the evidence lower bound of unlabeled data can be expressed by formula (4):

At this time, the method of reducing the interference caused by abnormal points is still available, and the ELBO that can satisfy the above two conditions at the same time can be expressed as:

In ELBO at this time, the label prediction distribution q _φ (y|x) is only similar to the unlabeled

Related, in order to allow the classifier to learn with labels, a classification loss is added to the objective function, and the extended ELBO is shown in formula (6):

The hyperparameter λ is used to balance the use of direct labeled data and predicted labeled data. Using this objective function, labeled and unlabeled data can be correctly evaluated. Finally, gradient descent is used to update the encoding network and decoding network. parameter. The method for anomaly detection of large-scale multivariate time series data in a cloud-oriented environment according to claim 1 or 2, wherein in the step (2), an anomaly detection model trained by an offline module is used for the data collected by the online module monitoring. Entities are detected, including: (2.1) Calculate the reconstruction probability: first read the data online; then, the read data is subjected to the same preprocessing as the offline module, and the hidden state is obtained through the encoder for the multivariate time series of each sliding window; then, the random variable is calculated The parameters of the prior diagonal Gaussian distribution in Z space, the random variable z drawn from the prior diagonal Gaussian distribution; finally, the data spliced with the random variable z and the predicted label are used for reconstruction

and through the reconstructed

Calculate the reconstruction probability; (2.2) Judging the entity state by the reconstruction probability score: using the reconstruction probability

As an anomaly detector,

Since the reconstruction probability is a negative number, Sigmoid is used to transform it into the range of [0,1], then the reconstruction score r ^(t) at time t can be expressed as

where f(x)=1/(1+e ^-x ); if r ^{(t) is} higher, it means that the reconstruction effect is better, and x ^(t) is more likely to be judged to be normal, and the entity is determined according to the set threshold status.

Images