CN118535441A - Disk fault prediction method, medium and device based on meta learning - Google Patents

Abstract

The present invention relates to the field of disk failure prediction, and in particular, to a disk failure prediction method, medium, and apparatus based on meta learning. Comprising the following steps: acquiring a feature matrix of a target disk; inputting the feature matrix into a target neural network model to generate a prediction result corresponding to a target disk; the target neural network model comprises an LSTM model which is obtained after training by a model independent element learning algorithm. The LSTM is guided to carry out gradient descent and parameter updating through model independent element learning, so that a new or changing task is quickly adapted, and the prediction problem of a small sample disk in disk data can be more quickly adapted. Meta-learning can use existing disk data to train a model that can quickly adapt to small amounts of new data, thereby reducing reliance on large-scale fault data. The heterogeneity among the disks of different models can be processed by utilizing the meta-learning framework, so that the prediction performance of the model on the model which is not seen is improved.

Description

A disk failure prediction method, medium and device based on meta-learning

Technical Field

The present invention relates to the field of disk failure prediction, and in particular to a disk failure prediction method, medium and device based on meta-learning.

Background Art

Disk failure prediction is a key link in computer hardware maintenance, and it has a direct impact on the operating efficiency and data security of data centers. Traditional prediction methods rely on analyzing large amounts of historical data and complex feature engineering to build prediction models, and these methods usually focus on failure prediction of homogeneous disks. However, in data centers, as the system continues to run, new models of disks continue to replace damaged disks, resulting in the presence of many heterogeneous disks from different manufacturers or models in data centers. This situation is particularly common in large data centers. In addition, as the storage system is continuously updated, some new models of disks will be added, which will also result in the number of some disk models being far less than other models. The disks generated in the above two situations are small sample disks, which usually refer to disks with a certain model number of about 1,500.

Therefore, in an actual data center environment, the failure prediction of a small sample of disks encounters the following two challenges. First, since multiple models of disks from different manufacturers are deployed, these disks have large differences in operating characteristics and failure modes, which makes it difficult for a single general model to be effectively applied to all models of disks. Second, for newly launched disk models on the market, it is usually difficult to accumulate enough failure data in a short period of time to support the training of traditional machine learning models, resulting in poor results in the initial stage of failure prediction of these new models of disks.

Summary of the invention

In view of the above technical problems, the technical solution adopted by the present invention is:

According to one aspect of the present invention, a disk failure prediction method based on meta-learning is provided, and the method comprises the following steps:

Obtain a feature matrix of the target disk, the feature matrix including the time series features and SMART attribute features of the disk;

Input the feature matrix into the target neural network model to generate a prediction result corresponding to the target disk; the target neural network model includes an LSTM model trained by a model-independent meta-learning algorithm;

The meta-task in meta-learning training is constructed as follows:

According to the SMART attribute values in the disk log, generate disk sample sets corresponding to different disk models; where D _m = {(x ^m ₁ ,y ^m ₁ ), (x ^m ₂ ,y ^m ₂ ), (x ^m ₃ ,y ^m ₃ ), …, (x ^m _n ,y ^m _n )}; D _m is the disk sample set corresponding to the mth disk model, x ^m _n is the feature matrix corresponding to the nth disk sample in D _m ; y ^m _n is the label corresponding to x _n in D _m ; n is the total number of disk samples in D _m ;

Obtain the support set and query set corresponding to the training tasks in multiple meta-tasks from _Dm ;

Obtain the support set and query set corresponding to the test task in multiple meta-tasks from _Ds ; _Ds is the disk sample set corresponding to the s-th disk model, and the disk models of _Dm and _Ds are different;

Use the support set and query set corresponding to the training task and test task in each meta-task to perform inner loop training on the LSTM model;

The support set and query set in the ith meta-task Task _i are obtained as follows:

Randomly select a faulty disk sample from the corresponding disk sample set, and calculate its KLD value with other faulty disk samples in the disk sample set;

Sort by KLD values from small to large to form a KLD sequence; there are 2K-1 KLD values in the KLD sequence; K is a positive integer;

The faulty disk samples corresponding to the first K-1 KLD values in the KLD sequence and a randomly selected faulty disk sample are used as the faulty sample set in the support set;

K healthy disk samples are randomly selected from the corresponding disk sample set as the healthy sample set in the support set.

According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned disk failure prediction method based on meta-learning is implemented.

According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned disk failure prediction method based on meta-learning when executing the computer program.

The present invention has at least the following beneficial effects:

Through the meta-learning method, a small number of disk samples can be used to quickly adapt to new models of disks. Specifically, Model-Agnostic Meta-Learning (MAML) is used to guide LSTM to perform gradient descent and parameter updates to quickly adapt to new or changing tasks, that is, it can more quickly adapt to drift problems in disk data or prediction problems of new models (small samples) of disks. Meta-learning can use existing disk data to train a model that can quickly adapt to a small amount of new data, thereby reducing dependence on large-scale fault data. The meta-learning framework can handle the heterogeneity between disks of different models, thereby improving the prediction performance of the model on unseen models.

At the same time, this scheme uses KLD (Kullback-Leibler Divergence, relative entropy) to select disk samples for each meta-task, and distributes samples to different tasks according to similarity through KLD. Specifically, for the support set, disk samples with smaller KLD values are selected as the support set, because these samples are closer to the benchmark samples in feature distribution and are easier for the model to learn. For the query set, samples with slightly larger KLD values are selected as the query set to test the model's adaptability to samples with slightly different feature distributions. This helps the model better understand the differences and similarities between different disk models, thereby improving the model's generalization and adaptability.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a disk failure prediction method based on meta-learning provided in an embodiment of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

As a possible embodiment of the present invention, as shown in FIG1 , a disk failure prediction method based on meta-learning is provided, and the method includes the following steps:

S100: Obtain a feature matrix of a target disk, where the feature matrix includes time series features and SMART attribute features of the disk.

Specifically, the feature matrix has the same form as that in S231, that is, to obtain the target SMART attributes in the recent 14-day historical records.

Specifically, each collected disk SMART sample contains more than 70 SMART attributes. However, many of these attributes have a large number of missing values, and some attributes have a low correlation with the disk failure prediction model because these attributes will not change significantly when the disk fails. Therefore, before modeling, only attributes that are closely related to the health of the disk are selected. At the same time, the data set used in this embodiment contains multiple disk models. Therefore, when selecting SMART attributes, it is necessary to select attributes that are common to different disk models. Finally, the selected target SMART attributes are shown in Table 1.

Table 1

The Chinese definitions corresponding to the attribute ID, attribute name, and attribute type in Table 1 above can be obtained based on the SMART attribute table in the prior art. Each SMART attribute contains a tuple consisting of five elements: ID, Normalized, RawThreshold, and Worst. Among them, ID represents the serial number of the SMART attribute; Normalized represents the value of the attribute after normalization, which is calculated from the original value by the disk manufacturer according to a specific algorithm; Raw represents the original value of the attribute statistics; Threshold represents the alarm threshold of the attribute; Worst represents the minimum or worst value recorded for the attribute. In the disk failure prediction embodiment, based on the disk data set provided by the data center, the three elements of ID, Normalized, and Raw are mainly concerned.

Since different SMART attributes have different value ranges, in order to ensure fair comparison between them, the SMART attribute values are standardized. The Min-Max standardization used in this embodiment is as follows:

Here, x is the attribute value corresponding to the SMART feature, and x _min and x _max are the minimum and maximum values of the attribute, respectively.

S200: Input the feature matrix into the target neural network model to generate the prediction result corresponding to the target disk. The target neural network model includes an LSTM model obtained after training with a model-independent meta-learning algorithm. LSTM (Long Short-Term Memory) is a special type of recurrent neural network (RNN) designed to solve the long-term dependency problem, that is, when processing time series and sequence data, it can remember long-term information without forgetting.

Specifically, the target neural network model also includes a dropout layer and a fully connected layer including a Sigmoid activation function, and the LSTM model includes four stacked LSTM layers. The LSTM model is connected to the dropout layer, and the dropout layer is connected to the fully connected layer including a Sigmoid activation function. The dropout layer is a regularization technology in deep learning networks, which is mainly used to reduce the overfitting problem of the model and improve the generalization ability of the model.

The last layer uses the Sigmoid function as the return value of the sigmoid function is in the range of 0 to 1, so this activation function is very suitable for binary classification prediction models. Unlike traditional neural networks, LSTM operates on time series vectors, so it can capture historical data on the health status of the disk, making it an ideal choice for disk failure prediction tasks.

In meta-learning, the model-independent meta-learning (MAML) algorithm is one of the effective methods to solve the small sample problem and is a meta-learning method based on optimization. MAML can provide a meta-learner to train the base learner, where the meta-learner is the main part for learning in MAML, and the base learner is a meta-learner trained on a data set and used for testing tasks. The key to MAML is to train a set of initialization parameters and quickly adapt to new tasks with a limited amount of data by applying one or more gradient update steps on top of the initial parameters. The model is meta-learned using N-way K-shot tasks (training tasks) to ensure that it obtains "prior knowledge" (initialization parameters). In addition, this "prior knowledge" can improve the performance of new N-way K-shot tasks. During MAML training, the optimization is divided into an inner loop and an outer loop: the inner loop is a training process for developing the basic ability to perform this task, while the outer loop is a meta-training process for enhancing cross-task generalization capabilities. N-way K-shot is a common experimental setting for meta-learning. In this embodiment, the settings are as follows: When constructing a classification task, N is fixed to 2, representing two categories: fault and health. For each category, K is set to 5, 10, and 15 per task. The entire learning process has two layers, the inner loop and the outer loop. The inner loop learning rate α and the outer loop learning rate β are 0.001 and 0.0001, respectively. The number of gradient updates in the inner loop is set to 10. The maximum number of iterations for training is set to 1000.

The goal of MAML is to find the optimal initial parameters θ that are suitable for most tasks. The initial parameters θ of the model are updated and improved through multiple iterations of training, so that the model can show good performance with very few learning steps when encountering new tasks. During the training process, the main process is as follows:

1. Inner loop. For each task Task _i , use the support set data of task Task _i to perform a finite number of steps of gradient descent update to minimize the loss function on task Task _i The task-specific parameters θ′ _i are obtained as shown in formula (1).

Among them, α is the inner loop learning rate, is the gradient of the loss function with respect to the parameter θ. At the same time, the performance is calculated on the corresponding query set using the updated parameters θ′ _i .

2. Outer loop. For all tasks, use the adapted parameters θ′ _i obtained from the inner loop to calculate the query set loss for each task The original parameters θ are updated using the losses of all tasks, as shown in formula (2).

Among them, β is the meta-learning rate and R is the number of tasks.

3. Repeat the inner and outer loops until the initial parameters of the model show good generalization ability in a multi-task learning environment.

In order to ensure that the model can not only handle the complexity of time series data in the disk dataset, but also quickly adapt to new or changing tasks, i.e., new models of disks, through the meta-learning method. In this embodiment, in order to obtain the timing information in the disk SMART data, LSTM is used to process the parsing of SMART time series data. At the same time, the rapid adaptability of the meta-learning method MAML is used to train the target neural network model.

The meta-task in meta-learning training is constructed as follows:

S201: Generate disk sample sets corresponding to different disk models according to the SMART attribute values in the disk log. Wherein, D _m ={(x ^m ₁ ,y ^m ₁ ),(x ^m ₂ ,y ^m ₂ ),(x ^m ₃ ,y ^m ₃ ),…,(x ^m _n ,y ^m _n )}. D _m is the disk sample set corresponding to the mth disk model, and x ^m _n is the feature matrix corresponding to the nth disk sample in D _m . y ^m _n is the label corresponding to x _n in D _m . n is the total number of disk samples in D _m .

Before constructing the disk failure prediction meta-task, it is necessary to extract the statistical features and time series features of the SMART attributes of each disk. In some previous embodiments, a single snapshot sample of the SMART attribute was used as training data for modeling, ignoring the rate of change of the SMART attribute value over time, which resulted in a low accuracy rate for disk failure prediction. At the same time, LSTM requires input in the form of 3D arrays or tensors. Therefore, in this embodiment, the time series features of the disk SMART attributes will be obtained.

Specifically, S201 includes:

S211: If the disk is healthy and has complete log records for a whole year, the log records are added to the healthy data pool corresponding to the disk model.

S221: Obtain monthly log records of some disks from the health data pool on average every month.

S231: According to the monthly log records, the target SMART attribute values corresponding to each disk for 14 consecutive days are obtained to generate an N×14 feature matrix corresponding to each disk, where N is the set of target SMART attribute values collected for each disk every day.

S241: If the disk is healthy and has only partial log records for a whole year, then the target SMART attribute values corresponding to the first 14 days in the log records are obtained.

S251: If the disk is a faulty disk, obtain the target SMART attribute value corresponding to the time period from T to T-13, where T is the date when the disk fails.

S261: Set the labels of the samples corresponding to the time period from T-13 to T-7 as healthy labels.

S271: Set the labels of the samples corresponding to the time period from T-6 to T as fault labels.

In order to obtain the time series features of the disk SMART attributes, according to the sequence length required by the input LSTM, a subset of each hard disk equal to the required sequence length is obtained, and SMART data for each disk is collected for 14 consecutive days according to the disk serial number. For a faulty disk, if a failure occurs on day T, SMART data for the continuous period from T to T-13 is collected; for healthy disks, according to the monthly distribution of healthy disks, the average number of healthy disks is collected each month, and SMART data for each healthy disk is collected for 14 consecutive days. At the same time, according to existing research, most SAMRT attributes will change significantly about 7 days before the disk failure is discovered. Therefore, for a faulty disk, the samples within 7 days before the disk failure are marked as faulty (that is, label y=1), and the other samples of the disk are marked as healthy (y=0); for a healthy disk, all continuously collected disk samples are marked as healthy (y=0). Finally, each disk obtains an N×14 feature matrix, where N is the set of SMART attribute values collected for each disk every day.

S202: Obtain support sets and query sets corresponding to training tasks in multiple meta-tasks from Dm.

S203: Obtain support sets and query sets corresponding to the test tasks in the multiple meta-tasks from _Ds . _Ds is a disk sample set corresponding to the s-th disk model, and the disk models of Dm and Ds are different.

In order to perform meta-training in this embodiment, it is first necessary to construct a training task and a test task respectively. The support set and query set in the training task and the support set and query set in the test task are both 2-way K-shot.

S204: Use the support set and query set corresponding to the training task and the test task in each meta-task to perform inner loop training on the LSTM model.

The support set and query set in the ith meta-task Taski are obtained as follows:

S214: randomly select a faulty disk sample from the corresponding disk sample set, and calculate the KLD value between the faulty disk sample and other faulty disk samples in the disk sample set. Specifically, the KLD of any two disk samples is obtained by a relative entropy algorithm.

KLD stands for Kullback-Leibler Divergence, also known as relative entropy, which is an indicator to measure the degree of match between two probability distributions. Generally speaking, the larger the KLD value, the greater the difference between the two distributions. Therefore, in this embodiment, KLD is used to select disk samples for each task, so as to help the model better understand the differences and similarities between different disk models, thereby improving the generalization ability and adaptability of the model.

Specifically, the probability distribution may be calculated for disks of the same model and disks of different models based on the SMART attribute values, and the KLD values between the different models may be calculated to measure the similarity of the distribution.

S224: Sort the KLD values from small to large to form a KLD sequence. There are 2K-1 KLD values in the KLD sequence. K is a positive integer.

S234: The faulty disk samples corresponding to the first K-1 KLD values in the KLD sequence and a randomly selected faulty disk sample are used as the faulty sample set in the support set.

S244: Use the faulty disk samples corresponding to the KLD values after the K-1th in the KLD sequence as the faulty sample set in the query set.

In existing optimization-based meta-learning, task samples are usually assigned by random selection without considering the order between tasks. This random task arrangement may introduce a high degree of randomness and may cause direct correlations between some tasks, while other tasks have almost no correlation. In order to effectively learn representative parameters and obtain better generalization performance when predicting failures of new models of disk models, it is necessary to arrange the learning order of tasks. At the same time, through the analysis of disk data sets, it is found that even for failed disks of the same model, there are certain differences in the distribution of SMART attributes. Therefore, in this embodiment, the disk failure prediction meta-task is constructed based on KLD, so that the model starts learning from similar tasks and gradually transitions to disk sample tasks with large distribution differences (i.e., larger KLD values). This progressive learning method helps the model gradually establish and strengthen its generalization ability, and also helps prevent early overfitting.

In this embodiment, when constructing the meta-task, the support set and query set of each task in the same model disk must first be defined. By selecting disk samples according to the size of the KLD value, the samples are assigned to different tasks according to similarity. Specifically, for the support set, disk samples with smaller KLD values are selected as the support set, because these samples are closer to the benchmark samples in feature distribution and are easier for the model to learn. For the query set, samples with slightly larger KLD values are selected as the query set to test the model's adaptability to samples with slightly different feature distributions.

S254: Randomly select K healthy disk samples from the corresponding disk sample set as the healthy sample set in the support set.

Correspondingly, in this embodiment, we can divide the disk data into multiple tasks according to different disk models, where each task includes a support set Support Set and a query set Query Set. Secondly, LSTM is used as a base learner and the parameters are continuously updated. During training, the support set disk data of task Task _i is used as the input of LSTM and the parameters θ are randomly initialized. Multiple gradient descent updates are performed to calculate its loss Update the parameters to θ′ _i according to formula (1); use the updated parameters θ′ _i to evaluate the query set of each task, calculate the total loss of all tasks on the query set, and update θ according to formula (2). Finally, during testing, use the support set consisting of a small amount of disk data from other models to optimize the model parameters and make predictions on the disk data in the query set.

The following is experimental data for evaluating the prediction performance of the meta-learning-based disk failure prediction method (hereinafter referred to as MLDF) provided in this embodiment.

1. Failure prediction of two types of heterogeneous small sample disks

We use ST12000NM0007 as the training task and ST14000NM0138 as the test task, ensuring that the data in the training set and the test set come from different disk models for the experiment, where K=15, and the experimental results are shown in Table 2. From the experimental data, we can see that in the prediction of heterogeneous small sample disk failures across disk models, MLDF can still maintain good prediction performance and quickly adapt to new model disks.

Table 2

2. Failure prediction of multiple heterogeneous small sample disks

We use ST12000NM0007 and ST10000NM0086 as training tasks, and ST14000NM0138 as test tasks, ensuring that the data in the training set and the test set come from different disk models for experiments, where K = 15, and the experimental results are shown in Table 3. It can be seen from the experimental data that for multi-model disk data sets, we can also have good prediction performance.

Table 3

In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and/or one step may be decomposed into multiple steps, etc.

Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation methods of the present disclosure.

In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

The electronic device according to this embodiment of the present invention is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

The storage stores program codes, which can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification.

The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory, and may further include read only memory (ROM).

The storage may also include a program/utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and/or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input/output (I/O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and/or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A disk failure prediction method based on meta-learning, characterized in that the method comprises the following steps: Obtaining a feature matrix of a target disk, wherein the feature matrix includes time series features and SMART attribute features of the disk; Inputting the feature matrix into a target neural network model to generate a prediction result corresponding to the target disk; the target neural network model includes an LSTM model obtained after training with a model-independent meta-learning algorithm; The meta-task in meta-learning training is constructed as follows: According to the SMART attribute values in the disk log, generate disk sample sets corresponding to different disk models; where D _m = {(x ^m ₁ ,y ^m ₁ ), (x ^m ₂ ,y ^m ₂ ), (x ^m ₃ ,y ^m ₃ ), …, (x ^m _n ,y ^m _n )}; D _m is the disk sample set corresponding to the mth disk model, x ^m _n is the feature matrix corresponding to the nth disk sample in D _m ; y ^m _n is the label corresponding to x _n in D _m ; n is the total number of disk samples in D _m ; Obtain the support set and query set corresponding to the training tasks in multiple meta-tasks from _Dm ; Obtain the support set and query set corresponding to the test task in multiple meta-tasks from _Ds ; _Ds is the disk sample set corresponding to the s-th disk model, and the disk models of _Dm and _Ds are different; Use the support set and query set corresponding to the training task and test task in each meta-task to perform inner loop training on the LSTM model; The support set and query set in the ith meta-task Task _i are obtained as follows: Randomly select a faulty disk sample from the corresponding disk sample set, and calculate its KLD value with other faulty disk samples in the disk sample set; Sort by KLD values from small to large to form a KLD sequence; there are 2K-1 KLD values in the KLD sequence; K is a positive integer; The faulty disk samples corresponding to the first K-1 KLD values in the KLD sequence and a randomly selected faulty disk sample are used as the faulty sample set in the support set; K healthy disk samples are randomly selected from the corresponding disk sample set as the healthy sample set in the support set. 2. The method according to claim 1, characterized in that after sorting the KLD values from small to large to form a KLD sequence, the method further comprises: The faulty disk samples corresponding to the KLD values after the K-1th in the KLD sequence are taken as the faulty sample set in the query set. 3. The method according to claim 1 is characterized in that the KLD of any two disk samples is obtained by a relative entropy algorithm. 4. The method according to claim 1 is characterized in that the target neural network model also includes a dropout layer and a fully connected layer including a Sigmoid activation function, and the LSTM model includes four stacked LSTM layers; the LSTM model is connected to the dropout layer, and the dropout layer is connected to the fully connected layer including the Sigmoid activation function. 5. The method according to claim 1, characterized in that generating disk sample sets corresponding to different disk models according to the SMART attribute values in the disk logs comprises: If the disk is healthy and has complete log records for a whole year, the log records are added to the healthy data pool corresponding to the disk model; Obtain monthly log records of a portion of disks from the health data pool on average every month; According to the monthly log records, the target SMART attribute values of each disk for 14 consecutive days are obtained to generate an N×14 feature matrix corresponding to each disk; where N is the set of target SMART attribute values collected for each disk every day. 6. The method according to claim 5, characterized in that generating disk sample sets corresponding to different disk models according to the SMART attribute values in the disk logs comprises: If the disk is healthy and has only partial log records for a whole year, the target SMART attribute values corresponding to the first 14 days in the log records are obtained. 7. The method according to claim 5, characterized in that generating disk sample sets corresponding to different disk models according to the SMART attribute values in the disk logs comprises: If the disk is a faulty disk, the target SMART attribute value corresponding to the time period from T to T-13 is obtained; T is the date when the disk fails. 8. The method according to claim 7 is characterized in that the labels of the samples corresponding to the time period from T-13 to T-7 are set as healthy labels; and the labels of the samples corresponding to the time period from T-6 to T are set as fault labels. 9. A non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the meta-learning-based disk failure prediction method according to any one of claims 1 to 8 is implemented. 10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a meta-learning-based disk failure prediction method as described in any one of claims 1 to 8 is implemented.