CN118070107A - Deep learning-oriented network anomaly detection method, device, storage medium and equipment - Google Patents

Abstract

The invention relates to the technical field of anomaly detection, and discloses a network anomaly detection method, device, storage medium and equipment for deep learning, which comprise the following steps: obtaining abnormal flow data to be detected, inputting the abnormal flow data to a pre-trained network abnormal detection model for classification detection, and obtaining a detection result; the training process of the trained network anomaly detection model comprises the following steps: preprocessing the acquired historical network flow data set, and then screening the characteristics of the data to solve the problem of data redundancy; performing balance treatment on the data subjected to the feature screening; constructing a mixed network of a deformable convolutional neural network DCNN and an improved transducer model for realizing network traffic classification; finishing the definition of a network training loss function; and training the hybrid network by using the training set on the basis of course learning to obtain a network anomaly detection model. The beneficial effects of the invention are as follows: the accuracy and the robustness of network anomaly detection are remarkably improved.

Description

A network anomaly detection method, device, storage medium and equipment for deep learning

Technical Field

The present invention relates to a network anomaly detection method, device, storage medium and equipment for deep learning, belonging to the technical field of network anomaly detection.

Background technique

The data generated during the operation of the network is a direct reflection of the network status. By extracting and analyzing the network data features, the network status can be detected in real time, thereby avoiding the losses caused by abnormal network behavior. However, as network technology is changing with each passing day, new network products emerge in an endless stream, network traffic data is growing exponentially, and the data composition is gradually becoming complex and diverse. How to extract effective features from huge data has attracted widespread attention from researchers. Network Anomaly Detection (NAD) is a key technology to improve the security of network systems. Different from the previous passive defense strategy, NAD adopts an active defense strategy. Through real-time monitoring of network behavior, it obtains and analyzes the data generated by the network system, deeply mines the hidden correlation information between network data features, and once data related to abnormal behavior is detected, it will be fed back to the network administrator, and corresponding strategies will be taken to deal with abnormal network behavior to minimize the harm caused by abnormal behavior.

At present, abnormal traffic detection technology has made great progress, but there are still some problems:

1) Feature redundancy problem: The more feature dimensions there are, the longer the model training time will be, the lower the model detection effect will be, and the generalization of anomaly detection will be affected. Although principal component analysis (PCA) can achieve a good effect on data dimensionality reduction, PCA is a linear dimensionality reduction method and has limited processing capabilities for nonlinear data.

2) Data imbalance problem: When a certain type of attack is underrepresented in the data set, that is, the data between different attacks is unbalanced, the generated model performs poorly in detecting uncommon attack types, affecting the accuracy of neural network training and model detection. Existing oversampling methods, such as the commonly used synthetic minority oversampling technique (SMOTE), can solve the problem of data imbalance, but in some cases may cause overfitting or underfitting problems.

3) Incomplete feature learning: Using Convolutional Neural Networks (CNN) alone can extract spatial and local features of data, but it is difficult to capture long-term dependencies.

Summary of the invention

The purpose of the present invention is to provide a network anomaly detection method, device, storage medium and equipment for deep learning, which can simultaneously extract the temporal features and spatial features of data, solve the problem of incomplete feature extraction, and help the model better understand and utilize the temporal and spatial relationships in the data, thereby improving the performance and generalization ability of the model.

To solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

In a first aspect, the present invention provides a network anomaly detection method for deep learning, comprising:

Obtain the abnormal traffic data to be detected, input it into the pre-trained network anomaly detection model for classification detection, and obtain the detection results;

The training process of the trained network anomaly detection model includes:

The acquired historical network traffic data set is divided into a training set and a test set, and a part of the training set is divided into a validation set, and the data in the training set is subjected to feature screening;

Balance the number of samples in each category for the data after feature screening;

A hybrid network model of a deformable convolutional neural network DCNN and an improved Transformer model is constructed to realize network traffic classification. According to the balanced data and combined with the hybrid network model, the loss function is defined, and the training set is used to complete the training of the hybrid network model based on course learning to obtain a network anomaly detection model.

In one embodiment, the network anomaly detection method of the present invention further includes: after obtaining the network anomaly detection model, using a validation set to verify the network anomaly detection model, and after the validation is passed, using the network anomaly detection model to complete the classification detection of abnormal traffic data. After the validation is passed, the test set is input into the network anomaly detection model to evaluate the performance of the network anomaly detection model.

In one embodiment, before dividing the network traffic data set into a training set and a validation set, the network traffic data set is preprocessed by using one-hot encoding to convert the nominal features into binary vectors to meet the model's input format requirements. Then, the data set is standardized, which helps to improve the convergence speed and accuracy of the model.

In one embodiment, the data set selected by the network traffic data set includes but is not limited to the NSL-KDD data set. When the network traffic data set selects the NSL-KDD data set, the data set is divided into a training set KDDTrain and a test set KDDTest, and a part of the training set is divided into a validation set.

In one embodiment, a denoising autoencoder (DAE) is used to perform feature screening on the data in the training set to solve the data redundancy problem. By training DAE, we can learn the key feature representation of the data by adding noise to the input data and reconstructing the original data, effectively reducing the dimension of the feature and the redundancy of the data, and improving the effect and computational efficiency of the model.

In one embodiment, an adaptive synthetic sampling algorithm is used to balance the number of samples of each category of the data after feature screening to solve the problem of data imbalance. The algorithm measures the imbalance between samples of different categories and generates synthetic new sample points for minority category samples, thereby achieving balanced adjustment of the data set. The goal of this method is to increase the number of samples of minority category data to improve the performance and accuracy of the model when processing unbalanced data. In one embodiment, the method for constructing the hybrid network model is:

The deformable convolutional neural network DCNN includes an input layer, a deformable convolution layer, a deformable ROI pooling layer and a Dropout layer;

The improved Transformer model includes an encoder, wherein the encoder includes an Embedding layer, a multi-head self-attention layer, a residual connection and a layer normalization layer, and a feedforward network layer;

The balanced data are respectively input into the deformable convolutional neural network DCNN and the improved Transformer model, and the deformable convolutional neural network DCNN and the improved Transformer model respectively extract features from the input data and output them, as follows:

The balanced data is input into the input layer of the deformable convolutional neural network DCNN for encoding, the encoded data is input into the deformable convolutional layer for the variable convolution step, the result after the variable convolution step is input into the deformable ROI pooling layer, and after the pooling process of the deformable ROI pooling layer, it is transmitted to the Dropout layer, and the Dropout layer discards some neurons and then outputs the model output result of the deformable convolutional neural network DCNN;

The balanced data is input into the Embedding layer of the improved Transformer model for encoding, and then fused with the preset position encoding. The fused encoding is processed by the multi-head self-attention layer, residual connection and layer normalization layer, and feedforward network layer in turn. After the feedforward network layer, residual connection and layer normalization are performed again to obtain the output result of the improved Transformer model.

The output results of the deformable convolutional neural network DCNN and the improved Transformer model are fused and input into the self-attention module for feature extraction again, and the output of the self-attention module passes through a fully connected layer and is classified using a softmax function.

The deformable convolution layer in the deformable convolutional neural network DCNN of the present invention can learn to adapt to features of different shapes and scales through deformable convolution operations, and introduce nonlinearity by applying nonlinear activation functions; the deformable ROI pooling layer performs spatial pyramid pooling on the input data, extracts key features from receptive fields of different scales, and can perform adaptive pooling according to the position and shape of the target to improve the invariance and robustness of the features. The data processed by feature screening and data balancing is input into the deformable convolutional neural network DCNN, and features of the input data are extracted and output. The present invention makes improvements on the traditional Transformer model, retaining only the encoder part, which can simplify the traditional Transformer model and reduce the calculation and memory requirements of the model.

In one embodiment, the process of training the hybrid network model to obtain a network anomaly detection model is as follows:

Initializing parameters of the hybrid network model;

Inputting the balanced data into the hybrid network model;

Calculate the error between the output of the hybrid network model and the true value using the defined loss function;

According to the gradient of the loss function, using a gradient descent algorithm, the error is back-propagated back to each layer of the hybrid network model, and the parameters of the hybrid network model are adjusted layer by layer to reduce the error;

Repeat the above steps to set the training round threshold; when the training round reaches the threshold, stop training, obtain the anomaly detection model, and save the trained anomaly detection model for subsequent use.

In a second aspect, the present invention provides a network anomaly detection device for deep learning, comprising:

The abnormal traffic data acquisition module is used to obtain the abnormal traffic data to be detected, input it into the pre-trained network anomaly detection model for classification detection, and obtain the detection result;

A feature screening module is used to divide the acquired historical network traffic data set into a training set and a test set, and to divide a part of the training set into a validation set, and to perform feature screening on the data in the training set;

The data imbalance processing module is used to balance the number of samples in each category of the data after feature screening;

The network anomaly detection model construction module is used to construct a hybrid network model of a deformable convolutional neural network DCNN and an improved Transformer model to realize network traffic classification; based on the balanced processed data and in combination with the hybrid network model, the loss function is defined, the hybrid network model is trained, and a network anomaly detection model is obtained.

In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned network anomaly detection method.

In a fourth aspect, the present invention provides a computer device, comprising:

Memory for storing computer programs;

The processor is used to execute the computer program to implement the steps of the above-mentioned network anomaly detection method.

In a fifth aspect, the present invention provides a computer program product, including a computer program, which implements the steps of the above-mentioned network anomaly detection method when executed by a processor.

Compared with the prior art, the present invention has the following beneficial effects:

(1) The present invention can simultaneously extract the temporal features and spatial features of data through the fusion model of deformable convolutional neural network (DCNN) and Transformer, thus solving the problem of incomplete feature extraction. DCNN can effectively capture local features and spatial relationships through convolution operations, while Transformer can capture global features and long-range dependencies through the self-attention mechanism. The fusion model can comprehensively consider temporal and spatial features, improve feature expression capabilities, enhance the ability to detect complex abnormal patterns, and has the ability of adaptive feature learning, thereby improving the performance and robustness of the network anomaly detection model.

(2) The present invention uses a denoising autoencoder (DAE) algorithm to perform feature screening on data to solve the problem of data redundancy. DAE can automatically learn important features in the data and map the data to a more compact and informative representation, thereby reducing redundant information. DAE can highlight abnormal features and suppress noise and redundant features that are not related to the abnormality, which helps to improve the identifiability of abnormal features and makes it easier for anomaly detection models to capture and distinguish abnormal behaviors, thereby improving detection accuracy. Reducing the dimension of features can also reduce the complexity of the model, reduce the risk of model overfitting, improve the generalization ability of the model, and significantly improve the accuracy and robustness of network anomaly detection.

(3) The present invention uses the adaptive synthetic sampling algorithm ADASYN to solve the problem of data imbalance. The ADASYN algorithm can balance the distribution of the data set by adaptively synthesizing new minority class samples, making the number of normal samples and abnormal samples closer, which helps to improve the model's ability to identify abnormal samples and reduce overfitting of normal samples. When synthesizing new samples, the ADASYN algorithm takes into account the distance and distribution between samples. It analyzes the majority class samples around the minority class samples and retains the characteristics of the original samples when synthesizing new minority class samples. This makes the synthesized samples more representative of the true abnormal sample distribution and avoids the information loss that may be caused by simple replication or random synthesis.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a network anomaly detection method according to an embodiment of the present invention;

FIG2 is a schematic diagram of a hybrid network model structure in an embodiment of the present invention;

FIG3 is a schematic diagram of the principle of a DCNN convolutional layer in an embodiment of the present invention.

FIG4 is a schematic diagram of the principle of the DCNNROI pooling layer in an embodiment of the present invention;

FIG5 is a schematic diagram of deformable convolution sampling in an embodiment of the present invention, wherein FIG5a is a standard sampling position in a conventional CNN convolution step (the convolution range is a standard rectangle, light gray dots), and FIG5b is a deformed sampling position in a DCNN convolution step of the present invention (the convolution range is an adaptive shape of the same size that is extended outward from a standard rectangle, dark gray dots);

FIG6 is a schematic diagram of the module structure of the improved Transformer model in an embodiment of the present invention.

Detailed ways

The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

The term "and/or" is only a way to describe the association relationship of related objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character "/" generally indicates that the related objects are in an "or" relationship.

Example 1

FIG1 is a schematic diagram of an embodiment of a network anomaly detection method disclosed in the present invention. The embodiment of FIG1 can be specifically executed by a computer, a network traffic monitoring device, a security information and event management (SIEM) system, and an Internet of Things (IoT) gateway and platform, and can be applied to the following scenarios: 1. In a cloud computing environment, network anomaly detection can monitor and protect the security of resources such as cloud services, virtual machines, and containers. It can help identify abnormal network traffic, abnormal user behavior, or abnormal application activity to prevent unauthorized access or attacks. 2. In the Internet of Things, various devices and sensors communicate and exchange data through the network. Network anomaly detection can help detect abnormal device behavior, abnormal sensor data, or network attack activities, thereby protecting the security and stable operation of the IoT system.

As shown in FIG. 1 , step S1 of this embodiment is: dividing the network traffic data set into a training set and a test set, and dividing a part of the training set into a validation set, and performing feature screening on the data in the training set.

In a specific implementation of this embodiment, the data set selected from the network traffic data set includes but is not limited to the NSL-KDD data set. When the network traffic data set selects the NSL-KDD data set, the data set is divided into a training set KDDTrain and a test set KDDTest, and a part of the training set is divided into a validation set.

Step S11: Divide the network traffic data set into a training set and a test set, and divide a part of the training set into a validation set.

The NSL-KDD dataset is divided into a training set KDDTrain and a test set KDDTest in a ratio of 5:1, and 20% of the training set is extracted as a validation set.

Step S12: Perform feature screening on the data in the training set.

Feature screening is an important step in abnormal traffic detection. Through feature screening, redundant and irrelevant features can be removed, thereby reducing noise and interference and reducing data dimensions. By selecting the optimal feature subset from a large feature set, the computational complexity can be reduced and the accuracy and generalization ability of the model can be improved. Therefore, a denoising autoencoder DAE with a weighted loss function is selected to perform feature screening on the training set. DAE can extract more meaningful features by introducing noise. Compared with the principal component analysis method, DAE is a nonlinear feature extraction method that can better capture nonlinear relationships and complex patterns in the data. DAE with a weighted loss function assigns different weights to normal traffic and abnormal traffic by using a weighted loss function, thereby inducing DAE to pay more attention to the reconstruction of attack samples during the training process, so that the selection result is conducive to improving the performance of anomaly detection.

Calculate the L2 norm of the row vector of each feature in the encoder weight matrix as the weight of the feature. for:

(1)

Where, m is the number of samples; is the reconstruction error calculated by the mean square error MSE; /> is the loss caused by the L2 regularization term; /> is the transpose of the weight matrix, and the final loss function is obtained by combining the weight matrix into the mean square error loss.

As shown in FIG. 1 , step S2 of this embodiment is: performing a balancing process on the number of samples of each category on the data after feature screening.

Adaptive Synthetic Sampling (ADASYN) is used to solve the problem of data imbalance. The algorithm measures the imbalance between samples of different categories and generates new synthetic sample points for minority category samples to achieve balanced adjustment of the data set. The goal of this method is to increase the number of samples of minority category data to improve the performance and accuracy of the model when processing unbalanced data. The ADASYN algorithm process is as follows:

Step S21, calculate the category imbalance degree d :

(2)

in, and/> Respectively represent the number of minority class samples and the number of majority class samples,/> ≤/> , , m is the number of samples.

Step S22: if d < ,/> is the preset threshold of the maximum class imbalance ratio, then:

Step S221, calculate the total number G of synthetic data examples that need to be generated for the minority class:

(3)

in, is a parameter that specifies the desired level of balance after generating synthetic data. /> Meaning a completely balanced dataset is created after the generalization process.

Step S222, for each sample in the minority class , find the K nearest neighbors based on the Euclidean distance in N-dimensional space and calculate the ratio/> :

(4)

in, For/> The number of examples belonging to the majority class among the K nearest neighbors of ,/> Indicates/> minority class samples.

Step S223: After standardization, the proportion of each category can be obtained/> .

(5)

is a density distribution, /> .

Step S224, calculate the samples required for each minority class The number of synthetic data examples generated/> .

(6)

where G is the total number of synthetic data examples that need to be generated for the minority class defined in Equation (3).

Step S225, for each sample in the minority class , follow the steps below to generate synthetic data samples.

1) From each minority class sample Randomly select a few data examples from the K nearest neighbors of /> .

2) Generate synthetic data examples :

(7)

in, is the difference vector in N-dimensional space;/> is a random number, /> . From 1 to /> Repeat the process until the required number of synthesized particles is reached.

A balanced dataset can more accurately reflect the relationship between different categories, which helps the hybrid network model to learn and classify better. Through data augmentation, we obtain more sample data, which can improve the generalization ability and robustness of the network anomaly detection model. Finally, the features are normalized to facilitate convergence.

As shown in Figures 1 and 2, step S3 of this embodiment: construct a hybrid network model of a deformable convolutional neural network DCNN and an improved Transformer model to realize network traffic classification; based on the balanced data, and in combination with the hybrid network model, complete the definition of the loss function, use the training set to complete the training of the hybrid network model based on course learning, obtain a network anomaly detection model, and use the network anomaly detection model to classify and detect abnormal traffic data to obtain detection results.

The deformable convolutional neural network DCNN includes an input layer, a deformable convolutional layer, a deformable ROI pooling layer, and a Dropout layer. There are two deformable convolutional layers and one deformable pooling layer. The deformable convolutional neural network DCNN is stacked by two layers, and its module structure is shown in Figure 2. DCNN improves two key modules on the basis of CNN, and the principles are shown in Figures 3 and 4. DCNN adds a new convolutional layer to the traditional convolution step to extract the geometric deformation offset of the convolution range. After the shape is changed, the range is no longer a standard rectangle, but an adaptive shape of the same size that expands outward, as shown in Figure 5. The deformable ROI pooling layer adds an additional pooling layer on the basis of the standard pooling layer to extract and generate the offset matrix. DCNN introduces the offset matrix in the traditional convolutional layer and pooling layer respectively, so that the model can adaptively extract more valuable feature information, greatly improving the local feature extraction ability of CNN.

It should be noted that in Figure 2, the random dropout layer is the Dropout layer, the fully connected layer is the FC, transformer represents a transformer model, which is a neural network model, and softmax represents the Softmax function. In Figure 4, RoI represents the region of interest, which is an operation used in convolutional neural networks.

The specific steps of step S3 are as follows:

Step S31, input the data processed by steps S1 and S2 into a deformable convolutional neural network to extract features from the data.

In this example, we set the convolution kernel to a one-dimensional convolution kernel with a window size of 3 and a stride of 1. The deformable convolution layer changes the sampling grid of the convolution operation by introducing a learnable offset, enabling it to dynamically adjust the shape and position according to the local features of the input. Specifically, the deformable convolution uses an offset prediction network at each position, which learns to generate an offset field to adjust the sampling position of the convolution kernel on the input. In this way, the deformable convolution can adaptively adapt to the deformation and posture changes of the target and model irregularly shaped targets more accurately. After changing the shape, the range is no longer a standard rectangle, but an adaptive shape of the same size that expands outward (see Figure 5). The migration matrix defines the size and range of the range of the convolution kernel. The closer the offset sampling point is to the key information point, the more valuable the extracted features are. The deformable convolution process can be expressed as:

(8)

in, Indicates the position of the point on the input feature map, /> Is the point in the range of the convolution kernel/> Position in, /> yes The offset value, n represents different position indexes. /> Each input /> Output features after the offset matrix; is the weight corresponding to the sampling position; /> is a discrete function; the offset value/> Not actual points. Therefore, we use bilinear difference to calculate the value of the feature at discrete locations in the one-dimensional feature map. /> The definition is as follows:

(9)

(10)

Here, q represents all integer positions on the input feature map and p represents any (fractional) position.

The output of the deformable convolutional layer is input into the deformable ROI pooling layer, and the window size of the deformable pooling layer is 2 and the stride is 1. The deformable ROI pooling layer adds an additional pooling layer on the basis of the standard pooling layer to extract and generate the offset matrix, and then the offset matrix and the standard pooling layer work together to obtain the information after the deformable ROI feature is summarized. The deformable ROI pooling layer can be expressed as follows:

(11)

in, Indicates the number of points in the range. Get /> The process is as follows. First, ROI pooling generates a pooled feature map. The FC layer generates a normalized offset based on the mapping. However,/> The size of each range /> The sizes are not the same, so we use the following formula to calculate:

(12)

in, is the gain scalar, default/> ;◦ represents the product of the elements in the matrix. /> Represents a two-dimensional offset. The input feature map outputs a feature map under the action of the generated offset matrix and the pooling layer to achieve feature extraction and dimensionality reduction.

The data after deformable ROI pooling is input into the Dropout layer. Dropout randomly discards some neurons. By randomly discarding neurons, Dropout reduces the correlation between neurons, enhances the generalization ability and robustness of the network, suppresses overfitting and co-adaptation, and improves the learning effect and stability of the network.

Step S32, input the data processed by steps S1 and S2 into the improved Transformer model to extract features from the data.

In this example, the schematic diagram of the improved Transformer model structure is shown in Figure 6. The traditional Transformer model includes an encoder and a decoder, while the present invention improves the Transformer model. The improved Transformer model only includes an encoder, which can simplify the model and reduce the calculation and memory requirements of the model. The encoder module includes an Embedding layer, a multi-head self-attention layer, a residual connection and a layer normalization layer, and a feedforward network layer.

In Transformer, in addition to the input embedding vector representing the semantic information of the input data, position encoding is also required to represent the position information of the data in the sequence. Position encoding is represented by PE , and the dimension of PE is the same as the input embedding vector. In Transformer, a position encoding method based on formula calculation is adopted. The calculation formula is as follows:

(13)

(14)

Among them, pos represents the position of the word in the sequence, represents the dimension of PE , 2t represents the even dimension, and 2t+1 represents the odd dimension (i.e./> ,/> ). PE _{( pos + k )} at any position (k represents the offset of the position) can be represented by a linear function of PE _{( pos )} :

(15)

(16)

in, Indicates the index of the current position, /> It is the parameter of position encoding, which is used to control the frequency and phase of position encoding.

In the Transformer model, the position encoding is added element by element to the input embedding vector to obtain the final input containing the position information. In this way, when performing self-attention calculations, the Transformer model can not only consider the semantic information of the data, but also its position information in the sequence, thereby better capturing the temporal relationship in the sequence. It should be noted that the position encoding is fixed and will not be updated as the model is trained. Its role is to introduce position information without introducing additional model parameters.

The Transformer model is built around the self-attention mechanism, which requires matrices Q (query), K (key), and V (value) for calculations. The self-attention mechanism receives input (matrix X composed of the data representation vector x) or the output of the previous Encoder. Q, K, and V are obtained by linearly transforming the input of the self-attention mechanism. The input of the self-attention mechanism is represented by the matrix X, and the linear transformation matrix W _q Q, W _k K, and W _v V can be used to calculate Q, K, and V. After obtaining the matrices Q, K, and V, the output of the self-attention mechanism can be calculated. The calculation formula is as follows:

(17)

Among them, ds is the number _of columns of the Q, K matrices, that is, the vector dimension.

The correlation between the current position feature and the features at other positions can be calculated by multiplying the Q (query) matrix and the K (key) matrix. The higher the correlation, the larger the value of the product result. Divide the result of the product by the square root of the degree of the vector to ensure that the degree is not too large or too small to remain stable. Then perform softmax processing on the structure to calculate the importance of the correlation between the current position feature and the features at other positions. The softmax operation can convert the correlation score into a probability distribution. Multiply the softmax calculation result with the V (value) matrix to give greater weight to features with higher correlation and weaken irrelevant information. This product result will be used as the output of the self-attention mechanism for subsequent calculations and processing.

The multi-head self-attention mechanism consists of multiple self-attention layers. First, the input X is passed to In different self-attention mechanisms, it is calculated that /> The multi-head self-attention mechanism concatenates them together and then passes them into a Linear layer to obtain the final output Z of the multi-head self-attention mechanism. The dimension of the matrix Z output by the multi-head self-attention mechanism is the same as its input matrix X. The entire calculation process can be expressed as:

(18)

(19)

in, ,/> is the additional weight matrix, /> is the matrix splicing method, /> Represents the embedding dimension/> and/> represents the hidden dimension of the projected subspace.

Residual connection refers to adding the input directly to the output of a layer so that the model can learn the residual information. In Transformer, each sublayer contains a residual connection to add the input to the output of the sublayer, indicating that the output of the sublayer is composed of the input and the transformation of the sublayer. This design helps to alleviate the gradient vanishing and gradient exploding problems in deep networks. Layer normalization is a normalization method used to normalize the input of each layer of a neural network. Through layer normalization, the training stability and generalization performance of the model can be improved. The calculation method is as follows:

(20)

(twenty one)

(twenty two)

in, and/> are the mean and standard deviation of each element of X along the last dimension, /> and/> is a learnable parameter, Indicates element-wise multiplication, /> is a constant, its function is to stabilize the value, c represents the total number of elements/> , j represents the jth element.

The feedforward network is a fully connected feedforward network. The data at each position passes through this identical feedforward neural network separately. The feedforward network is used to perform nonlinear transformations on the hidden layer at each position, mapping the output of the multi-head attention layer to another dimensional space, thereby improving the expressiveness of the model. The feedforward network consists of two fully connected layers. The activation function of the first fully connected layer is the ReLU activation function, and the second fully connected layer does not use an activation function, which can be expressed as:

(twenty three)

in, is the activation function, W1 _and W2 _are weight parameters, _and b1 _and b2 are bias parameters.

Step S33, the output of the deformable convolutional neural network DCNN and the output of the improved Transformer model are fused and input into the self-attention module, and the process is shown in Figure 2. Through this mechanism, the hybrid network model can adaptively adjust the connection weights and the importance of features, so that the features that are critical to the task get higher weights, thereby improving the accuracy and performance of the network anomaly detection model. The output of the self-attention module passes through a fully connected layer and is classified by the softmax function to obtain the classification result.

Step S34, combining the data sets processed by step S1 and step S2, and combining the scale of the hybrid network model to complete the definition of the loss function, the loss function is a cross entropy function commonly used in multi-classification, and the calculation formula is as follows:

(twenty four)

in, is the true label, /> is the predicted class distribution, /> Indicates the first/> samples, z is the total number of samples.

Step S35, the training process of the hybrid network model is as follows:

Step S351, initializing model parameters: initially setting the parameters of the deformable convolutional neural network and the improved Transformer model, as well as the weights and biases of the fully connected layer.

Step S352, input data: input the training samples in the data set into the hybrid network model to obtain the output of the hybrid network model.

Step S353, calculating the loss function: using the defined loss function to calculate the error between the output of the hybrid network model and the true value.

Step S354, back propagation and parameter update: using the gradient descent method or other optimization algorithms, according to the gradient of the loss function, the error is propagated layer by layer and the parameters of the hybrid network model are adjusted to reduce the error between the true value and the model output.

Step S355, repeat the training steps: loop through steps S352 to S354, input the sample into the hybrid network model, calculate the error, perform back propagation and update the parameters. Through iterative optimization, the hybrid network model gradually learns and adjusts itself to better fit the training data.

Step S356, determine the stop condition: monitor the indicators during the training process and set the stop condition. Once the stop condition is met, that is, the preset training round threshold is reached, the training is stopped, the network anomaly detection model is obtained, and the trained network anomaly detection model is saved.

As shown in FIG. 1 , step S4 of this embodiment: using a verification set to verify the network anomaly detection model, after the verification is passed, using the network anomaly detection model to complete classification detection of abnormal traffic data.

Specific steps are as follows:

Step S41: input the verification set divided in step S1 into the trained network anomaly detection model for verification.

Step S42: input the test set into the network anomaly detection model to evaluate the performance of the network anomaly detection model.

After verification, the test set KDDTest is input into the saved network anomaly detection model for testing. We will record the test results, as shown in Table 1, for further analysis, evaluation, and improvement of the model's performance. This process is a key step in evaluating the model, which can help us verify the robustness and reliability of the model and provide valuable guidance for anomaly detection problems in practical applications.

In summary, through the network anomaly detection model of the present invention, we can effectively perform network anomaly detection and improve the accuracy and robustness of detection. This provides important technical support for the field of network security and helps us better protect network systems from malicious attacks and abnormal behaviors.

Table 1: Test results

.

Example 2

Based on the same inventive concept as Example 1, this embodiment introduces a network anomaly detection device for deep learning, including:

The abnormal traffic data acquisition module is used to obtain the abnormal traffic data to be detected, input it into the pre-trained network anomaly detection model for classification detection, and obtain the detection result;

A feature screening module is used to divide the acquired historical network traffic data set into a training set and a test set, and to divide a part of the training set into a validation set, and to perform feature screening on the data in the training set;

The data imbalance processing module is used to balance the number of samples in each category of the data after feature screening;

The network anomaly detection model construction module is used to construct a hybrid network model of a deformable convolutional neural network DCNN and an improved Transformer model to realize network traffic classification; based on the balanced processed data and in combination with the hybrid network model, the loss function is defined, the hybrid network model is trained, and a network anomaly detection model is obtained.

Furthermore, the network anomaly detection device of the present invention also includes a verification module, which is used to verify the network anomaly detection model using a verification set after obtaining the network anomaly detection model. After the verification is passed, the test set is input into the network anomaly detection model to evaluate the performance of the network anomaly detection model.

Example 3

Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned network anomaly detection method are implemented.

Example 4

Based on the same inventive concept as other embodiments, this embodiment introduces a computer device, including: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the above-mentioned network anomaly detection method.

Example 5

Based on the same inventive concept as other embodiments, this embodiment introduces a computer program product, including a computer program, which implements the steps of the above-mentioned network anomaly detection method when executed by a processor.

In summary, the present invention can extract the temporal and spatial features of data simultaneously through the fusion model of deformable convolutional neural network DCNN and Transformer, solving the problem of incomplete feature extraction; using the denoising autoencoder DAE algorithm to perform feature screening on data to solve the problem of data redundancy; using the adaptive synthetic sampling algorithm ADASYN to solve the problem of data imbalance. Therefore, the accuracy and robustness of network anomaly detection are significantly improved.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

The present invention is described with reference to the flowcharts and/or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the processes and/or boxes in the flowchart and/or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims (9)

1. The network anomaly detection method for deep learning is characterized by comprising the following steps of:

Obtaining abnormal flow data to be detected, inputting the abnormal flow data to a pre-trained network abnormal detection model for classification detection, and obtaining a detection result;

the training process of the trained network anomaly detection model comprises the following steps:

dividing the acquired historical network flow data set into a training set and a testing set, dividing a part of the training set into a verification set, and screening the characteristics of the data in the training set;

Performing balancing processing on the sample number of each class on the data subjected to the feature screening;

Constructing a mixed network model of a deformable convolutional neural network DCNN and an improved transducer model for realizing network traffic classification; according to the data after the balance processing, the definition of a loss function is completed by combining the hybrid network model, and the hybrid network model is trained to obtain a network anomaly detection model;

the construction method of the hybrid network model comprises the following steps:

The deformable convolutional neural network DCNN comprises an input layer, a deformable convolutional layer, a deformable ROI pooling layer and a Dropout layer;

The improved transducer model includes an encoder comprising Embedding layers, a multi-headed self-attention layer, a residual connection and layer normalization layer, and a feed forward network layer;

The data after the balance processing are respectively input into a deformable convolutional neural network DCNN and an improved transducer model, and the deformable convolutional neural network DCNN and the improved transducer model respectively extract and output the characteristics of the input data, specifically as follows:

Inputting the balanced data into an input layer of a Deformable Convolutional Neural Network (DCNN) for encoding, inputting the encoded data into the deformable convolutional layer for performing a deformable convolutional step, inputting the result processed by the deformable convolutional step into a deformable ROI pooling layer, transmitting the result to a Dropout layer after pooling processing of the deformable ROI pooling layer, and outputting a model output result of the Deformable Convolutional Neural Network (DCNN) after discarding part of neurons by the Dropout layer;

inputting the balanced data into Embedding layers of an improved transducer model for encoding, fusing the encoded data with preset position codes, sequentially processing the fused codes through a multi-head self-attention layer, a residual error connection and layer normalization layer and a feedforward network layer, and processing the residual error connection and layer normalization layer after the feedforward network layer to obtain an improved transducer model output result after processing;

And merging and inputting the output results of the deformable convolutional neural network DCNN and the improved transducer model into a self-attention module to perform feature extraction again, and classifying the output of the self-attention module through a full-connection layer and then utilizing a softmax function.

2. The deep learning oriented network anomaly detection method of claim 1, further comprising: and after the network anomaly detection model is obtained, the network anomaly detection model is verified by using a verification set, and after the verification is passed, the classification detection of the anomaly traffic data is completed by using the network anomaly detection model.

3. The deep learning-oriented network anomaly detection method of claim 1, wherein the method comprises the steps of: the data in the training set is feature filtered using a de-noised self-encoder DAE.

4. The deep learning-oriented network anomaly detection method of claim 1, wherein the method comprises the steps of: and carrying out balancing processing on the sample number of each class on the data subjected to the feature screening by using an adaptive synthetic sampling algorithm.

5. The deep learning-oriented network anomaly detection method of claim 1, wherein training the hybrid network model to obtain a network anomaly detection model comprises:

initializing parameters of the hybrid network model;

Inputting the balanced processed data into the hybrid network model;

calculating an error between the hybrid network model output and the true value using the defined loss function;

According to the gradient of the loss function, using a gradient descent algorithm to reversely propagate the error back to each layer of the hybrid network model, and adjusting parameters of the hybrid network model layer by layer;

Repeating the steps, and setting a training round threshold value; and stopping training when the training round reaches the threshold value to obtain an abnormality detection model.

6. The network anomaly detection device facing deep learning is characterized by comprising:

the abnormal flow data acquisition module is used for acquiring abnormal flow data to be detected, inputting the abnormal flow data to a pre-trained network abnormal detection model for classification detection, and obtaining a detection result;

the feature screening module is used for dividing the acquired historical network flow data set into a training set and a testing set, dividing a part of the training set into a verification set, and carrying out feature screening on the data in the training set;

The data unbalance processing module is used for carrying out balance processing on the sample number of each class on the data subjected to the feature screening;

The network anomaly detection model construction module is used for constructing a mixed network model of a deformable convolutional neural network DCNN and an improved transducer model and is used for realizing network traffic classification; and according to the data after the balance processing, the definition of the loss function is completed by combining the hybrid network model, and the hybrid network model is trained to obtain a network anomaly detection model.

7. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processor, performs the steps of the network anomaly detection method according to any one of claims 1 to 5.

8. A computer device, comprising:

A memory for storing a computer program;

A processor for executing the computer program to implement the steps of the network anomaly detection method of any one of claims 1 to 5.

9. A computer program product comprising a computer program characterized by: the computer program when executed by a processor implements the steps of the network anomaly detection method of any one of claims 1 to 5.