CN111371637A - An abnormal data analysis method based on circular multi-moving fog nodes - Google Patents

Abstract

The invention discloses a method for analyzing abnormal data based on circular multi-moving fog nodes. The steps are as follows: a cloud server adopts a clustering algorithm to divide an area into a plurality of sub-areas, and sets the centroid position of the sub-areas as the location of the mobile fog node. Then, by constructing a sparse coefficient matrix, the eigenvalues of all sub-region data are obtained and broadcast; multiple mobile fog nodes visit all resident points; when moving, the mobile fog nodes upload the data collected at the last resident point to The cloud server is used to update the characteristic value and rebroadcast it to all mobile fog nodes; when the mobile fog node is at the resident point, it collects the data of the sub-area and analyzes the abnormality according to the data characteristic value provided by the cloud server, and uploads the analysis results. To the cloud server, the cloud server takes corresponding measures according to the analysis results. The invention is suitable for the large-scale Internet of Things, can collect network data in time and perform abnormal analysis, and realize the real-time collection, analysis and processing of the whole network data.

Description

An abnormal data analysis method based on circular multi-moving fog nodes

technical field

The invention relates to an abnormal data analysis method based on cyclic multi-moving fog nodes, belonging to the technical field of big data and machine learning.

Background technique

In the traditional edge computing model, fog nodes are deployed around sensor nodes to provide computing, storage and other services nearby. As middleware to connect sensors and cloud servers, fog nodes can eliminate the data processing burden of centralized infrastructure, and can also generate faster network service responses, which can meet the basic needs of the industry in terms of real-time, reliability, and security. However, one of the key problems encountered by the application of edge computing technology in the IoT environment is its deployment and maintenance costs. Especially in the industrial IoT environment, the high-noise and low-latency communication environment requires higher fog node deployment and maintenance costs.

SUMMARY OF THE INVENTION

Aiming at the deficiencies of the prior art, the present invention provides an abnormal data analysis method based on cyclic multi-mobile fog nodes, which improves the reliability and real-time performance of data collection and data analysis in the Industrial Internet of Things, and reduces the deployment and maintenance of edge nodes at the same time. cost.

In order to achieve the above object, the technical scheme of the present invention is achieved in this way:

A method for analyzing abnormal data based on circular multi-moving fog nodes, comprising the following steps:

Step 1: According to the size and density of the responsible area and the location of each IoT node, the cloud server uses a conventional clustering algorithm to divide the area into multiple sub-areas, and set the centroid of the sub-area as the location of the mobile fog node. stay point;

Step 2: The cloud server obtains the eigenvalues of all sub-region data by constructing a sparse coefficient matrix, and broadcasts it to all mobile fog nodes;

Step 3: Multiple mobile fog nodes use a conventional traversal algorithm to iteratively visit all resident points;

Step 4: When the mobile fog node is moving, upload the data collected at the last resident point to the cloud server. The cloud server updates the data feature value according to the original data and the newly collected data, and rebroadcasts it to all mobile fog nodes. During this movement, the information required for anomaly analysis is exchanged between the mobile fog nodes;

Step 5: When the mobile fog node is at the resident point, collect the data of the IoT nodes in the sub-region and analyze the abnormality according to the data characteristic values provided by the cloud server, upload the analysis results to the cloud server, and the cloud server will take corresponding measures according to the analysis results. measure;

Step 6: When multiple mobile fog nodes move cyclically at multiple residence points, they continue to collect data, analyze abnormality, upload analysis results, and upload data.

Preferably, the mobile fog node can communicate with all sensor nodes in the area it is responsible for, and can communicate with the mobile fog nodes in the front and rear of it, and all the mobile fog nodes can directly communicate with the cloud server .

Preferably, the communication, computing and storage capabilities of the mobile fog node are responsible for data collection and anomaly analysis in the area it needs to be responsible for.

Preferably, in the step 1, the conventional clustering algorithms used include but are not limited to location-based clustering algorithms, hierarchical-based clustering algorithms, density-based clustering algorithms and model-based clustering algorithms.

Preferably, in the step 2, the cloud server obtains the classification features and private features of the data by performing sparse transformation on the collected data.

Preferably, in the step 3, the conventional traversal algorithms used include, but are not limited to, the lawnmower algorithm, the A* algorithm, the depth-first search algorithm and the breadth-first search algorithm.

Preferably, in the step 6, the movement and stay of a plurality of mobile fog nodes are coordinated and performed sequentially, that is, data collection, analysis, and upload are coordinated and performed sequentially, and the data collection, analysis, and upload performed by the mobile fog nodes are responsible for For small-scale data in a sub-region, the data features extracted by the cloud server are large-scale data in the entire region.

Beneficial effects: Aiming at the large-scale industrial Internet of Things environment, the present invention studies a highly reliable and low-cost edge computing method based on mobile fog nodes, and proposes an abnormal data analysis method based on circular multi-mobile fog nodes, which can timely and accurately collect It can realize real-time monitoring and response to industrial IoT anomalies, and reduce the deployment and maintenance costs of fog node equipment.

Description of drawings

1 is a flow chart of a method according to Embodiment 1 of the present invention;

FIG. 2 is a schematic structural diagram of the present invention.

Detailed ways

In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present application, and Not all examples. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application.

A method for analyzing abnormal data based on circular multi-moving fog nodes, comprising the following steps:

Step 1: According to the size and density of the responsible area and the location of each IoT node, the cloud server uses a conventional clustering algorithm to divide the area into multiple sub-areas, and set the centroid of the sub-area as the location of the mobile fog node. stay point;

Step 2: The cloud server obtains the eigenvalues of all sub-region data by constructing a sparse coefficient matrix, and broadcasts it to all mobile fog nodes;

Step 3: Multiple mobile fog nodes use a conventional traversal algorithm to iteratively visit all resident points;

Step 4: When the mobile fog node is moving, upload the data collected at the last resident point to the cloud server. The cloud server updates the data feature value according to the original data and the newly collected data, and rebroadcasts it to all mobile fog nodes. During this movement, the information required for anomaly analysis can be exchanged between the mobile fog nodes;

Step 5: When the mobile fog node is at the resident point, collect the data of the IoT nodes in the sub-region and analyze the abnormality according to the data characteristic values provided by the cloud server, upload the analysis results to the cloud server, and the cloud server will take corresponding measures according to the analysis results. measure;

Step 6: When multiple mobile fog nodes move cyclically at multiple residence points, they continue to collect data, analyze abnormality, upload analysis results, and upload data.

Preferably, the mobile fog node can communicate with all sensor nodes in the area it is responsible for, and can communicate with the mobile fog nodes in its front and rear positions, and all the mobile fog nodes can communicate directly with the cloud server.

Preferably, the communication, computing and storage capabilities of the mobile fog node can undertake data collection and anomaly analysis in the area it needs to be responsible for, and the communication, computing and storage capabilities of the cloud server meet all communication, computing and storage requirements.

Preferably, the conventional clustering algorithms employed include, but are not limited to, location-based clustering algorithms (such as K-MEANS), hierarchical-based clustering algorithms (such as BRICH), density-based clustering algorithms (such as DBSCAN), and model-based clustering algorithms clustering algorithms such as GMM.

Preferably, in the step 2, the cloud server obtains the classification features and private features of the data by performing sparse transformation on the collected data, and the data type may be large-scale high-dimensional data.

Preferably, in the step 3, the conventional traversal algorithms used include, but are not limited to, the lawnmower algorithm, the A* algorithm, the depth-first search algorithm and the breadth-first search algorithm.

Preferably, in the step 6, the movement and stay of the multiple mobile fog nodes are not synchronized, that is, the data collection, analysis, and upload are not carried out at the same time, but are carried out in sequence in coordination with each other, which can effectively improve the utilization rate of communication resources. Among them, the data collection, analysis and upload of the mobile fog node is responsible for the data in the sub-area, and the data feature extraction by the cloud server is the data of the whole area.

Example 1:

Taking the data analysis in the large-scale industrial Internet of Things as an example, the mobile edge server of the touring movement can be an AGV car in the factory, and various equipment in the factory can be a data collection point. Cloud servers perform data collection, exception analysis, and exception handling.

A flowchart of a method for analyzing abnormal data based on cyclic multi-mobile fog nodes in this embodiment is shown in FIG. 1 , and the method includes the following steps:

S1 area division, set the resident point of each sub-area;

S2 obtains the characteristic value of the sub-region and broadcasts it;

S3 fog nodes access all residencies;

Upload data and update eigenvalues when S4 moves;

Abnormal analysis is performed when S5 stays;

S6 repeats steps S3-S5.

In step S1, the cloud server can learn the locations of all sensor nodes (IoT nodes) in the area and the data types collected, and use conventional clustering algorithms to classify all sensor nodes according to the correlation between the locations and data types. Conventional clustering algorithms include, but are not limited to, location-based clustering (such as K-MEANS), hierarchical-based clustering (such as BRICH), density-based clustering (such as DBSCAN), and model-based clustering (such as GMM) algorithms . The cloud server divides the area Z into multiple sub-areas {Z ₁ , Z ₂ , Z ₃ ... Z _n |n∈N ⁺ } according to the classification result, and sets the centroid position of the sub-area as the resident point of the mobile fog node {S ₁ , S ₂ , S ₃ . . . S _n |n ∈ N ⁺ }.

。根据稀疏矩阵分解法，传感器节点的数据可以表示为In step S2, the cloud server obtains data of sensor nodes in all sub-areas by moving the fog nodes. The cloud server obtains the eigenvalues of all sub-region data by constructing a sparse coefficient matrix, and broadcasts it to all moving fog nodes. For example, in the same sub-region Z ₁ , there are sensor nodes data χ=[x ₁ , x ₂ , x ₃ ...... x _M ],

. According to the sparse matrix factorization method, the data of sensor nodes can be expressed as

x=Ψα(1);

where Ψ is the sparse basis and α is the sparse coefficient.

For a given sparse base, the sparse coefficients are unique, and the sparse coefficients of the same data class are related, and can be represented linearly by the sparse coefficient set of the same data class. Therefore, the sparse coefficient can be used as the main parameter of classification, and the sparse coefficient obtained by the sparse transformation can be used to classify the data of the sensor node.

则In the same sub-region Z1, the data _of sensor nodes have similar characteristics and belong to the same class. Assuming that there are n sensor node data and c data classes, sensor data x _i (i=1,2,...,n) can be represented, and

but

Where M is the dimension of the spatial domain, y _i is the class label corresponding to the data x _i , then any data class can be represented by a matrix, as shown in formula (3):

Where x _{j, q} represents the jth data class, n _j is the amount of sensor data,

According to the data representation method in the joint sparse model, the public sparse and unique sparse are combined to represent the data _{j, q} , as shown in formula (4):

x _j,q =z _cm,j +z _j,q =Ψα _cm,j +Ψα _s,j (q=1,2,...,n _j ,j=1,2,...,c)(4);

where z _cm,j and z _j,q represent the common and unique parts of x _j ,q respectively, and z _cm,j =Ψα _cm,j , z _j,q =Ψα _s,j , Ψ sparse basis, α _cm,j and α _s,j are the common and unique parts of sparse coefficients, and all data classes contain common parts, and their unique parts are different from each other.

The cloud server uses the public part of the sparse coefficient to obtain the classification feature of the data, and obtains the private feature of the data through the unique part of the sparse coefficient. Obviously, their combination can uniquely represent and identify any data, and is suitable for large-scale high-dimensional data. After obtaining all classified features and private features, the cloud server broadcasts it to all mobile fog nodes.

In step S3, the mobile fog node can select the optimal traversal algorithm to visit all the resident points according to the scale of the environment, the type of the environment, and the distribution of the resident points. For example, the scale of the environment is small, and the resident points are regularly and evenly distributed, which can be traversed by the lawn mower algorithm.

In step S4, the mobile fog node has two states of moving and staying. The mobile fog node mainly performs data transmission when moving. The first task is to upload the original data and abnormal analysis results to the cloud server. The cloud server uses the original data instead of the processed data for feature analysis, which is conducive to improving the detection accuracy. The abnormal analysis result is used to determine what measures to take; the second task is to receive the instructions of the cloud server, receive the characteristic values re-given by the cloud server to update the characteristic values, and receive the operation instructions of what measures to take.

In step S5, when the mobile fog node stays at the resident point, anomaly analysis is mainly performed according to the classification features and private features given by the cloud server. In the initial stage, unsupervised learning is used, and the conventional clustering algorithm in step S1 is used to obtain the preliminary classification of the data, that is, normal and abnormal, and at this stage, the true positive rate (the probability that normal data is detected as normal) is maximized; The outliers obtained in the initial stage are refined and classified (normal values, noise anomalies, environmental anomalies, equipment anomalies, attack anomalies, etc.).

In step S6, the movement and stay of the multiple mobile fog nodes are not synchronized, that is, the data collection, analysis, and upload are not performed simultaneously, as shown in FIG. 2 . According to the time required for moving and staying, it can effectively improve the utilization rate of communication resources. The data collection, analysis and upload of the mobile fog node is responsible for the small-scale data in the sub-area, and the data characteristics of the cloud server. Extracted as large-scale data for the whole region.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Both modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (7)

1. An abnormal data analysis method based on circulating multi-mobile-mist nodes is characterized by comprising the following steps:

step 1: the cloud server divides the region into a plurality of sub-regions by adopting a conventional clustering algorithm according to the size and density of the responsible region and the positions of the nodes of the Internet of things, and sets the mass center positions of the sub-regions as the residence points of the mobile fog nodes;

step 2: the cloud server acquires characteristic values of all sub-region data by constructing a sparse coefficient matrix and broadcasts the characteristic values to all mobile fog nodes;

and step 3: a plurality of mobile fog nodes circularly visit all the residence points by adopting a conventional traversal algorithm;

and 4, step 4: when the mobile fog nodes move, the data collected at the last resident point is uploaded to the cloud server, the cloud server updates the characteristic values of the data according to the original data and the newly collected data and broadcasts the characteristic values to all the mobile fog nodes again, and information required by anomaly analysis is exchanged among the mobile fog nodes during the moving period;

and 5: when the mobile fog node is at a residence point, collecting data of the sub-area Internet of things node, performing anomaly analysis on the data according to data characteristic values provided by the cloud server, uploading an analysis result to the cloud server, and taking corresponding measures by the cloud server according to the analysis result;

step 6: and when the plurality of mobile fog nodes circularly move at the plurality of residence points, the tasks of collecting data, analyzing the abnormity, uploading the analysis result and uploading the data are continuously carried out.

2. The method for analyzing the abnormal data based on the circulating type multiple mobile fog nodes is characterized in that the mobile fog nodes can communicate with all sensor nodes in the area in which the mobile fog nodes are responsible, can communicate with the mobile fog nodes in the front and back positions of the mobile fog nodes, and can directly communicate with the cloud server.

3. The abnormal data analysis method based on the circulating multi-mobile-fog-node is characterized in that the communication, calculation and storage capacity of the mobile fog node is used for data collection and abnormal analysis of the required responsible area.

4. The method for analyzing abnormal data based on the circulating multi-mobile fog node as claimed in claim 1, wherein the conventional clustering algorithm adopted in the step 1 includes but is not limited to a location-based clustering algorithm, a hierarchy-based clustering algorithm, a density-based clustering algorithm and a model-based clustering algorithm.

5. The abnormal data analysis method based on the circulating multi-mobile-fog node as claimed in claim 1, wherein in the step 2, the cloud server obtains the classification characteristics and the private characteristics of the data by performing sparse transformation on the collected data.

6. The method for analyzing the abnormal data based on the circulating multi-mobile-fog node as claimed in claim 1, wherein the conventional traversal algorithm adopted in the step 3 includes but is not limited to a mower algorithm, an a-x algorithm, a depth-first search algorithm and a breadth-first search algorithm.

7. The abnormal data analysis method based on the circulating type multiple mobile fog nodes as claimed in claim 1, wherein in the step 6, the movement and the stay of the multiple mobile fog nodes are coordinated with each other, that is, the collection, the analysis and the uploading of the data are coordinated with each other, wherein the collection, the analysis and the uploading of the data by the mobile fog nodes are small-scale data in the sub-area, and the data feature extraction by the cloud server is large-scale data in the whole area.

Images