WO2017000693A1 - Performance synchronization and statistics method for cluster device and system - Google Patents

Abstract

A performance synchronization and statistics method for a cluster device and a system. The method is applied to a first node as a master node in a cluster device, and comprises: collecting counter values of statistical objects on all nodes comprising the first node in a cluster; respectively summarizing a counter value of a statistical object on each node, and generating a statistics record; and synchronizing the statistics record to other nodes in the cluster device except for the first node.

Description

Cluster device performance synchronization statistical method and system Technical field

The present invention relates to, but is not limited to, the field of NAS (Network Attached Storage) storage of the cluster technology, and in particular, to a cluster device performance synchronization statistics method and system.

Background technique

In a traditional multi-device environment, the performance statistics are generally implemented by the network management system actively communicating with the service device agent or by the device agent to obtain the original performance parameter values and processing. This implementation mainly has the following three characteristics. The first is that a central host must be required to run the network management system; the second is that the network management system and the service device agent are implemented in C/S mode; the third is each device. The statistical objects are all irrelevant.

Clustering technology is a relatively new technology that yields relatively high performance in terms of performance, reliability, and flexibility at a lower cost. It is a whole set of independent devices connected by high-speed network. The cluster environment is also a multi-device environment, but the cluster environment has its own uniqueness. Take the NAS storage cluster environment as an example. The following three situations are not It is suitable for the above traditional statistical methods: 1) switching of the running node of the volume statistical object; 2) joining or exiting the cluster, and 3) adding or deleting statistical objects on the node. The traditional network management system cannot dynamically detect changes in the above three cases, and it is impossible to perform normal performance statistics.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

A cluster device performance synchronization statistics method is applied to a first node of a cluster device as a master node, wherein the cluster device performance synchronization statistics method includes:

Collecting counter values of statistical objects on all nodes including the first node in the cluster;

Collecting the counter values of the statistical objects on each of the nodes to generate a statistical record;

The statistical record is synchronized to other nodes in the cluster device than the first node.

Optionally, the method further includes:

Before collecting the counter value of the statistical object on all nodes including the first node in the cluster, when the first node is powered on, a performance statistics database is created, and the default statistical object type is in the database. Create a database table on it;

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

Optionally, the method further includes:

After the counter values of the statistical objects on each of the nodes are respectively summarized to generate a statistical record, the statistical records are stored in the performance statistics database.

Optionally, the method further includes:

Receiving, by the second node that is a slave node in the cluster, data information including a memory capacity of the second node;

Comparing the memory capacity of the second node with the memory capacity of the first node;

If the memory capacity of the second node is smaller than the memory capacity of the first node, deleting part of the statistical records in the performance statistics database.

Optionally, the method further includes:

Receiving a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction;

Determining, by the request parameter carried in the preset request instruction, a first statistical record corresponding to the statistical object that matches the request parameter in the statistical record;

Sending the first statistical record to the sender of the preset request instruction.

Optionally, the method further includes:

Monitor system time changes;

Obtaining the current system time of the first node when the preset acquisition time arrives when the system time change is detected;

Calculating the current system time, the last statistical time, and the preset acquisition time to obtain an operation result;

Comparing the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, determining whether the operation result is less than zero;

If the operation result is less than zero, the statistical time of the statistical record is subtracted from the operation The absolute value of the result obtains a first result, and the first result is used as the latest statistical time of the statistical record; if the operation result is greater than or equal to zero, the statistical time of the statistical record is added to the operation result The absolute value yields a second result, which is used as the most recent statistical time of the statistical record.

Optionally, the method further includes:

Monitoring whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object;

If it is detected that the pre-processing operation exists, acquiring statistical object data stored in the cluster management and index array data of the statistical object;

Comparing the statistical object data with the index array data, and preprocessing the index array data according to the comparison result.

A cluster device performance synchronization statistics system is applied to a first node in a cluster device as a master node, and the system includes:

The first collection module is configured to collect a counter value of a statistical object on all nodes including the first node in the cluster;

The first statistical record generating module is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record;

The first synchronization module is configured to synchronize the statistical record to other nodes in the cluster device than the first node.

Optionally, the statistical system further includes:

The first initialization module is configured to create a performance statistics database and preset the performance when the first node is powered on before collecting the counter value of the statistical object on all nodes including the first node in the cluster. Create a database table on the database for each type of statistical object;

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

Optionally, the first statistical record generating module is further configured to: after collecting the counter values of the statistical objects on each of the nodes respectively, and generating the statistical records, storing the statistical records in the performance statistics. database.

Optionally, the system further includes:

a first receiving module, configured to receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node;

Comparing a module, configured to compare a memory capacity of the second node with a memory capacity of the first node;

The deleting module is configured to delete part of the statistical records in the performance statistics database if the memory capacity of the second node is smaller than the memory capacity of the first node.

Optionally, the system further includes:

The second receiving module is configured to receive a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction;

a locating module, configured to search for a first statistic record corresponding to the statistic object that matches the request parameter, according to the request parameter carried by the preset request instruction;

The information feedback module is configured to send the first statistical record to the sender of the preset request instruction.

Optionally, the system further includes:

The first monitoring module is configured to monitor whether the system time has changed;

The time acquisition module is configured to acquire the current system time of the first node when the preset acquisition time arrives when the system time is changed;

The calculation module is configured to calculate the current system time, the last statistical time, and the preset acquisition time to obtain an operation result;

The determining module is configured to compare the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, further determine whether the operation result is less than zero;

Calculating an evaluation module, if the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the latest of the statistical record Statistics time; if the operation result is greater than or equal to zero, adding the statistical time of the statistical record to the absolute value of the operation result to obtain a second result, and using the second result as the latest statistical time of the statistical record .

The system also includes:

a second monitoring module, configured to monitor whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object;

a statistical object obtaining module, configured to acquire statistical object data stored in the cluster management and index array data of the statistical object if the preprocessing operation is detected;

The comparison processing module is configured to compare the statistical object data with the index array data, and preprocess the index array data according to the comparison result.

A cluster device performance synchronization statistics method is applied to a second node in a cluster device as a slave node, and the method includes:

Obtain a statistical record on the first node and store the statistical record.

Optionally, the method further includes:

Before the step of acquiring the statistical record on the first node and storing the statistical record, when the second node is powered on, a performance statistics database is created, and each type of the statistical object is preset. Create a database table on the database;

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

Optionally, the method further includes:

Detecting whether a change signal changed from the second node to the first node has been generated;

If the change signal has been generated, collecting a counter value of a statistical object on all nodes including the second node that is powered on in the cluster;

Collecting the counter values of the statistical objects on each of the nodes to generate a statistical record;

Synchronizing the statistical record to other nodes in the cluster device than the second node.

A cluster device performance synchronization statistics system is applied to a second node in a cluster device as a slave node, and the system includes:

The obtaining module is configured to acquire a statistical record on the first node and store the statistical record.

Optionally, the system further includes:

a second initialization module, configured to create a performance statistics database when the second node is powered on, and create a database table on the database for each of the preset statistical object types;

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

Optionally, the system further includes:

a detecting module configured to detect whether a change signal changed from the second node to the first node has been generated;

a second collection module, configured to: if the change signal has been generated, collect a counter value of a statistical object on all nodes including the second node that is powered on in the cluster;

a second statistical record generating module is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record;

A second synchronization module is configured to synchronize the statistical record to other nodes in the cluster device than the second node.

A computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the cluster device performance synchronization statistics method described above.

According to the solution of the embodiment of the present invention, the statistical value of the statistical object on each node in the cluster is calculated by the primary node in the cluster, and the statistical record is generated, and the statistical record is synchronized to each slave node in the cluster to ensure the master of the cluster. When the node is powered off or down, the slave nodes in the cluster can continue to complete the work of the master node according to the synchronized statistics records, ensuring that the cluster performance statistics are highly sustainable.

BRIEF abstract

1 is a general flowchart of a method for synchronizing statistical performance of a cluster device applied to a primary node according to an embodiment of the present invention;

2 is a block diagram showing a cluster device performance synchronization statistics system applied to a master node according to an embodiment of the present invention;

3 is a schematic diagram showing interaction between a master device and a cluster device performance synchronization statistics system on a slave node according to an embodiment of the present invention;

4 is a flowchart of processing an abnormal situation 1 of a cluster device performance synchronization statistics system according to an embodiment of the present invention;

FIG. 5 is a flowchart of processing an abnormal situation 2 of a cluster device performance synchronization statistics system according to an embodiment of the present invention;

6 is a flowchart 1 of processing an abnormal situation 3 of a cluster device performance synchronization statistics system according to an embodiment of the present invention;

FIG. 7 is a flowchart 2 of the process for the abnormal situation 3 of the cluster device performance synchronization statistics system according to the embodiment of the present invention;

8 is a flowchart of processing an abnormal situation 4 of a cluster device performance synchronization statistics system according to an embodiment of the present invention;

FIG. 9 is a flowchart of processing an abnormal situation 5 of a cluster device performance synchronization statistics system according to an embodiment of the present invention.

Embodiments of the invention

Embodiments of the present invention will be described below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

The embodiments of the present invention can not dynamically monitor the changes of each node in the cluster when the device statistics are collected by using the traditional statistics in the related clusters, resulting in the problem that the performance of the cluster performance is not high.

As shown in FIG. 1 , an embodiment of the present invention provides a cluster device performance synchronization statistics method, which is applied to a first node that is a master node in a cluster device, and the method includes steps S101 to S103:

S101. Collect a counter value of a statistical object on all nodes including the first node in the cluster.

S102: Perform a summary process on the counter values of the statistical objects on each of the nodes to generate a statistical record.

S103. Synchronize the statistical record to other nodes in the cluster device except the first node.

In the above solution of the embodiment of the present invention, a statistical record is generated by using a primary node in a cluster to collect a counter value of a statistical object on each node in the cluster, and the statistical record is synchronized to each slave node in the cluster, thereby ensuring When the primary node of the cluster is powered off or down, the slave nodes in the cluster can continue to complete the work of the master node according to the synchronization statistics, and maintain the high degree of sustainability of the cluster performance statistics.

It should be noted that, in step S101, the statistical object on each node is pre-defined by the cluster device administrator and stored in the node of the cluster, and each node in the cluster runs with itself when it is running. Collecting data information generated for these statistical objects, and then the node processes the data information, and saves the obtained result in a counter of the node, which should be stated The counter can be regarded as a storage area for storing data information of the statistical object on the node. It should be noted that the process of storing the data information of the statistical object into the storage area after processing the data information is well known to those skilled in the art, and will not be described in detail herein.

Step S102 is a process of collecting a counter value of a statistical object on a primary node of the cluster to generate a statistical record. Optionally, the main implementation manner includes: using a corresponding calculation manner, for example, traffic statistics, according to a counter value of the statistical object. The formula performs the flow counter value/statistical time interval; the calculated cache hit ratio is calculated by the number of hits executed by the formula/(number of hits + number of misses). The counter value is processed into a readable data item according to different calculation methods, and the statistical record of the statistical object is composed of a plurality of such data items.

Step S103 is implemented to synchronize the statistical records to other nodes in the cluster. Alternatively, the implementation manner may include: after reaching a fixed time interval, the primary node sends its own saved statistical records to other slave nodes to implement Sharing with data from the slave node.

It should be noted that the statistical records should be stored one by one according to the type of the statistical object, and in order to conveniently implement the query and management of the data, the database is usually used for data storage, so when the cluster primary node is powered on for the first time, You need to create a database to save statistics records. Optionally, the implementation methods include:

The method further includes: before collecting the counter value of the statistical object on all nodes including the first node in the cluster (ie, before step S101), creating a performance statistics database when the first node is powered on And create a database table on the database for each of the default statistical object types.

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

It should be noted that, according to the created performance statistics database, optionally, the method further includes:

After the counter values of the statistical objects on each of the nodes are respectively summarized to generate a statistical record, the statistical records are stored in the performance statistics database.

It should be noted that a database table is created for each statistical object type according to the statistical object type. When the statistical information is recorded, the statistical records are stored one by one according to the type of the statistical object to the corresponding database table of the performance statistics database. For example, the statistics object is port 1, port 2, virtual disk a, and volume 1. When obtaining the statistics record of each statistical object, port 1 and port are required. The statistical records of 2 are stored in the network port database table, and the statistical records of the virtual disk a are stored in the virtual disk database table, and the statistical records of the volume 1 are stored in the volume database table. The statistical object type and the statistical object may be added or deleted in real time according to the statistical requirements of the cluster device during actual operation. By storing the statistical records of each statistical object according to the statistical object type, it is convenient to manage the statistical information.

The database in the embodiment of the present invention needs to implement sharing on the primary node and the secondary node of the cluster, that is, the same performance statistics database is saved on both the primary node and the secondary node. Therefore, the primary node and the secondary node need to be guaranteed. The method can further include the performance statistics database. Therefore, in the embodiment of the present invention, the method further includes steps S201 to S203:

S201. Receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node.

S202. Compare a memory capacity of the second node with a memory capacity of the first node.

S203. If the memory capacity of the second node is smaller than the memory capacity of the first node, delete a part of the statistical records in the performance statistics database.

It should be noted that when the slave node capacity is greater than or equal to the master node capacity, there is no need to delete the database. When the database must be deleted, optionally, the earliest generated statistical records can be deleted in chronological order, and the most recent latest statistical records are retained.

Because the database occupies memory, you cannot save the data indefinitely. You need to determine the restriction mode. The statistical object of each statistical object type is used to limit the maximum number of statistical records. The maximum number of statistical records is determined according to the storage time. For example, the statistical records of the set statistics objects are set. It needs to be saved for one day, and the collection interval of the counter of the statistical object is 15 seconds. The maximum number of statistical records is 24*60*60/15=5760. Before saving the statistical record, determine whether the number of statistical records of the current statistical object is The maximum value is reached. If the maximum value is reached, the oldest statistical record of the statistical object needs to be deleted. In this case, the new statistical record can be saved to the database, and the new statistical record is synchronized by the database synchronization mechanism to other slave nodes.

The cluster device performance synchronization statistics method in the embodiment of the present invention further includes steps S301 to S303, in order to facilitate the cluster administrator to perform better management on the cluster according to the statistical records.

S301. Receive a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction.

S302. The request parameter carried according to the preset request instruction (it should be noted that the data included in the request parameter may be a statistical object ID, a measurement type, or a time interval), and the statistical record is searched for and described in the statistical record. The first statistical record corresponding to the statistical object whose matching parameters are requested.

S303. Send the first statistical record to a sender of the preset request instruction.

It should be noted that, when the statistical record is viewed according to the statistical object ID, according to the statistical object ID input by the cluster administrator, the element ID in the array of a certain type of statistical object is matched first, and if there is no match, the statistics are directly returned. The object does not exist, or directly returns an error. If it matches, all the statistical records of the statistical object are queried from the database. According to the time interval, for example, 1 hour, the filter obtains multiple statistical records of the last hour, according to the measurement type. Measurement types include traffic, delay, IOPS (Input/Output Operations Per Second), and then extract the required fields of each statistical record (such as the node statistics traffic object measurement type). , as a query response. Because the space for saving the database on the node is limited, some statistical objects in the database may need to be saved for a long time. In this case, some objects in the database are manually exported and stored in other devices, and the exported statistical records are recorded. The request carrying parameter may have a statistical object ID or no statistical object ID. If there is a statistical object ID, according to the statistical object ID, first match with the element ID in the array of a certain type of statistical object, if there is no match, then The direct return statistical object does not exist, or directly returns an error. Otherwise, all statistical records of the statistical object are queried from the database. If the request carries the parameter without the statistical object ID, all statistical records of all statistical objects are queried, and the statistic will be queried. The record is saved to a file, the file is sent to the requester, and the file is deleted.

Because a statistical record corresponds to a unique statistical time, if the initial device system time is incorrect, when the statistics are performed, the date of these errors will be added to the statistical record, and the device system time is corrected later, and the statistical time of the previously generated statistical records is corrected. If the correction is not obtained, the time of the saved statistical record may be inaccurate. Therefore, the cluster device performance synchronization statistical method in the embodiment of the present invention further includes steps S401 to S405:

S401. Whether the monitoring system time changes; the system time change mainly refers to the adjustment of the system time when the cluster administrator finds that the system time is incorrect.

S402. When the system time is changed, when the preset acquisition time arrives, the current system time of the first node is obtained.

S403: Calculate the current system time, the last statistical time, and the preset collection time to obtain an operation result.

S404: Compare an absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, further determine whether the operation result is less than zero.

S405. If the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the latest statistical time of the statistical record; If the operation result is greater than or equal to zero, the statistical time of the statistical record is added to the absolute value of the operation result to obtain a second result, and the second result is used as the latest statistical time of the statistical record.

It should be noted that the above steps are performed only when the system time is adjusted. By correcting the statistical time of the statistical records after correcting the system time, it is ensured that one statistical object only corresponds to one unique statistical record at a time.

Because a large number of statistical objects are stored in the cluster, in order to facilitate the management of the statistical objects, the statistical objects are generally stored in the database, but when the statistical records of the statistical objects are viewed, the statistical objects in the database are retrieved. There will be a problem that the data reading speed is slow and takes a long time. Therefore, the index array of the statistical object needs to be established according to the statistical object in the database, and when the statistical record of the statistical object is viewed, the index array of the statistical object is searched. Corresponding statistical records, this way increases the rate of data read. Whether the deletion or the addition of the statistical object is performed, the database is modified first. In order to ensure that the statistical object in the database is consistent with the indexed array of the statistical object, the cluster device performance synchronization statistical method in the embodiment of the present invention further includes the step S501~ S503:

S501. Monitor whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object;

S502. If the pre-processing operation is detected, acquiring statistical object data stored in the cluster management and index array data of the statistical object;

S503. Compare the statistical object data with the index array data, and preprocess the index array data according to the comparison result.

It should be noted that when the statistical object is added, the statistical object data stored in the database is compared with the indexed array data of the statistical object, and the statistical object that does not exist in the indexed array data is newly added. Object, at this time to increase the statistics object to the number of index arrays According to the data; when the statistical object is deleted, the statistical object data stored in the database is compared with the indexed array data of the statistical object, and some statistical objects in the indexed array data are found in the statistical object data stored in the database. If it does not exist, you need to delete the non-existing statistics object in the index array data.

In the above solution, the master node in the cluster is responsible for collecting the counter information of the node device statistics object from each node of the cluster, processing, storing, and synchronizing to other nodes, and also providing performance statistics query to ensure high reliability of cluster performance statistics. .

As shown in FIG. 2, the embodiment of the present invention further provides a cluster device performance synchronization statistics system 1 applied to a first node as a master node in a cluster device, where the cluster device performance synchronization statistics system includes:

The first collection module 101 is configured to collect a counter value of a statistical object on all nodes including the first node in the cluster;

The first statistical record generating module 102 is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record;

The first synchronization module 103 is configured to synchronize the statistical record to other nodes in the cluster device than the first node.

Optionally, the cluster device performance synchronization statistics system further includes:

The first initialization module 104 is configured to create a performance statistics database and set a preset when the first node is powered on before collecting the counter value of the statistical object on all nodes including the first node in the cluster. Each type of statistical object creates a database table on the database.

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

Optionally, the first statistic record generating module 102 is further configured to: after collecting the counter values of the statistic objects on each of the nodes respectively, and generating the statistic records, storing the statistic records in the performance Statistical database.

Optionally, the cluster device performance synchronization statistics system further includes:

The first receiving module 105 is configured to receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node.

The comparing module 106 is configured to set the memory capacity of the second node to be within the first node The storage capacity is compared.

The deleting module 107 is configured to delete part of the statistical records in the performance statistics database if the memory capacity of the second node is smaller than the memory capacity of the first node.

Optionally, the cluster device performance synchronization statistics system further includes:

The second receiving module 108 is configured to receive a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction.

The searching module 109 is configured to search, according to the request parameter carried by the preset request instruction, a first statistical record corresponding to the statistical object that matches the request parameter in the statistical record.

The information feedback module 110 is configured to send the first statistical record to the sender of the preset request instruction.

Optionally, the cluster device performance synchronization statistics system further includes:

The first monitoring module 111 is configured to monitor whether the system time has changed.

The time acquisition module 112 is configured to acquire the current system time of the first node when the preset acquisition time arrives when the system time is changed.

The calculation module 113 is configured to calculate the current system time, the last statistical time, and the preset acquisition time to obtain an operation result.

The determining module 114 is configured to compare the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, determine whether the operation result is less than zero.

Calculating the evaluation module 115, if the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the statistical record The latest statistical time; if the operation result is greater than or equal to zero, the statistical time of the statistical record is added to the absolute value of the operation result to obtain a second result, and the second result is used as the latest statistics of the statistical record. time.

Optionally, the cluster device performance synchronization statistics system further includes:

The second monitoring module 116 is configured to monitor whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object.

The statistical object obtaining module 117 is configured to acquire the statistical object data stored in the cluster management and the index array data of the statistical object if the preprocessing operation is detected;

The comparison processing module 118 is configured to enter the statistical object data and the index array data into Line alignment, preprocessing the index array data according to the comparison result.

It should be noted that the cluster device performance synchronization statistics system is a system corresponding to the foregoing cluster device performance synchronization statistics method, and all implementation manners of the cluster device performance synchronization statistics method are applicable to the cluster device performance synchronization statistics system, The same technical effect as the above-mentioned cluster device performance synchronization statistical method can be achieved.

The embodiment of the present invention further provides a cluster device performance synchronization statistics method, which is applied to a second node that is a slave node in a cluster device, and the cluster device performance synchronization statistics method includes:

Obtain a statistical record on the first node and store the statistical record.

Optionally, the cluster device performance synchronization statistics method further includes:

Before acquiring the statistical record on the first node and storing the statistical record, when the second node is powered on, a performance statistics database is created, and the default statistical object type is in the database. Create a database table on it.

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

It should be noted that, since the statistical objects corresponding to the slave node and the master node may be different, when the slave node is powered on for the first time, it is also necessary to establish a performance statistics database for its own statistical object, and then, when interacting with the master node, The statistics records obtained from the master node are saved to their own performance statistics database.

It should be noted that, in the running process of the cluster, the master node may be powered off and down. Therefore, in order to ensure that the statistics in the cluster are not interrupted, the cluster device performance synchronization statistical method of the present invention further includes the steps. S601～S604:

S601. Detect whether a change signal that is changed from the second node to the first node has been generated.

S602. If the change signal has been generated, collect a counter value of a statistical object on all nodes including the second node that is powered on in the cluster.

S603. Perform a summary process on the counter values of the statistical objects on each of the nodes to generate a statistical record.

S604. Synchronize the statistical record to other nodes in the cluster device except the second node.

It should be noted that when the main node is powered off or down, one of the slave nodes is selected. The node implements the function of the master node, performs statistical data collection and statistical record generation, and uses this slave node as the master node, and this node can implement all the functions of the above-mentioned master node, ensuring that the cluster performance statistics are highly sustainable.

For the above-mentioned cluster device performance synchronization statistics method, the embodiment of the present invention provides a cluster device performance synchronization statistics system 2, which is applied to a second node as a slave node in the cluster device, and the cluster device performance synchronization statistics system includes:

The obtaining module 201 is configured to acquire a statistical record on the first node, and store the statistical record.

Optionally, the cluster device performance synchronization statistics system further includes:

The second initialization module 202 is configured to create a performance statistics database when the second node is powered on, and create a database table on the database for each of the preset statistical object types.

The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.

Optionally, the cluster device performance synchronization statistics system further includes:

The detecting module 204 is configured to detect whether a change signal that is changed from the second node to the first node is generated;

The second collection module 205 is configured to: if the change signal has been generated, collect a counter value of a statistical object on all nodes including the second node that is powered on in the cluster;

The second statistical record generating module 206 is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record;

The second synchronization module 207 is configured to synchronize the statistical record to other nodes in the cluster device than the second node.

It should be noted that the cluster device performance synchronization statistics system is a system corresponding to the foregoing cluster device performance synchronization statistics method, and all implementation manners of the cluster device performance synchronization statistics method are applicable to the cluster device performance synchronization statistics system, The same technical effect as the above-mentioned cluster device performance synchronization statistical method can be achieved.

With reference to the foregoing embodiment and the actual usage, the clustering device performance synchronization statistics system is illustrated as follows:

The cluster device performance synchronization statistics system can be divided into: power-on initialization unit according to functions 11. The original counter value acquisition unit 12, the performance statistics data generation unit 13, the master-slave node interaction unit 14, the takeover performance statistics unit 15, and the query and export statistics unit 16. It should be noted that the cluster device performance synchronization statistics system exists on each node of the cluster, but in the actual application, the working unit of the cluster device performance synchronization statistics system on the master node is the power-on initialization unit 11, The original counter value collection unit 12, the performance statistics data generation unit 13, the master-slave node interaction unit 14, and the query and export statistics unit 16, the takeover performance statistics unit 15 is in an inoperative state; and the cluster device performance on the master node The working units of the synchronous statistical system are the power-on initialization unit 11, the master-slave node interaction unit 14, and the takeover performance statistics unit 15, the original counter value collection unit 12, the performance statistics data generation unit 13, and the query and export statistics unit 16 are not working. State; when the master node is powered off or down, one of the slave nodes acts instead of the master node, at this time the original counter value collection unit 12, the performance statistics data generating unit 13 on the slave node, and the query and export statistics The unit 16 is changed from the inoperative state to the working state, and the takeover performance statistical unit 15 is operated. Switch inoperative state; and operating units from the primary node of the node, as shown in FIG. 3, FIG. 3 the master node and the unit is inoperative from the node, not shown.

The main functions of the above units are:

The power-on initialization unit 11: runs when all nodes of the cluster are powered on, is responsible for creating a performance statistics database, creating a database table on the database for each statistical object type, and synchronizing database data from the node (synchronous performance statistics database synchronization request is initiated from the node, The data of each statistical object type table of the performance statistics database on the master node is synchronized to the corresponding table of the database on the slave node, and the data changes of the database on the master node are synchronously incremented to the slave node database), and the registration node status change notification is The takeover performance statistics unit 15) is notified once the node status changes.

The original counter value collection unit 12: runs on the cluster master node, enables the data collection timer, and the collection interval can be configured as needed, for example, 15 seconds. When the timer message arrives, the cluster node list information is obtained through cluster management. The address information of the node in the cluster node list is sent to each service subsystem of the node to obtain the original counter value of the corresponding statistical object and the two statistical time intervals before and after, and the original counter value of each statistical object obtained is obtained. And the statistics interval is sent to the performance statistics generating unit 13; the need to explain here is that the business subsystem should clear the original counter value of the statistical object after processing the request.

Performance statistics generating unit 13: running on the cluster master node, each cluster to be collected The raw counter value of the statistical object of the node is summarized and processed (using the corresponding calculation method, such as traffic statistics, the execution flow original counter value/statistical time interval is obtained; and the cache hit rate is calculated, the number of hits is executed/(the number of hits + the number of losses) Calculated, according to different calculation methods, the original counter value is processed into a readable data item, the statistical object statistical record is composed of a plurality of such data items, the statistical object statistical record needs to be stored in the database), and the readability is readability for the user. A strong statistical record, a statistical object corresponds to a statistical record at a statistical time point, the statistical record is stored in the database, and then the database synchronization mechanism synchronizes the statistical record to all cluster slave nodes as a backup; meanwhile, the in-memory database occupies memory, cannot To save data indefinitely, you need to determine the restriction method. The statistical object of each statistical object type is used to limit the maximum number of statistical records. The maximum number of statistical records is determined according to how long the storage time is. Before the statistical record is saved, the statistical record bar of the current statistical object is determined. Whether the number is up to To the maximum value, if the maximum value is reached, the oldest statistical record of the statistical object needs to be deleted, and then the new statistical record can be saved to the database, and the new statistical record is synchronized by the database synchronization mechanism to other slave nodes.

The master-slave node interaction unit 14: runs when the cluster slave node is powered on, and informs the cluster master node of the memory size of the own node device, and the master node receives the request, and determines the data that the database needs to retain according to the request parameter.

Takeover performance statistics unit 15: Because the cluster performance statistics are performed on the primary node, if the primary node is abnormal, performance statistics cannot be performed normally; when the cluster primary node is down due to human restart or unknown reasons, the cluster management is in the cluster secondary node. A change is performed in the takeover performance statistics unit 15 of the cluster slave node that is newly elected as the cluster master node, and the performance statistics task is taken over, and all statistical objects are read from the database and saved to various In the type statistics object index array, the notification service subsystem clears each original counter value, and enables the timer to start cluster device performance statistics.

The query and export statistics unit 16: receives the query request sent by the user, returns the corresponding statistical record of all the statistical objects according to the parameters carried by the request, and the client uses the statistical records to draw a dynamic statistical graph; and receives the export request sent by the user. According to the request parameter, the corresponding statistical data record of the selected statistical object is returned in the form of a file, and the user selects to save the file to the local directory to achieve the purpose of saving the overall view or the historical statistical record.

It should be noted that the cluster device performance synchronization statistics system also needs the following abnormal conditions. Processing:

Case 1:

The original counter value is the value of the variable. If the system runs for a long time, it will inevitably overflow, and since the request is not successful every time, if there is a failure, the performance statistics generating unit does not know the specific statistical time interval, which will inevitably lead to calculation. Not precise enough.

The processing method for case one is steps S701 to S706 (shown in FIG. 4):

S701. The original counter value collecting unit 12 first requests the service subsystem to obtain the original counter value of the statistical object.

S702. The service subsystem records the time of the request statistics, returns the original counter value, and clears the original counter value.

S703. The original counter value collection unit 12 again requests the service subsystem to obtain the original counter value.

S704: The service module is very busy, and the request is not responded in time, and the request times out.

S705. The original counter value collection unit 12 continues to request the service module to obtain the original counter value.

S706: In response to the request, calculate a current system time, the time is subtracted from the statistical time recorded in step S702, as a statistical time interval between two points, save the system time as the statistical time of the request, and return the original counter value. The statistical interval is given to the original counter value acquisition unit 12, and the original counter value is cleared.

Case 2:

A statistical record corresponds to a unique statistical time. If the initial device system time is incorrect, the date of these errors will be added to the statistical record during the statistics. The device system time is corrected later. The statistical time of the previously generated statistical records is not available. Get corrections.

The processing method for case 2 is steps S801 to S803 (as shown in FIG. 5):

S801, the system time is abnormal, and the performance statistical data generating unit 13 saves the statistical record to the database, and records the system time as the latest statistical time.

S802, the user corrects the system time.

S803: The statistic timer arrives, the timer period is 15 seconds, and the current system time is obtained, and a first time value is subtracted. The first time value=the statistical time of the last statistical system is +15, and then the absolute value of the difference is taken. If the absolute value is greater than a certain number, the configuration here is 8 seconds, indicating that the system time is positive. In the correction, at this time, it is determined that the current system time minus the first time value, that is, the statistical time of the latest statistical system +15) is less than 0, indicating that the time is changed back, and the timestamp field values of all statistical records are subtracted from the previous calculation. The absolute value is saved. If it is determined that the current system time minus the first time value, that is, the statistical time of the last statistical system +15) is greater than or equal to 0, the absolute value is saved, and the current system time is finally recorded as the latest statistics. time.

Case 3:

Statistical objects are dynamically added and deleted at any time. If the statistical object increases, it is obviously not possible to query the statistical information of the statistical object. You need to increase the statistical information of the statistical object. Similarly, if a statistical object is deleted, all its statistics need to be deleted.

The treatment for Case 3 is:

1. Add statistical objects, including steps S901 to S903 (as shown in Figure 6):

S901, the user added a statistical object.

S902. The performance statistics data generating unit 13 acquires all statistical objects of the current cluster.

S903. The statistical object obtained in the previous step is sequentially queried in the index array of the statistical object. If the query is not found, the statistical object is added, and a new statistical object is added in the index array of the statistical object, and the statistical record of the new statistical object is calculated and saved. Go to the database.

2. Delete the statistical object, including steps S1001 to S1003 (as shown in Figure 7).

S1001, the user deletes the statistical object.

S1002: The performance statistical data generating unit 13 acquires a current cluster statistical object.

S1003: Each statistical object saved in the statistical object index array is queried according to the ID of the cluster statistical object obtained in the previous step. If the query is not found, the statistical object has been deleted. The statistical object index array needs to delete the statistical object and delete all statistical records of the statistical object from the database.

Situation 4:

Some statistical objects belong to the entire cluster, and others belong to a single node. For statistical objects belonging to a single node, as long as the cluster is not exited, it is still possible to run with the node. Therefore, it is necessary to ensure that the original statistical records are not deleted and the nodes are not running. Statistics also need to be calculated during this time, and the statistical values need to be presented in some way.

The processing method for case 4 includes steps S1101 to S1104 (shown in FIG. 8):

S1101: A node is powered off.

S1102: Obtain cluster node list information, and obtain a node that is powered off according to the state of the node.

S1103: When all statistical objects of the power-off node are obtained, and performance statistics are performed, the data field values of the statistical records of the statistical object are all set to zero.

S1104: Save the statistical record of the statistical object to the database.

Case 5:

The original counter value collection unit initiates the acquisition request. Since the service module is busy and cannot process the request, the timing collection performance statistics unit cannot wait for a long time. Therefore, the acquisition of the original counter value is not successful every time, and must be measured in some manner. Let the user feel that there is no problem with the system.

The processing manner for case 5 includes steps S1201 to S1204 (as shown in FIG. 9):

S1201, the original counter value collecting unit 12 requests the service subsystem to obtain the original counter value.

S1202: The service subsystem does not respond in time, and the request is overdue.

S1203: The data field of the statistical record of the current statistical object is assigned by the statistical record number field of the object last counted, and the new statistical record only modifies the statistical time.

S1204. Save the new statistical record to the database.

The main features of the cluster device performance synchronization statistics system in the embodiment of the present invention are as follows: 1) The master node in the cluster is responsible for collecting, storing, synchronizing the raw counter information of the node device statistics object from each node of the cluster, and providing performance statistics query. In fact, the primary node is equivalent to the central host. When the current primary node fails, the performance statistics will be forwarded to the newly elected primary node to maintain high performance statistics; 2) performance statistics have been generated and saved in the cluster. On the node, data statistics can be obtained through B/S mode and C/S mode; 3) node join or exit occurs in the cluster, the volume statistics object switches the running node, the user increases and deletes the statistical object, etc., and can dynamically detect Measured and evaluated for performance.

One of ordinary skill in the art will appreciate that all or a portion of the steps of the above-described embodiments can be implemented using a computer program flow, which can be stored in a computer readable storage medium, such as on a corresponding hardware platform (eg, The system, device, device, device, etc. are executed, and when executed, include one or a combination of the steps of the method embodiments.

Alternatively, all or part of the steps of the above embodiments may also be implemented using an integrated circuit. The steps may be separately fabricated into individual integrated circuit modules, or a plurality of modules or steps may be fabricated into a single integrated circuit module.

The devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.

When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. The above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Industrial applicability

According to the solution of the embodiment of the present invention, the statistical value of the statistical object on each node in the cluster is calculated by the primary node in the cluster, and the statistical record is generated, and the statistical record is synchronized to each slave node in the cluster to ensure the master of the cluster. When the node is powered off or down, the slave nodes in the cluster can continue to complete the work of the master node according to the synchronized statistics records, ensuring that the cluster performance statistics are highly sustainable.

Claims (21)

A cluster device performance synchronization statistics method is applied to a first node of a cluster device as a master node, and the cluster device performance synchronization statistics method includes: Collecting counter values of statistical objects on all nodes including the first node in the cluster; Collecting the counter values of the statistical objects on each of the nodes to generate a statistical record; The statistical record is synchronized to other nodes in the cluster device than the first node. The cluster device performance synchronization statistics method according to claim 1, further comprising: Before collecting the counter value of the statistical object on all nodes including the first node in the cluster, when the first node is powered on, a performance statistics database is created, and the default statistical object type is in the database. Create a database table on it; The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number. The cluster device performance synchronization statistics method according to claim 2, the method further comprising: After the counter values of the statistical objects on each of the nodes are respectively summarized to generate a statistical record, the statistical records are stored in the performance statistics database. The cluster device performance synchronization statistics method according to claim 3, the method further comprising: Receiving, by the second node that is a slave node in the cluster, data information including a memory capacity of the second node; Comparing the memory capacity of the second node with the memory capacity of the first node; If the memory capacity of the second node is smaller than the memory capacity of the first node, deleting part of the statistical records in the performance statistics database. The cluster device performance synchronization statistics method according to claim 1, wherein the method further comprises: Receiving a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction; Determining, by the request parameter carried in the preset request instruction, a first statistical record corresponding to the statistical object that matches the request parameter in the statistical record; Sending the first statistical record to the sender of the preset request instruction. The cluster device performance synchronization statistics method according to claim 1, wherein the method further comprises: Monitor system time changes; Obtaining the current system time of the first node when the preset acquisition time arrives when the system time change is detected; Calculating the current system time, the last statistical time, and the preset acquisition time to obtain an operation result; Comparing the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, determining whether the operation result is less than zero; If the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the latest statistical time of the statistical record; If the operation result is greater than or equal to zero, the statistical time of the statistical record is added to the absolute value of the operation result to obtain a second result, and the second result is used as the latest statistical time of the statistical record. The cluster device performance synchronization statistics method according to claim 1, further comprising: Monitoring whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object; If it is detected that the pre-processing operation exists, acquiring statistical object data stored in the cluster management and index array data of the statistical object; Comparing the statistical object data with the index array data, and preprocessing the index array data according to the comparison result. A cluster device performance synchronization statistics system is applied to a first node of a cluster device as a master node, and the cluster device performance synchronization statistics system includes: The first collection module is configured to collect a counter value of a statistical object on all nodes including the first node in the cluster; The first statistical record generating module is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record; a first synchronization module, configured to synchronize the statistical record to the first node in the cluster device Other nodes outside. The cluster device performance synchronization statistics system of claim 8, the system further comprising: The first initialization module is configured to create a performance statistics database and preset the performance when the first node is powered on before collecting the counter value of the statistical object on all nodes including the first node in the cluster. Create a database table on the database for each type of statistical object; The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number. The cluster device performance synchronization statistics system according to claim 9, wherein the first statistical record generation module is further configured to: after separately collecting counter values of statistical objects on each of the nodes, and generating a statistical record, The statistical record is stored in the performance statistics database. The cluster device performance synchronization statistics system of claim 10, the system further comprising: a first receiving module, configured to receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node; Comparing a module, configured to compare a memory capacity of the second node with a memory capacity of the first node; The deleting module is configured to delete part of the statistical records in the performance statistics database if the memory capacity of the second node is smaller than the memory capacity of the first node. The cluster device performance synchronization statistics system of claim 8, the system further comprising: a second receiving module, configured to receive a preset request instruction, wherein the preset request instruction is a query request instruction or an export request instruction; a locating module, configured to search for a first statistic record corresponding to the statistic object that matches the request parameter, according to the request parameter carried by the preset request instruction; The information feedback module is configured to send the first statistical record to the sender of the preset request instruction. The cluster device performance synchronization statistics system of claim 8, the system further comprising: The first monitoring module is configured to monitor whether the system time has changed; Time acquisition module, set to preset acquisition time when the system time is detected to change Upon arrival, obtaining the current system time of the first node; The calculation module is configured to calculate the current system time, the last statistical time, and the preset acquisition time to obtain an operation result; The determining module is configured to compare the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, determine whether the operation result is less than zero; Calculating an evaluation module, if the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the latest of the statistical record Statistics time; if the operation result is greater than or equal to zero, adding the statistical time of the statistical record to the absolute value of the operation result to obtain a second result, and using the second result as the latest statistical time of the statistical record . The cluster device performance synchronization statistics system of claim 8, the system further comprising: a second monitoring module, configured to monitor whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object; a statistical object obtaining module, configured to acquire statistical object data stored in the cluster management and index array data of the statistical object if the preprocessing operation is detected; The comparison processing module is configured to compare the statistical object data with the index array data, and preprocess the index array data according to the comparison result. A cluster device performance synchronization statistics method is applied to a second node in a cluster device as a slave node, and the method includes: Obtain a statistical record on the first node and store the statistical record. The cluster device performance synchronization statistics method according to claim 15, the method further comprising: Before obtaining the statistical record on the first node and storing the statistical record, when the second node is powered on, a performance statistics database is created, and the default statistical object type is created on the database. Database Table; The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number. The cluster device performance synchronization statistics method according to claim 15, wherein the method further comprises include: Detecting whether a change signal changed from the second node to the first node has been generated; If the change signal has been generated, collecting a counter value of a statistical object on all nodes including the second node that is powered on in the cluster; Collecting the counter values of the statistical objects on each of the nodes to generate a statistical record; Synchronizing the statistical record to other nodes in the cluster device than the second node. A cluster device performance synchronization statistics system is applied to a second node in a cluster device as a slave node, and the system includes: The obtaining module is configured to acquire a statistical record on the first node and store the statistical record. The cluster device performance synchronization statistics system of claim 18, the system further comprising: a second initialization module, configured to create a performance statistics database when the second node is powered on, and create a database table on the database for each preset statistical object type; The statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number. The cluster device performance synchronization statistics system of claim 18, the system further comprising: a detecting module configured to detect whether a change signal changed from the second node to the first node has been generated; a second collection module, configured to: if the change signal has been generated, collect a counter value of a statistical object on all nodes including the second node that is powered on in the cluster; a second statistical record generating module is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record; A second synchronization module is configured to synchronize the statistical record to other nodes in the cluster device than the second node. A computer readable storage medium storing computer executable instructions, the computer The cluster device performance synchronization statistics method described above is implemented when the executable instructions are executed by the processor.

Images