[go: up one dir, main page]

WO2017000693A1 - Procédé de synchronisation de performance et de génération de statistiques pour un dispositif de grappe et un système - Google Patents

Procédé de synchronisation de performance et de génération de statistiques pour un dispositif de grappe et un système Download PDF

Info

Publication number
WO2017000693A1
WO2017000693A1 PCT/CN2016/082350 CN2016082350W WO2017000693A1 WO 2017000693 A1 WO2017000693 A1 WO 2017000693A1 CN 2016082350 W CN2016082350 W CN 2016082350W WO 2017000693 A1 WO2017000693 A1 WO 2017000693A1
Authority
WO
WIPO (PCT)
Prior art keywords
statistical
node
cluster
record
statistics
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/082350
Other languages
English (en)
Chinese (zh)
Inventor
许显月
李明杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Publication of WO2017000693A1 publication Critical patent/WO2017000693A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the present invention relates to, but is not limited to, the field of NAS (Network Attached Storage) storage of the cluster technology, and in particular, to a cluster device performance synchronization statistics method and system.
  • NAS Network Attached Storage
  • the performance statistics are generally implemented by the network management system actively communicating with the service device agent or by the device agent to obtain the original performance parameter values and processing.
  • This implementation mainly has the following three characteristics. The first is that a central host must be required to run the network management system; the second is that the network management system and the service device agent are implemented in C/S mode; the third is each device. The statistical objects are all irrelevant.
  • Clustering technology is a relatively new technology that yields relatively high performance in terms of performance, reliability, and flexibility at a lower cost. It is a whole set of independent devices connected by high-speed network.
  • the cluster environment is also a multi-device environment, but the cluster environment has its own uniqueness. Take the NAS storage cluster environment as an example. The following three situations are not It is suitable for the above traditional statistical methods: 1) switching of the running node of the volume statistical object; 2) joining or exiting the cluster, and 3) adding or deleting statistical objects on the node.
  • the traditional network management system cannot dynamically detect changes in the above three cases, and it is impossible to perform normal performance statistics.
  • a cluster device performance synchronization statistics method is applied to a first node of a cluster device as a master node, wherein the cluster device performance synchronization statistics method includes:
  • the statistical record is synchronized to other nodes in the cluster device than the first node.
  • the method further includes:
  • a performance statistics database is created, and the default statistical object type is in the database. Create a database table on it;
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the method further includes:
  • the statistical records are stored in the performance statistics database.
  • the method further includes:
  • the memory capacity of the second node is smaller than the memory capacity of the first node, deleting part of the statistical records in the performance statistics database.
  • the method further includes:
  • the method further includes:
  • the statistical time of the statistical record is subtracted from the operation
  • the absolute value of the result obtains a first result, and the first result is used as the latest statistical time of the statistical record; if the operation result is greater than or equal to zero, the statistical time of the statistical record is added to the operation result
  • the absolute value yields a second result, which is used as the most recent statistical time of the statistical record.
  • the method further includes:
  • the pre-processing operation is an adding operation or a deleting operation of the statistical object
  • a cluster device performance synchronization statistics system is applied to a first node in a cluster device as a master node, and the system includes:
  • the first collection module is configured to collect a counter value of a statistical object on all nodes including the first node in the cluster;
  • the first statistical record generating module is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record
  • the first synchronization module is configured to synchronize the statistical record to other nodes in the cluster device than the first node.
  • the statistical system further includes:
  • the first initialization module is configured to create a performance statistics database and preset the performance when the first node is powered on before collecting the counter value of the statistical object on all nodes including the first node in the cluster. Create a database table on the database for each type of statistical object;
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the first statistical record generating module is further configured to: after collecting the counter values of the statistical objects on each of the nodes respectively, and generating the statistical records, storing the statistical records in the performance statistics. database.
  • system further includes:
  • a first receiving module configured to receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node
  • Comparing a module configured to compare a memory capacity of the second node with a memory capacity of the first node
  • the deleting module is configured to delete part of the statistical records in the performance statistics database if the memory capacity of the second node is smaller than the memory capacity of the first node.
  • system further includes:
  • the second receiving module is configured to receive a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction;
  • a locating module configured to search for a first statistic record corresponding to the statistic object that matches the request parameter, according to the request parameter carried by the preset request instruction;
  • the information feedback module is configured to send the first statistical record to the sender of the preset request instruction.
  • system further includes:
  • the first monitoring module is configured to monitor whether the system time has changed
  • the time acquisition module is configured to acquire the current system time of the first node when the preset acquisition time arrives when the system time is changed;
  • the calculation module is configured to calculate the current system time, the last statistical time, and the preset acquisition time to obtain an operation result
  • the determining module is configured to compare the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, further determine whether the operation result is less than zero;
  • Calculating an evaluation module if the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the latest of the statistical record Statistics time; if the operation result is greater than or equal to zero, adding the statistical time of the statistical record to the absolute value of the operation result to obtain a second result, and using the second result as the latest statistical time of the statistical record .
  • the system also includes:
  • a second monitoring module configured to monitor whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object;
  • a statistical object obtaining module configured to acquire statistical object data stored in the cluster management and index array data of the statistical object if the preprocessing operation is detected;
  • the comparison processing module is configured to compare the statistical object data with the index array data, and preprocess the index array data according to the comparison result.
  • a cluster device performance synchronization statistics method is applied to a second node in a cluster device as a slave node, and the method includes:
  • the method further includes:
  • a performance statistics database is created, and each type of the statistical object is preset. Create a database table on the database;
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the method further includes:
  • a cluster device performance synchronization statistics system is applied to a second node in a cluster device as a slave node, and the system includes:
  • the obtaining module is configured to acquire a statistical record on the first node and store the statistical record.
  • system further includes:
  • a second initialization module configured to create a performance statistics database when the second node is powered on, and create a database table on the database for each of the preset statistical object types
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • system further includes:
  • a detecting module configured to detect whether a change signal changed from the second node to the first node has been generated
  • a second collection module configured to: if the change signal has been generated, collect a counter value of a statistical object on all nodes including the second node that is powered on in the cluster;
  • a second statistical record generating module is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record
  • a second synchronization module is configured to synchronize the statistical record to other nodes in the cluster device than the second node.
  • a computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the cluster device performance synchronization statistics method described above.
  • the statistical value of the statistical object on each node in the cluster is calculated by the primary node in the cluster, and the statistical record is generated, and the statistical record is synchronized to each slave node in the cluster to ensure the master of the cluster.
  • the slave nodes in the cluster can continue to complete the work of the master node according to the synchronized statistics records, ensuring that the cluster performance statistics are highly sustainable.
  • FIG. 1 is a general flowchart of a method for synchronizing statistical performance of a cluster device applied to a primary node according to an embodiment of the present invention
  • FIG. 2 is a block diagram showing a cluster device performance synchronization statistics system applied to a master node according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram showing interaction between a master device and a cluster device performance synchronization statistics system on a slave node according to an embodiment of the present invention
  • FIG. 4 is a flowchart of processing an abnormal situation 1 of a cluster device performance synchronization statistics system according to an embodiment of the present invention
  • FIG. 5 is a flowchart of processing an abnormal situation 2 of a cluster device performance synchronization statistics system according to an embodiment of the present invention
  • FIG. 6 is a flowchart 1 of processing an abnormal situation 3 of a cluster device performance synchronization statistics system according to an embodiment of the present invention
  • FIG. 7 is a flowchart 2 of the process for the abnormal situation 3 of the cluster device performance synchronization statistics system according to the embodiment of the present invention.
  • FIG. 8 is a flowchart of processing an abnormal situation 4 of a cluster device performance synchronization statistics system according to an embodiment of the present invention
  • FIG. 9 is a flowchart of processing an abnormal situation 5 of a cluster device performance synchronization statistics system according to an embodiment of the present invention.
  • the embodiments of the present invention can not dynamically monitor the changes of each node in the cluster when the device statistics are collected by using the traditional statistics in the related clusters, resulting in the problem that the performance of the cluster performance is not high.
  • an embodiment of the present invention provides a cluster device performance synchronization statistics method, which is applied to a first node that is a master node in a cluster device, and the method includes steps S101 to S103:
  • S101 Collect a counter value of a statistical object on all nodes including the first node in the cluster.
  • S102 Perform a summary process on the counter values of the statistical objects on each of the nodes to generate a statistical record.
  • a statistical record is generated by using a primary node in a cluster to collect a counter value of a statistical object on each node in the cluster, and the statistical record is synchronized to each slave node in the cluster, thereby ensuring When the primary node of the cluster is powered off or down, the slave nodes in the cluster can continue to complete the work of the master node according to the synchronization statistics, and maintain the high degree of sustainability of the cluster performance statistics.
  • step S101 the statistical object on each node is pre-defined by the cluster device administrator and stored in the node of the cluster, and each node in the cluster runs with itself when it is running. Collecting data information generated for these statistical objects, and then the node processes the data information, and saves the obtained result in a counter of the node, which should be stated
  • the counter can be regarded as a storage area for storing data information of the statistical object on the node. It should be noted that the process of storing the data information of the statistical object into the storage area after processing the data information is well known to those skilled in the art, and will not be described in detail herein.
  • Step S102 is a process of collecting a counter value of a statistical object on a primary node of the cluster to generate a statistical record.
  • the main implementation manner includes: using a corresponding calculation manner, for example, traffic statistics, according to a counter value of the statistical object.
  • the formula performs the flow counter value/statistical time interval; the calculated cache hit ratio is calculated by the number of hits executed by the formula/(number of hits + number of misses).
  • the counter value is processed into a readable data item according to different calculation methods, and the statistical record of the statistical object is composed of a plurality of such data items.
  • Step S103 is implemented to synchronize the statistical records to other nodes in the cluster.
  • the implementation manner may include: after reaching a fixed time interval, the primary node sends its own saved statistical records to other slave nodes to implement Sharing with data from the slave node.
  • the statistical records should be stored one by one according to the type of the statistical object, and in order to conveniently implement the query and management of the data, the database is usually used for data storage, so when the cluster primary node is powered on for the first time, You need to create a database to save statistics records.
  • the implementation methods include:
  • the method further includes: before collecting the counter value of the statistical object on all nodes including the first node in the cluster (ie, before step S101), creating a performance statistics database when the first node is powered on And create a database table on the database for each of the default statistical object types.
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the method further includes:
  • the statistical records are stored in the performance statistics database.
  • a database table is created for each statistical object type according to the statistical object type.
  • the statistical records are stored one by one according to the type of the statistical object to the corresponding database table of the performance statistics database.
  • the statistics object is port 1, port 2, virtual disk a, and volume 1.
  • port 1 and port are required.
  • the statistical records of 2 are stored in the network port database table, and the statistical records of the virtual disk a are stored in the virtual disk database table, and the statistical records of the volume 1 are stored in the volume database table.
  • the statistical object type and the statistical object may be added or deleted in real time according to the statistical requirements of the cluster device during actual operation.
  • the database in the embodiment of the present invention needs to implement sharing on the primary node and the secondary node of the cluster, that is, the same performance statistics database is saved on both the primary node and the secondary node. Therefore, the primary node and the secondary node need to be guaranteed.
  • the method can further include the performance statistics database. Therefore, in the embodiment of the present invention, the method further includes steps S201 to S203:
  • S201 Receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node.
  • slave node capacity is greater than or equal to the master node capacity, there is no need to delete the database.
  • the database must be deleted, optionally, the earliest generated statistical records can be deleted in chronological order, and the most recent latest statistical records are retained.
  • the statistical object of each statistical object type is used to limit the maximum number of statistical records.
  • determine whether the number of statistical records of the current statistical object is The maximum value is reached. If the maximum value is reached, the oldest statistical record of the statistical object needs to be deleted. In this case, the new statistical record can be saved to the database, and the new statistical record is synchronized by the database synchronization mechanism to other slave nodes.
  • the cluster device performance synchronization statistics method in the embodiment of the present invention further includes steps S301 to S303, in order to facilitate the cluster administrator to perform better management on the cluster according to the statistical records.
  • S301 Receive a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction.
  • the request parameter carried according to the preset request instruction (it should be noted that the data included in the request parameter may be a statistical object ID, a measurement type, or a time interval), and the statistical record is searched for and described in the statistical record.
  • the filter obtains multiple statistical records of the last hour, according to the measurement type. Measurement types include traffic, delay, IOPS (Input/Output Operations Per Second), and then extract the required fields of each statistical record (such as the node statistics traffic object measurement type). , as a query response.
  • the request carrying parameter may have a statistical object ID or no statistical object ID. If there is a statistical object ID, according to the statistical object ID, first match with the element ID in the array of a certain type of statistical object, if there is no match, then The direct return statistical object does not exist, or directly returns an error. Otherwise, all statistical records of the statistical object are queried from the database. If the request carries the parameter without the statistical object ID, all statistical records of all statistical objects are queried, and the statistic will be queried. The record is saved to a file, the file is sent to the requester, and the file is deleted.
  • the cluster device performance synchronization statistical method in the embodiment of the present invention further includes steps S401 to S405:
  • the system time change mainly refers to the adjustment of the system time when the cluster administrator finds that the system time is incorrect.
  • S403 Calculate the current system time, the last statistical time, and the preset collection time to obtain an operation result.
  • S404 Compare an absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, further determine whether the operation result is less than zero.
  • the cluster device performance synchronization statistical method in the embodiment of the present invention further includes the step S501 ⁇ S503:
  • the statistical object data stored in the database is compared with the indexed array data of the statistical object, and the statistical object that does not exist in the indexed array data is newly added.
  • Object at this time to increase the statistics object to the number of index arrays According to the data; when the statistical object is deleted, the statistical object data stored in the database is compared with the indexed array data of the statistical object, and some statistical objects in the indexed array data are found in the statistical object data stored in the database. If it does not exist, you need to delete the non-existing statistics object in the index array data.
  • the master node in the cluster is responsible for collecting the counter information of the node device statistics object from each node of the cluster, processing, storing, and synchronizing to other nodes, and also providing performance statistics query to ensure high reliability of cluster performance statistics. .
  • the embodiment of the present invention further provides a cluster device performance synchronization statistics system 1 applied to a first node as a master node in a cluster device, where the cluster device performance synchronization statistics system includes:
  • the first collection module 101 is configured to collect a counter value of a statistical object on all nodes including the first node in the cluster;
  • the first statistical record generating module 102 is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record
  • the first synchronization module 103 is configured to synchronize the statistical record to other nodes in the cluster device than the first node.
  • the cluster device performance synchronization statistics system further includes:
  • the first initialization module 104 is configured to create a performance statistics database and set a preset when the first node is powered on before collecting the counter value of the statistical object on all nodes including the first node in the cluster.
  • Each type of statistical object creates a database table on the database.
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the first statistic record generating module 102 is further configured to: after collecting the counter values of the statistic objects on each of the nodes respectively, and generating the statistic records, storing the statistic records in the performance Statistical database.
  • the cluster device performance synchronization statistics system further includes:
  • the first receiving module 105 is configured to receive, by the second node that is a slave node in the cluster, data information that includes a memory capacity of the second node.
  • the comparing module 106 is configured to set the memory capacity of the second node to be within the first node The storage capacity is compared.
  • the deleting module 107 is configured to delete part of the statistical records in the performance statistics database if the memory capacity of the second node is smaller than the memory capacity of the first node.
  • the cluster device performance synchronization statistics system further includes:
  • the second receiving module 108 is configured to receive a preset request instruction, where the preset request instruction is a query request instruction or an export request instruction.
  • the searching module 109 is configured to search, according to the request parameter carried by the preset request instruction, a first statistical record corresponding to the statistical object that matches the request parameter in the statistical record.
  • the information feedback module 110 is configured to send the first statistical record to the sender of the preset request instruction.
  • the cluster device performance synchronization statistics system further includes:
  • the first monitoring module 111 is configured to monitor whether the system time has changed.
  • the time acquisition module 112 is configured to acquire the current system time of the first node when the preset acquisition time arrives when the system time is changed.
  • the calculation module 113 is configured to calculate the current system time, the last statistical time, and the preset acquisition time to obtain an operation result.
  • the determining module 114 is configured to compare the absolute value of the operation result with a preset value, and if the absolute value is greater than the preset value, determine whether the operation result is less than zero.
  • Calculating the evaluation module 115 if the operation result is less than zero, subtracting the statistical value of the statistical record from the absolute value of the operation result to obtain a first result, and using the first result as the statistical record The latest statistical time; if the operation result is greater than or equal to zero, the statistical time of the statistical record is added to the absolute value of the operation result to obtain a second result, and the second result is used as the latest statistics of the statistical record. time.
  • the cluster device performance synchronization statistics system further includes:
  • the second monitoring module 116 is configured to monitor whether there is a pre-processing operation on the statistical object data stored in the cluster management, where the pre-processing operation is an adding operation or a deleting operation of the statistical object.
  • the statistical object obtaining module 117 is configured to acquire the statistical object data stored in the cluster management and the index array data of the statistical object if the preprocessing operation is detected;
  • the comparison processing module 118 is configured to enter the statistical object data and the index array data into Line alignment, preprocessing the index array data according to the comparison result.
  • cluster device performance synchronization statistics system is a system corresponding to the foregoing cluster device performance synchronization statistics method, and all implementation manners of the cluster device performance synchronization statistics method are applicable to the cluster device performance synchronization statistics system, The same technical effect as the above-mentioned cluster device performance synchronization statistical method can be achieved.
  • the embodiment of the present invention further provides a cluster device performance synchronization statistics method, which is applied to a second node that is a slave node in a cluster device, and the cluster device performance synchronization statistics method includes:
  • the cluster device performance synchronization statistics method further includes:
  • a performance statistics database is created, and the default statistical object type is in the database. Create a database table on it.
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the statistical objects corresponding to the slave node and the master node may be different, when the slave node is powered on for the first time, it is also necessary to establish a performance statistics database for its own statistical object, and then, when interacting with the master node, The statistics records obtained from the master node are saved to their own performance statistics database.
  • the cluster device performance synchronization statistical method of the present invention further includes the steps. S601 ⁇ S604:
  • the node implements the function of the master node, performs statistical data collection and statistical record generation, and uses this slave node as the master node, and this node can implement all the functions of the above-mentioned master node, ensuring that the cluster performance statistics are highly sustainable.
  • the embodiment of the present invention provides a cluster device performance synchronization statistics system 2, which is applied to a second node as a slave node in the cluster device, and the cluster device performance synchronization statistics system includes:
  • the obtaining module 201 is configured to acquire a statistical record on the first node, and store the statistical record.
  • the cluster device performance synchronization statistics system further includes:
  • the second initialization module 202 is configured to create a performance statistics database when the second node is powered on, and create a database table on the database for each of the preset statistical object types.
  • the statistical object types include: a node, a network port, a virtual disk, a volume, and a logical unit number.
  • the cluster device performance synchronization statistics system further includes:
  • the detecting module 204 is configured to detect whether a change signal that is changed from the second node to the first node is generated;
  • the second collection module 205 is configured to: if the change signal has been generated, collect a counter value of a statistical object on all nodes including the second node that is powered on in the cluster;
  • the second statistical record generating module 206 is configured to separately collect the counter values of the statistical objects on each of the nodes to generate a statistical record
  • the second synchronization module 207 is configured to synchronize the statistical record to other nodes in the cluster device than the second node.
  • cluster device performance synchronization statistics system is a system corresponding to the foregoing cluster device performance synchronization statistics method, and all implementation manners of the cluster device performance synchronization statistics method are applicable to the cluster device performance synchronization statistics system, The same technical effect as the above-mentioned cluster device performance synchronization statistical method can be achieved.
  • the clustering device performance synchronization statistics system is illustrated as follows:
  • the cluster device performance synchronization statistics system can be divided into: power-on initialization unit according to functions 11.
  • the cluster device performance synchronization statistics system exists on each node of the cluster, but in the actual application, the working unit of the cluster device performance synchronization statistics system on the master node is the power-on initialization unit 11,
  • the working units of the synchronous statistical system are the power-on initialization unit 11, the master-slave node interaction unit 14, and the takeover performance statistics unit 15, the original counter value collection unit 12, the performance statistics data generation unit 13, and the query and export statistics unit 16 are not working.
  • the power-on initialization unit 11 runs when all nodes of the cluster are powered on, is responsible for creating a performance statistics database, creating a database table on the database for each statistical object type, and synchronizing database data from the node (synchronous performance statistics database synchronization request is initiated from the node, The data of each statistical object type table of the performance statistics database on the master node is synchronized to the corresponding table of the database on the slave node, and the data changes of the database on the master node are synchronously incremented to the slave node database), and the registration node status change notification is The takeover performance statistics unit 15) is notified once the node status changes.
  • the original counter value collection unit 12 runs on the cluster master node, enables the data collection timer, and the collection interval can be configured as needed, for example, 15 seconds.
  • the cluster node list information is obtained through cluster management.
  • the address information of the node in the cluster node list is sent to each service subsystem of the node to obtain the original counter value of the corresponding statistical object and the two statistical time intervals before and after, and the original counter value of each statistical object obtained is obtained.
  • the statistics interval is sent to the performance statistics generating unit 13; the need to explain here is that the business subsystem should clear the original counter value of the statistical object after processing the request.
  • Performance statistics generating unit 13 running on the cluster master node, each cluster to be collected
  • the raw counter value of the statistical object of the node is summarized and processed (using the corresponding calculation method, such as traffic statistics, the execution flow original counter value/statistical time interval is obtained; and the cache hit rate is calculated, the number of hits is executed/(the number of hits + the number of losses)
  • the original counter value is processed into a readable data item
  • the statistical object statistical record is composed of a plurality of such data items, the statistical object statistical record needs to be stored in the database), and the readability is readability for the user.
  • a strong statistical record a statistical object corresponds to a statistical record at a statistical time point, the statistical record is stored in the database, and then the database synchronization mechanism synchronizes the statistical record to all cluster slave nodes as a backup; meanwhile, the in-memory database occupies memory, cannot To save data indefinitely, you need to determine the restriction method.
  • the statistical object of each statistical object type is used to limit the maximum number of statistical records. The maximum number of statistical records is determined according to how long the storage time is. Before the statistical record is saved, the statistical record bar of the current statistical object is determined.
  • the oldest statistical record of the statistical object needs to be deleted, and then the new statistical record can be saved to the database, and the new statistical record is synchronized by the database synchronization mechanism to other slave nodes.
  • the master-slave node interaction unit 14 runs when the cluster slave node is powered on, and informs the cluster master node of the memory size of the own node device, and the master node receives the request, and determines the data that the database needs to retain according to the request parameter.
  • Takeover performance statistics unit 15 Because the cluster performance statistics are performed on the primary node, if the primary node is abnormal, performance statistics cannot be performed normally; when the cluster primary node is down due to human restart or unknown reasons, the cluster management is in the cluster secondary node. A change is performed in the takeover performance statistics unit 15 of the cluster slave node that is newly elected as the cluster master node, and the performance statistics task is taken over, and all statistical objects are read from the database and saved to various In the type statistics object index array, the notification service subsystem clears each original counter value, and enables the timer to start cluster device performance statistics.
  • the query and export statistics unit 16 receives the query request sent by the user, returns the corresponding statistical record of all the statistical objects according to the parameters carried by the request, and the client uses the statistical records to draw a dynamic statistical graph; and receives the export request sent by the user.
  • the request parameter the corresponding statistical data record of the selected statistical object is returned in the form of a file, and the user selects to save the file to the local directory to achieve the purpose of saving the overall view or the historical statistical record.
  • the original counter value is the value of the variable. If the system runs for a long time, it will inevitably overflow, and since the request is not successful every time, if there is a failure, the performance statistics generating unit does not know the specific statistical time interval, which will inevitably lead to calculation. Not precise enough.
  • the processing method for case one is steps S701 to S706 (shown in FIG. 4):
  • the original counter value collecting unit 12 first requests the service subsystem to obtain the original counter value of the statistical object.
  • the service subsystem records the time of the request statistics, returns the original counter value, and clears the original counter value.
  • the original counter value collection unit 12 again requests the service subsystem to obtain the original counter value.
  • S704 The service module is very busy, and the request is not responded in time, and the request times out.
  • the original counter value collection unit 12 continues to request the service module to obtain the original counter value.
  • step S706 In response to the request, calculate a current system time, the time is subtracted from the statistical time recorded in step S702, as a statistical time interval between two points, save the system time as the statistical time of the request, and return the original counter value. The statistical interval is given to the original counter value acquisition unit 12, and the original counter value is cleared.
  • a statistical record corresponds to a unique statistical time. If the initial device system time is incorrect, the date of these errors will be added to the statistical record during the statistics. The device system time is corrected later. The statistical time of the previously generated statistical records is not available. Get corrections.
  • the processing method for case 2 is steps S801 to S803 (as shown in FIG. 5):
  • the system time is abnormal, and the performance statistical data generating unit 13 saves the statistical record to the database, and records the system time as the latest statistical time.
  • the statistic timer arrives, the timer period is 15 seconds, and the current system time is obtained, and a first time value is subtracted.
  • the first time value the statistical time of the last statistical system is +15, and then the absolute value of the difference is taken. If the absolute value is greater than a certain number, the configuration here is 8 seconds, indicating that the system time is positive. In the correction, at this time, it is determined that the current system time minus the first time value, that is, the statistical time of the latest statistical system +15) is less than 0, indicating that the time is changed back, and the timestamp field values of all statistical records are subtracted from the previous calculation. The absolute value is saved. If it is determined that the current system time minus the first time value, that is, the statistical time of the last statistical system +15) is greater than or equal to 0, the absolute value is saved, and the current system time is finally recorded as the latest statistics. time.
  • Statistical objects are dynamically added and deleted at any time. If the statistical object increases, it is obviously not possible to query the statistical information of the statistical object. You need to increase the statistical information of the statistical object. Similarly, if a statistical object is deleted, all its statistics need to be deleted.
  • the performance statistics data generating unit 13 acquires all statistical objects of the current cluster.
  • the statistical object obtained in the previous step is sequentially queried in the index array of the statistical object. If the query is not found, the statistical object is added, and a new statistical object is added in the index array of the statistical object, and the statistical record of the new statistical object is calculated and saved. Go to the database.
  • the performance statistical data generating unit 13 acquires a current cluster statistical object.
  • the processing method for case 4 includes steps S1101 to S1104 (shown in FIG. 8):
  • S1102 Obtain cluster node list information, and obtain a node that is powered off according to the state of the node.
  • S1104 Save the statistical record of the statistical object to the database.
  • the original counter value collection unit initiates the acquisition request. Since the service module is busy and cannot process the request, the timing collection performance statistics unit cannot wait for a long time. Therefore, the acquisition of the original counter value is not successful every time, and must be measured in some manner. Let the user feel that there is no problem with the system.
  • the processing manner for case 5 includes steps S1201 to S1204 (as shown in FIG. 9):
  • the original counter value collecting unit 12 requests the service subsystem to obtain the original counter value.
  • S1202 The service subsystem does not respond in time, and the request is overdue.
  • S1203 The data field of the statistical record of the current statistical object is assigned by the statistical record number field of the object last counted, and the new statistical record only modifies the statistical time.
  • the main features of the cluster device performance synchronization statistics system in the embodiment of the present invention are as follows: 1) The master node in the cluster is responsible for collecting, storing, synchronizing the raw counter information of the node device statistics object from each node of the cluster, and providing performance statistics query. In fact, the primary node is equivalent to the central host. When the current primary node fails, the performance statistics will be forwarded to the newly elected primary node to maintain high performance statistics; 2) performance statistics have been generated and saved in the cluster. On the node, data statistics can be obtained through B/S mode and C/S mode; 3) node join or exit occurs in the cluster, the volume statistics object switches the running node, the user increases and deletes the statistical object, etc., and can dynamically detect Measured and evaluated for performance.
  • all or part of the steps of the above embodiments may also be implemented using an integrated circuit.
  • the steps may be separately fabricated into individual integrated circuit modules, or a plurality of modules or steps may be fabricated into a single integrated circuit module.
  • the devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.
  • the device/function module/functional unit in the above embodiment When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium.
  • the above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.
  • the statistical value of the statistical object on each node in the cluster is calculated by the primary node in the cluster, and the statistical record is generated, and the statistical record is synchronized to each slave node in the cluster to ensure the master of the cluster.
  • the slave nodes in the cluster can continue to complete the work of the master node according to the synchronized statistics records, ensuring that the cluster performance statistics are highly sustainable.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Debugging And Monitoring (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé de synchronisation de performance et de génération de statistiques pour un dispositif de grappe et un système. Le procédé est appliqué à un premier nœud considéré comme nœud maître dans un dispositif de grappe, et comprend les étapes consistant à : collecter des valeurs de comptage d'objets statistiques sur tous les nœuds y compris le premier nœud dans une grappe ; résumer respectivement une valeur de comptage d'un objet statistique sur chaque nœud, et générer un enregistrement statistique ; et synchroniser l'enregistrement statistique avec d'autres nœuds du dispositif de grappe à l'exception du premier nœud.
PCT/CN2016/082350 2015-06-30 2016-05-17 Procédé de synchronisation de performance et de génération de statistiques pour un dispositif de grappe et un système Ceased WO2017000693A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510385906.4A CN106331047A (zh) 2015-06-30 2015-06-30 一种集群设备性能同步统计方法及系统
CN201510385906.4 2015-06-30

Publications (1)

Publication Number Publication Date
WO2017000693A1 true WO2017000693A1 (fr) 2017-01-05

Family

ID=57607676

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/082350 Ceased WO2017000693A1 (fr) 2015-06-30 2016-05-17 Procédé de synchronisation de performance et de génération de statistiques pour un dispositif de grappe et un système

Country Status (2)

Country Link
CN (1) CN106331047A (fr)
WO (1) WO2017000693A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110879825A (zh) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 一种数据同步方法及装置
WO2020053290A1 (fr) 2018-09-12 2020-03-19 Smith & Nephew Plc Dispositif, appareil et procédé de détermination de pression de perfusion cutanée
CN113743564A (zh) * 2021-01-19 2021-12-03 北京沃东天骏信息技术有限公司 一种计数方法、装置、电子设备和存储介质
CN115203331A (zh) * 2022-07-25 2022-10-18 济南浪潮数据技术有限公司 一种性能数据缓存管理方法、装置、设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106713487B (zh) * 2017-01-16 2020-10-09 腾讯科技（深圳）有限公司 数据的同步方法和装置
CN108924206B (zh) * 2018-06-26 2021-07-16 郑州云海信息技术有限公司 一种分布式系统的集群事件同步方法、装置及设备
CN111125253A (zh) * 2019-12-22 2020-05-08 北京浪潮数据技术有限公司 一种数据同步方法、装置、设备及存储介质
CN111694694A (zh) * 2020-05-22 2020-09-22 北京三快在线科技有限公司 数据库集群的处理方法、装置、存储介质和节点
CN114138825B (zh) * 2021-11-24 2024-08-23 聚好看科技股份有限公司 一种服务器及为应用程序提供数据查询服务的方法
CN115348185B (zh) * 2022-08-19 2023-12-05 招银云创信息技术有限公司 一种分布式查询引擎的控制方法及其控制装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101820384A (zh) * 2010-02-05 2010-09-01 浪潮(北京)电子信息产业有限公司 一种集群服务动态分配方法及装置
CN102571452A (zh) * 2012-02-20 2012-07-11 华为技术有限公司 多节点管理的方法和系统
US20130286893A1 (en) * 2011-05-25 2013-10-31 Huawei Technologies Co., Ltd. Route calculation method and master node device in virtual network element
CN103582122A (zh) * 2012-07-19 2014-02-12 中兴通讯股份有限公司 一种基于数字集群通信系统的组呼建立方法、网络侧设备
CN104320459A (zh) * 2014-10-24 2015-01-28 杭州华三通信技术有限公司 一种节点管理方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1591394A (zh) * 2003-09-02 2005-03-09 华为技术有限公司 数字内容的使用统计方法及系统
CN102833129A (zh) * 2012-08-15 2012-12-19 苏州迈科网络安全技术股份有限公司 网站访问率统计方法及系统
CN104104717B (zh) * 2014-06-30 2017-11-03 广州唯品会网络技术有限公司 投放渠道数据统计方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101820384A (zh) * 2010-02-05 2010-09-01 浪潮(北京)电子信息产业有限公司 一种集群服务动态分配方法及装置
US20130286893A1 (en) * 2011-05-25 2013-10-31 Huawei Technologies Co., Ltd. Route calculation method and master node device in virtual network element
CN102571452A (zh) * 2012-02-20 2012-07-11 华为技术有限公司 多节点管理的方法和系统
CN103582122A (zh) * 2012-07-19 2014-02-12 中兴通讯股份有限公司 一种基于数字集群通信系统的组呼建立方法、网络侧设备
CN104320459A (zh) * 2014-10-24 2015-01-28 杭州华三通信技术有限公司 一种节点管理方法和装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110879825A (zh) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 一种数据同步方法及装置
CN110879825B (zh) * 2018-09-06 2023-04-28 阿里巴巴集团控股有限公司 一种数据同步方法及装置
WO2020053290A1 (fr) 2018-09-12 2020-03-19 Smith & Nephew Plc Dispositif, appareil et procédé de détermination de pression de perfusion cutanée
CN113743564A (zh) * 2021-01-19 2021-12-03 北京沃东天骏信息技术有限公司 一种计数方法、装置、电子设备和存储介质
CN113743564B (zh) * 2021-01-19 2023-12-05 北京沃东天骏信息技术有限公司 一种计数方法、装置、电子设备和存储介质
CN115203331A (zh) * 2022-07-25 2022-10-18 济南浪潮数据技术有限公司 一种性能数据缓存管理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN106331047A (zh) 2017-01-11

Similar Documents

Publication Publication Date Title
WO2017000693A1 (fr) Procédé de synchronisation de performance et de génération de statistiques pour un dispositif de grappe et un système
AU2019250229B2 (en) Replication lag-constrained deletion of data in a large-scale distributed data storage system
US10496669B2 (en) System and method for augmenting consensus election in a distributed database
US9037677B2 (en) Update protocol for client-side routing information
CN108121782B (zh) 查询请求的分配方法、数据库中间件系统以及电子设备
US10769114B2 (en) Database syncing
EP2653986B1 (fr) Mise en cache côté client de jeton de transaction de base de données.
US9367261B2 (en) Computer system, data management method and data management program
JP6028850B2 (ja) データ多重化システム
JP5686034B2 (ja) クラスタシステム、同期制御方法、サーバ装置および同期制御プログラム
CN111552701B (zh) 确定分布式集群中数据一致性的方法及分布式数据系统
CN108183965A (zh) 一种数据同步方法、装置、设备、系统及可读存储介质
WO2019057193A1 (fr) Procédé de suppression de données et système de stockage réparti
CN107451013A (zh) 基于分布式系统的数据恢复方法、装置及系统
Didona et al. Okapi: Causally consistent geo-replication made faster, cheaper and more available
US9648104B2 (en) Configuration information acquisition method and management computer
WO2016095716A1 (fr) Procédé de traitement d'informations de défaillance et dispositif correspondant
US20150347516A1 (en) Distributed storage device, storage node, data providing method, and medium
JPWO2014199568A1 (ja) 永続記憶装置へのデータ書込制御方法
Pankowski Consistency and availability of Data in replicated NoSQL databases
US10860580B2 (en) Information processing device, method, and medium
US20180165380A1 (en) Data processing system and data processing method
WO2023071367A1 (fr) Procédé et appareil de traitement pour données de service de communication, et support de stockage informatique
JP2009282563A (ja) データ保存システム、プログラム及び方法、並びに監視装置
CN115168366A (zh) 数据处理方法、装置、电子设备以及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16817057

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16817057

Country of ref document: EP

Kind code of ref document: A1