CN116560945A - Database performance monitoring method, device, equipment and storage medium - Google Patents
Database performance monitoring method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116560945A CN116560945A CN202310547154.1A CN202310547154A CN116560945A CN 116560945 A CN116560945 A CN 116560945A CN 202310547154 A CN202310547154 A CN 202310547154A CN 116560945 A CN116560945 A CN 116560945A
- Authority
- CN
- China
- Prior art keywords
- target
- database
- index
- time sequence
- target index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computer Hardware Design (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The method, the device, the equipment and the storage medium for monitoring the database performance are characterized in that a current time sequence of a target index is generated based on real-time performance data of the target database, then the similarity between the current time sequence of the target index and a historical time sequence of the target index is calculated and used as an abnormal degree measurement value of the target index, and the historical time sequence is a time sequence when the target database is in an abnormal state, so that the higher the abnormal degree measurement value of the target index is, the higher the possibility of the index value abnormality of the target index is, the higher the possibility of the target database is in the abnormal state is, the state of the target database can be determined according to the database performance index value within a period of time, the condition that the database performance is instantaneously dithered can be prevented from being misjudged as the database abnormality, the interference of instantaneous dithering is eliminated, the accuracy of database abnormality judgment is improved, and the performance monitoring task of the database is realized.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a method, an apparatus, a device, and a storage medium for monitoring database performance.
Background
In online systems such as mobile banking, navigation systems and online games, a database is mostly used for providing read-write services, and the database needs to provide low-delay, efficient and stable service support to ensure system performance. However, in practical applications, due to improper index setting, poor query statement quality, or more concurrent requests, the load of the database, such as the load of the CPU, the memory, and the I/O, may be increased, thereby increasing the read-write delay of the database, degrading the performance, even the tamping of the database, and affecting the system functions. Therefore, the performance of the database needs to be monitored in real time, and the database abnormality is found in time so as to ensure the stability of the database service.
In the conventional database performance monitoring method, when the real-time performance index exceeds the preset threshold, the database is judged to be abnormal, and erroneous judgment is easy to be caused, for example, the instantaneous performance jitter of the database is judged to be abnormal, and the accuracy of the abnormal judgment is poor.
Disclosure of Invention
In view of the above problems, the present application is provided to provide a method, an apparatus, a device, and a storage medium for monitoring database performance, so as to realize the task of monitoring database performance and improve the accuracy of database anomaly determination.
The specific scheme is as follows:
in a first aspect, a database performance monitoring method is provided, including:
acquiring real-time performance data of a target database, wherein the real-time performance data comprises index values of a plurality of performance indexes in a current period of time;
generating a current time sequence of at least one target index according to the real-time performance data, wherein the current time sequence of each target index is a sequence formed by index values of the target indexes arranged according to time sequence;
for each target index, calculating the similarity between the current time sequence of the target index and the historical time sequence of the target index as an abnormality degree measurement value of the target index, wherein the historical time sequence of the target index is the time sequence when the target database is in an abnormal state;
and judging whether the state of the target database is abnormal or not according to the abnormality degree measurement value of each target index.
In a second aspect, there is provided a database performance monitoring apparatus comprising:
the data acquisition unit is used for acquiring real-time performance data of the target database, wherein the real-time performance data comprises index values of a plurality of performance indexes in a current period of time;
a current time sequence configuration unit, configured to generate a current time sequence of at least one target index according to the real-time performance data, where each current time sequence of the target indexes is a sequence formed by index values of the target indexes arranged in time sequence;
a similarity calculation unit configured to calculate, for each of the target indexes, a similarity between a current time series of the target index and a historical time series of the target index, as an abnormality degree measurement value of the target index, wherein the historical time series of the target index is a time series when the target database is in an abnormal state;
and the state judging unit is used for judging whether the state of the target database is abnormal according to the abnormality degree measurement value of each target index.
In a third aspect, a database performance monitoring apparatus is provided, comprising: a memory and a processor;
the memory is used for storing programs;
the processor is configured to execute the program to implement each step of the database performance monitoring method.
In a fourth aspect, a storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the steps of the database performance monitoring method described above.
By means of the technical scheme, the similarity between the current time sequence of the target index and the historical time sequence of the abnormality of the target index is calculated, the similarity is used as the abnormality degree measurement value of the target index, whether the real-time performance of the target database is similar to the performance of the target database in the abnormality or not can be determined through the abnormality degree measurement value of each target index in the current period, and it is required to explain that the more similar the real-time performance is to the performance in the abnormality, the higher the possibility that the state of the current target database is abnormal is, so that whether the state of the target database is abnormal or not can be judged according to the abnormality degree measurement value of each target index. According to the method and the device for determining the state of the target database, the state of the target database is determined according to the database performance index value in a period of time, the situation that the database performance instantaneously shakes can be prevented from being misjudged as the database abnormality, the interference of the instantaneous shake is eliminated, and the accuracy of judging the database abnormality is improved.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 is a flow chart of a database performance monitoring method according to an embodiment of the present application;
FIG. 2 shows a performance monitoring snapshot of a database MongoDB;
FIG. 3 illustrates real-time performance data of a MongoDB database;
FIG. 4 is a flowchart illustrating another database performance monitoring method according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a database performance monitoring device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a database performance monitoring device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The application provides a database performance monitoring method, device, equipment and storage medium, which can be suitable for realizing a monitoring task of database performance in a designated monitoring period.
The scheme can be realized based on the terminal with the data processing capability, and the terminal can be a mobile phone, a computer, a server, a cloud terminal and the like.
FIG. 1 is a flow chart of a database performance monitoring method according to an embodiment of the present application, and in conjunction with FIG. 1, the method may include the following steps:
step S101, acquiring real-time performance data of a target database.
The real-time performance data comprises index values of a plurality of performance indexes in a current period of time, wherein the current period of time refers to a currently appointed monitoring period of time, that is, performance index values of the target database can be acquired according to a preset time interval in the current period of time to obtain a plurality of performance index sequences at different acquisition moments, each performance index sequence forms the real-time performance data of the target database, and each performance index sequence refers to a sequence formed by index values of each performance index of the target database at a certain acquisition moment. Alternatively, the time interval may be 1 second, and the duration of one of the monitoring periods may be ten minutes to several hours.
Step S102, generating a current time sequence of at least one target index according to the real-time performance data.
The current time sequence of the target indexes refers to a time sequence corresponding to the current period of time, and each time sequence of the target indexes is a sequence formed by index values of the target indexes arranged according to time sequence. Alternatively, the current time series of one target index may include hundreds to thousands of index values of the target index.
Step S103, similarity calculation is performed based on the current time sequence and the historical time sequence.
Specifically, for each target indicator, calculating a similarity between a current time sequence of the target indicator and a historical time sequence of the target indicator, and taking the similarity between the current time sequence of the target indicator and the historical time sequence of the target indicator as an abnormal degree measurement value of the target indicator, where the similarity is used to represent a degree of similarity between two time sequences of the target indicator, and the historical time sequence of the target indicator is a time sequence when the target database is in an abnormal state, and specifically, a configuration process of the historical time sequence of the target indicator may include: and acquiring historical performance data in a period of time when the target database is in an abnormal state, and arranging historical index values of the target indexes in the historical performance data according to time sequence to obtain a historical time sequence of the target indexes. Optionally, the historical time sequence of the target index may be manually configured according to a change characteristic of an index value of the target index when the target database is in an abnormal state.
In addition, the current time sequence of each target index can form a current time sequence file querySeries List, the historical time sequence of each target index can form a historical time sequence file timeeriesList, and then the abnormality degree measurement value of each target index is calculated by adopting a matrix operation mode.
And step S104, judging whether the state of the target database is abnormal according to the abnormality degree measurement value of each target index.
It should be noted that, since the historical time series of each target index is the time series when the target database is in an abnormal state, the more similar the current time series of the target index is to the historical time series of the target index, the higher the abnormality degree measurement value of the target index is, the higher the possibility that the target database is in an abnormal state at present, that is, the higher the abnormality degree measurement value of each target index is, and it can be determined whether the target database is in an abnormal state at present.
According to the scheme, the similarity between the current time sequence of the target index and the historical time sequence of the abnormality of the target index is calculated, the similarity is used as the abnormality degree measurement value of the target index, whether the state of the target database is abnormal is judged according to the abnormality degree measurement value of each target index, and the performance monitoring task of the target database is realized.
Compared with the scheme of determining the state of the database according to the index values of a single index or a plurality of indexes at a certain moment, the scheme can avoid misjudging the condition of the instantaneous jitter of the performance of the database as the abnormal condition of the database and eliminate the interference of the instantaneous jitter by determining the state of the database according to the performance index value in a period of time.
In addition, the method and the device can capture the periodicity and the trend characteristics in the time sequence, can identify the periodic variation condition of the database performance through configuration of the monitoring time period of the minute level and even the hour level, and judge the database state under the condition, namely, by increasing the length of the time sequence, the possibility of misjudging the short-time abnormality of the database performance as the abnormality of the database can be reduced, the interference of the periodic variation of the database performance on the judgment of the database state is reduced, and the accuracy of the abnormality judgment of the database is improved.
Compared with the scheme of determining the state of the database according to the index value of the fixed index and the corresponding index value threshold value, the scheme of determining the state of the database according to the similarity between the real-time performance and the abnormal performance of the database can determine the state of the database according to the time sequence of a plurality of different target index types, that is, the scheme of the application has good expandability on the performance index type of the database and the type of the database, and can be suitable for realizing performance monitoring tasks of different databases.
In some embodiments provided herein, the target database may be a mongo db database.
The MongoDB database is an open-source, high-performance, schema-free and document-oriented database written in the C++ language, can provide an extensible high-performance data storage solution for applications, and is a non-relational database most similar to a relational database.
It should be noted that, the component suitable for implementing the performance monitoring task of the MongoDB database is only the charging monitoring service MongoDB Atlas provided by the authorities, and the MongoDB Atlas is a cloud-hosted database, i.e. a service, for running, monitoring and maintaining deployment and various performance indexes of the MongoDB, and can provide visual display and alarm according to the collected data. Because there is no MongoDB database monitoring component with universal open source, most of clients currently monitor the server index of the database only through Zabbix, such as network, I/O, CPU and memory, etc., the Zabbix is an enterprise-level open source solution of distributed system monitoring and network monitoring functions based on WEB interfaces, and can be used for monitoring various network parameters, ensuring the safe operation of the server system, and providing a flexible notification mechanism, so that the system administrator can quickly locate and solve the problems of the server system. In general, the existing MongoDB database performance monitoring scheme has fewer performance index types capable of being monitored and is not friendly to medium and small-sized database users.
To solve the above problem, in some embodiments provided in the present application, the acquiring real-time performance data of the target database includes:
and acquiring real-time performance data of the MongoDB database by using a performance monitoring tool Mongostat.
It should be noted that, after configuring database connection parameters such as MONGOIP (IP of monglos node), monoport (port of monglos node), mongade (database name), mognoposername, and mongaswawd (password), real-time performance data of the monglodb database to be monitored may be obtained through the performance monitoring tool monglostat. The monglostat is a background performance monitoring tool of the mongloDB, and can acquire the current running state of the mongloDB, such as indexes of cache, dirty page utilization rate, request queue, read-write and the like, and output the indexes to a command line so as to derive real-time performance data of the mongloDB.
By way of example, fig. 2 shows a performance monitoring snapshot of a database MongoDB, and table 1 shows an illustration of performance metrics of the database MongoDB, it being noted that for a performance metric insert, if the metric value of the insert follows, this is indicative of a copy operation; for the performance index getcore, if the data volume of one query is larger and exceeds the maximum data volume of one query of MongoDB, the MongoDB divides the data to be queried into multiple queries and returns query results respectively; for the performance index flush, in a WiredTiger storage engine, flush refers to the time interval of creating check points by the WiredTiger circulation, and the MongoDB writes the data on the memory into the hard disk at intervals, if the flush value is larger, the performance of the database is affected; for the performance index repl, the state of the replica set is represented, specifically, PRI represents Primary, SEC represents Secondary, M represents master, REC represents recovery, UNK represents unow, SLV represents slave, and ARB represents aribater.
TABLE 1
By analyzing the monglostat monitoring snapshot when the target database is abnormal, abnormal performance data of the mongloDB database can be obtained so as to generate a historical time sequence of the target index; database performance data per second can be acquired in real time by means of mondostat to generate a current time series of target indicators.
Fig. 3 illustrates real-time performance data of a mongo db database in the form of a table file, in which a time sequence of individual performance indicators of the mongo db database is included.
FIG. 4 is a flow chart of another database performance monitoring method according to an embodiment of the present application, and in conjunction with FIG. 4, the method may include the following steps:
steps S201 to S202 are identical to steps S101 to S102 described above, and will not be described here again.
For each of the target indicators, the subsequent steps S203 to S206 are performed.
Step S203, calculating a similarity between the current time sequence of the target index and the historical time sequence of the target index.
And then, taking the similarity between the current time sequence of the target index and the historical time sequence of the target index as an abnormality degree measurement value of the target index.
Step S204, determining whether the abnormality degree measurement value of the target indicator is greater than a preset measurement value threshold of the target indicator, if yes, executing step S205, and if no, executing step S206.
Step S205, the judging result of the target index includes: the current time sequence of the target index is an abnormal sequence.
Step S206, the judging result of the target index includes: the current time sequence of the target index is a normal sequence.
Optionally, for each target indicator, a plurality of historical time sequences of the target indicator may be obtained, each historical time sequence may correspond to a different abnormal state of the database, then for each historical time sequence of the target indicator, the step S203 is performed to obtain a plurality of similarities, and then based on each similarity, an abnormality degree measurement value of the target indicator is determined, so as to determine whether the current time sequence of the target indicator is an abnormality sequence based on the abnormality degree measurement value of the target indicator.
And S207, determining the state of the target database according to the judging result of each target index.
According to the scheme, the state of the current time sequence of each target index is determined according to the comparison result of the abnormal degree measurement value and the preset measurement value threshold value, so that the state of the target database is determined, and the task of monitoring the performance of the database is realized.
In some embodiments provided in the present application, the determining the state of the target database in step S207 according to the determination result of each target index may include:
and determining the state of the target database as abnormal when the current time sequence of any one of the target indexes is an abnormal sequence.
Optionally, after determining the state of the target database as abnormal, the method may further include:
and outputting an alarm signal.
Illustratively, the outputting the alarm signal may include:
and a short message interface is called up, and an alarm signal is sent to a mobile phone of a development and maintenance personnel of the target database, so that the development and maintenance personnel can find and solve problems in time, and the high-efficiency and stable operation of the target database is ensured.
Alternatively, the anomaly level of the target database may be determined according to the number of anomaly sequences, and different alarm signals are respectively output for different anomaly levels.
The following describes a concrete expression of the similarity between the current time series and the historical time series.
In some embodiments provided herein, the calculating the similarity between the current time sequence of the target indicator and the historical time sequence of the target indicator as the abnormality degree measurement value of the target indicator may include the following steps:
and step A, calculating the Euclidean distance between the current time sequence of the target index and the historical time sequence of the target index.
And B, determining an abnormality degree measurement value of the target index based on the Euclidean distance.
Wherein the Euclidean distance is inversely related to the abnormality degree measurement value of the target index.
Alternatively, the euclidean distance may be inversely proportional to the abnormality degree metric value of the target index. In addition to Euclidean distance, norms or other distances may be used to measure similarity between two time series.
For example, assuming that the current time series file querySeries List includes a current time series querySeries of a target index, and the historical time series file timeeriesList includes a plurality of historical time series of the target index, different historical time series respectively correspond to different abnormal states of the database, a process of applying the similarity between Euclidean distance measurement sequences and determining whether the current time series of the target index is abnormal may include the following steps C-J:
and C, acquiring a first historical time sequence of the target index from the historical time sequence file timeeriesList.
And D, calculating the Euclidean distance between the current time sequence querySeries and the historical time sequence timeeries.
It should be noted that the lengths of the current time series querySeries and the historical time series timeies may be different, and the second length l of the historical time series timeies 2 A first length l greater than the current time sequence querySeries 1 . Thus, in calculating the euclidean distance, n subsequences may be sequentially truncated from the historical time series in time order, where n=l 2 -1 1 +1, exemplary, assuming the historical time series timeeries= [1,2,3,4,5]The first length l of the current time sequence querySeries 1 =3, then the individual subsequences can be denoted as [1,2,3 ]]、[2,3,4]And [3,4,5 ]]The method comprises the steps of carrying out a first treatment on the surface of the Then for each sub-sequence, calculate the currentThe Euclidean distance of the pre-time sequence querySeries and the subsequence; and taking the Euclidean distance with the smallest calculated value as the Euclidean distance between the current time sequence querySeries and the historical time sequence timeeries.
Alternatively, in order to reduce the calculation complexity of the euclidean distance or the similarity and improve the calculation efficiency, the length of the time series may be specified, for example, the euclidean distance or the similarity may be calculated based on the time series of the specified length in the original current time series.
And E, judging whether the Euclidean distance is larger than the maximum value of the Euclidean distance, and if so, executing the step F.
The maximum value of the initial euclidean distance is a preset value, for example, -1, and the time sequence corresponding to the maximum value of the initial euclidean distance is null.
And F, taking the Euclidean distance as a maximum value of the Euclidean distance, and taking the historical time sequence as a time sequence corresponding to the maximum value of the Euclidean distance.
And G, acquiring the next historical time series, and returning to the step D until the Euclidean distance corresponding to each historical time series is calculated.
And step H, judging whether the maximum value of the Euclidean distance is smaller than a preset distance threshold, if so, executing the step I, and if not, executing the step J.
And step I, determining the current time sequence querySeries of the target index as an abnormal sequence.
And step J, determining the current time sequence querySeries of the target index as a normal sequence.
Optionally, a plurality of different distance ranges may be preset, and according to the difference of the distance ranges where the maximum value of the euclidean distance is located, determining the abnormal level of the current time sequence querySeries of the target index, and then for different abnormal levels, taking different follow-up measures.
Optionally, the abnormality diagnosis may be performed on the current time sequence querySeries based on the time sequence corresponding to the maximum value of the euclidean distance, and the abnormality diagnosis process may include the following steps K-N:
and K, acquiring a time sequence corresponding to the maximum value of the Euclidean distance.
And step L, acquiring a median mean and a median absolute deviation mad of the time sequence.
And M, taking the sum of the median media and the median absolute deviation mad which is 3 times as a threshold value.
And step N, determining the current time sequence querySeries of the target index as an abnormal sequence under the condition that the maximum value of the Euclidean distance is smaller than the threshold value.
Alternatively, the similarity between the current time sequence querySeries and the historical time sequence timeeries, that is, the abnormal degree metric value of the target index, may be determined based on a mapping relationship between a preset euclidean distance and the abnormal degree metric value, for example, an inverse relationship, and the distance threshold or the metric value threshold corresponding to the threshold may be determined based on the mapping relationship, and the current time sequence querySeries of the target index may be determined as the abnormal sequence if the abnormal degree metric value of the target index is greater than the metric value threshold.
That is, the metric threshold of the target indicator may be determined based on a statistical result of the historical time series of the target indicator, the statistical result being a sum of median absolute deviations of 3 times.
The above process may be implemented in accordance with pre-written code, as follows:
it should be noted that, after the historical abnormal time sequence file and the real-time sequence file of the target database are configured, the similarity between the historical time sequence and the current time sequence can be calculated based on a Tima algorithm (Time series Indexing, matching and Alignment), specifically, the Tima algorithm is an algorithm based on time sequence similarity matching, the main idea is that two time sequences are respectively regarded as two sets formed by a plurality of subsequences, the similarity between the two subsequences is measured by calculating the similarity between the two subsequences, specifically, for each original time sequence, the original time sequence is processed by utilizing a sliding window with a preset length to obtain a plurality of subsequences with a preset length, and it should be noted that the subsequence lengths of the two original time sequences can be the same or different; calculating the statistical characteristics of each subsequence, wherein the statistical characteristics can comprise mean value, variance, maximum value, minimum value and the like, and measuring the similarity degree between the two subsequences according to the similarity degree between the statistical characteristics of the two subsequences, namely, an index structure is used by a Tima algorithm; and determining two subsequences with highest similarity, aligning the two subsequences to ensure that the lengths of the two subsequences are equal, and taking the similarity between the two subsequences after the alignment as the similarity between the two original time sequences.
Because the Tima algorithm uses an index structure, namely, the similarity of the statistical characteristics of the two subsequences is used as the similarity of the two subsequences, the similarity of the two subsequences is not directly calculated, the calculation speed of similarity calculation is improved, and a large number of time sequences can be rapidly processed; the similarity of the historical time sequence and the current time sequence is compared based on a Tima algorithm, so that the characteristics such as periodicity, trend and the like in the time sequence can be accurately captured; and when the Tima algorithm is applied, the similarity can be measured in various forms such as Euclidean distance, and various types of statistical characteristics can be used, so that the method can be suitable for various application scenes; in addition, the alignment processing can be realized by adopting various interpolation methods, so that the Tima algorithm can be suitable for a scene with data loss, and the robustness and the accuracy of a time sequence can be maintained by supplementing data through interpolation, therefore, the Tima algorithm has certain robustness. In conclusion, based on the Tima algorithm, the time sequence similarity calculation task with high efficiency and high accuracy can be realized.
The database performance monitoring device provided in the embodiments of the present application will be described below, and the database performance monitoring device described below and the database performance monitoring method described above may be referred to correspondingly.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a database performance monitoring device according to an embodiment of the present application.
As shown in fig. 5, the apparatus may include:
a data acquisition unit 11, configured to acquire real-time performance data of a target database, where the real-time performance data includes index values of a plurality of performance indexes in a current period of time;
a current time sequence configuration unit 12, configured to generate a current time sequence of at least one target index according to the real-time performance data, where each current time sequence of the target indexes is a sequence composed of index values of the target indexes arranged in chronological order;
a similarity calculation unit 13 for calculating, for each of the target indexes, a similarity between a current time series of the target index and a historical time series of the target index, which is a time series when the target database is in an abnormal state, as an abnormality degree measurement value of the target index;
and a state determination unit 14 configured to determine whether the state of the target database is abnormal based on the abnormality degree measurement values of the respective target indexes.
In some embodiments provided herein, the process of calculating, by the similarity calculating unit 13, the similarity between the current time series of the target indicator and the historical time series of the target indicator as the abnormality degree measurement value of the target indicator may include:
calculating the Euclidean distance between the current time sequence of the target index and the historical time sequence of the target index;
and determining an abnormality degree measurement value of the target index based on the Euclidean distance, wherein the Euclidean distance is in negative correlation with the abnormality degree measurement value of the target index.
In some embodiments provided herein, the process of determining, by the state determining unit 14, whether the state of the target database is abnormal according to the abnormality degree metric value of each target index may include:
judging whether the abnormality degree measurement value of each target index is larger than a preset measurement value threshold value of the target index, if so, judging that the current time sequence of the target index is an abnormal sequence, and if not, judging that the current time sequence of the target index is a normal sequence;
and determining the state of the target database according to the judging result of each target index.
In some embodiments provided herein, the metric threshold for the target indicator is determined from statistics of historical time series of the target indicator, the statistics being a sum of median absolute deviations from 3 times.
In some embodiments provided herein, the process of determining the state of the target database by the state determining unit 14 according to the determination result of each target index may include:
and determining the state of the target database as abnormal when the current time sequence of any one of the target indexes is an abnormal sequence.
In some embodiments provided herein, the target database may be a mongo db database.
In some embodiments provided herein, the process of acquiring the real-time performance data of the target database by the data acquiring unit 11 may include:
and acquiring real-time performance data of the MongoDB database by using a performance monitoring tool Mongostat.
The database performance monitoring device provided by the embodiment of the application can be applied to database performance monitoring equipment, such as a terminal: cell phones, computers, etc. Alternatively, fig. 6 shows a block diagram of a hardware structure of the database performance monitoring apparatus, and referring to fig. 6, the hardware structure of the database performance monitoring apparatus may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
in the embodiment of the application, the number of the processor 1, the communication interface 2, the memory 3 and the communication bus 4 is at least one, and the processor 1, the communication interface 2 and the memory 3 complete communication with each other through the communication bus 4;
processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention, etc.;
the memory 3 may comprise a high-speed RAM memory, and may further comprise a non-volatile memory (non-volatile memory) or the like, such as at least one magnetic disk memory;
wherein the memory stores a program, the processor is operable to invoke the program stored in the memory, the program operable to:
acquiring real-time performance data of a target database, wherein the real-time performance data comprises index values of a plurality of performance indexes in a current period of time;
generating a current time sequence of at least one target index according to the real-time performance data, wherein the current time sequence of each target index is a sequence formed by index values of the target indexes arranged according to time sequence;
for each target index, calculating the similarity between the current time sequence of the target index and the historical time sequence of the target index as an abnormality degree measurement value of the target index, wherein the historical time sequence of the target index is the time sequence when the target database is in an abnormal state;
and judging whether the state of the target database is abnormal or not according to the abnormality degree measurement value of each target index.
Alternatively, the refinement function and the extension function of the program may be described with reference to the above.
The embodiment of the application also provides a storage medium, which may store a program adapted to be executed by a processor, the program being configured to:
acquiring real-time performance data of a target database, wherein the real-time performance data comprises index values of a plurality of performance indexes in a current period of time;
generating a current time sequence of at least one target index according to the real-time performance data, wherein the current time sequence of each target index is a sequence formed by index values of the target indexes arranged according to time sequence;
for each target index, calculating the similarity between the current time sequence of the target index and the historical time sequence of the target index as an abnormality degree measurement value of the target index, wherein the historical time sequence of the target index is the time sequence when the target database is in an abnormal state;
and judging whether the state of the target database is abnormal or not according to the abnormality degree measurement value of each target index.
Alternatively, the refinement function and the extension function of the program may be described with reference to the above.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and may be combined according to needs, and the same similar parts may be referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for monitoring database performance, comprising:
acquiring real-time performance data of a target database, wherein the real-time performance data comprises index values of a plurality of performance indexes in a current period of time;
generating a current time sequence of at least one target index according to the real-time performance data, wherein the current time sequence of each target index is a sequence formed by index values of the target indexes arranged according to time sequence;
for each target index, calculating the similarity between the current time sequence of the target index and the historical time sequence of the target index as an abnormality degree measurement value of the target index, wherein the historical time sequence of the target index is the time sequence when the target database is in an abnormal state;
and judging whether the state of the target database is abnormal or not according to the abnormality degree measurement value of each target index.
2. The method according to claim 1, wherein calculating a similarity between a current time series of the target index and a historical time series of the target index as an abnormality degree measurement value of the target index includes:
calculating the Euclidean distance between the current time sequence of the target index and the historical time sequence of the target index;
and determining an abnormality degree measurement value of the target index based on the Euclidean distance, wherein the Euclidean distance is in negative correlation with the abnormality degree measurement value of the target index.
3. The method of claim 1, wherein determining whether the state of the target database is abnormal based on the degree of abnormality metric value of each of the target indexes comprises:
judging whether the abnormality degree measurement value of each target index is larger than a preset measurement value threshold value of the target index, if so, judging that the current time sequence of the target index is an abnormal sequence, and if not, judging that the current time sequence of the target index is a normal sequence;
and determining the state of the target database according to the judging result of each target index.
4. A method according to claim 3, wherein the metric threshold for the target indicator is determined from statistics of historical time series of the target indicator, the statistics being the sum of the median absolute deviation from 3 times the median.
5. A method according to claim 3, wherein said determining the state of the target database according to the determination result of each target index comprises:
and determining the state of the target database as abnormal when the current time sequence of any one of the target indexes is an abnormal sequence.
6. The method of any one of claims 1-5, wherein the target database is a MongoDB database.
7. The method of claim 6, wherein the obtaining real-time performance data of the target database comprises:
and acquiring real-time performance data of the MongoDB database by using a performance monitoring tool Mongostat.
8. A database performance monitoring apparatus, comprising:
the data acquisition unit is used for acquiring real-time performance data of the target database, wherein the real-time performance data comprises index values of a plurality of performance indexes in a current period of time;
a current time sequence configuration unit, configured to generate a current time sequence of at least one target index according to the real-time performance data, where each current time sequence of the target indexes is a sequence formed by index values of the target indexes arranged in time sequence;
a similarity calculation unit configured to calculate, for each of the target indexes, a similarity between a current time series of the target index and a historical time series of the target index, as an abnormality degree measurement value of the target index, wherein the historical time series of the target index is a time series when the target database is in an abnormal state;
and the state judging unit is used for judging whether the state of the target database is abnormal according to the abnormality degree measurement value of each target index.
9. A database performance monitoring apparatus, comprising: a memory and a processor;
the memory is used for storing programs;
the processor being configured to execute the program to implement the steps of the database performance monitoring method according to any one of claims 1-7.
10. A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the database performance monitoring method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310547154.1A CN116560945A (en) | 2023-05-15 | 2023-05-15 | Database performance monitoring method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310547154.1A CN116560945A (en) | 2023-05-15 | 2023-05-15 | Database performance monitoring method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116560945A true CN116560945A (en) | 2023-08-08 |
Family
ID=87501471
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310547154.1A Pending CN116560945A (en) | 2023-05-15 | 2023-05-15 | Database performance monitoring method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116560945A (en) |
-
2023
- 2023-05-15 CN CN202310547154.1A patent/CN116560945A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111064614B (en) | Fault root cause positioning method, device, equipment and storage medium | |
| US7676458B2 (en) | System and method for historical diagnosis of sensor networks | |
| CN110851338A (en) | Abnormality detection method, electronic device, and storage medium | |
| CN114528934A (en) | Time series data abnormity detection method, device, equipment and medium | |
| US11010260B1 (en) | Generating a data protection risk assessment score for a backup and recovery storage system | |
| CN114564345B (en) | A server anomaly detection method, device, equipment and storage medium | |
| CN110933115B (en) | Analysis object behavior abnormity detection method and device based on dynamic session | |
| WO2021061090A1 (en) | Time-series anomaly detection using an inverted index | |
| US20180167260A1 (en) | Resource and Metric Ranking by Differential Analysis | |
| CN111367747B (en) | Index abnormal detection early warning device based on time annotation | |
| CN108551412A (en) | Monitoring data noise reduction process method and apparatus | |
| US7739661B2 (en) | Methods and systems for planning and tracking software reliability and availability | |
| CN113487086B (en) | Method, device, computer equipment and medium for predicting residual service life of equipment | |
| US7386417B1 (en) | Method and apparatus for clustering telemetry signals to facilitate computer system monitoring | |
| CN109560978B (en) | Network traffic detection method, device and system, and computer-readable storage medium | |
| US10409662B1 (en) | Automated anomaly detection | |
| CN112364900A (en) | Equipment alarm management method, device, client and medium for intelligent building | |
| Fattah et al. | Long-term IaaS selection using performance discovery | |
| US20140280860A1 (en) | Method and system for signal categorization for monitoring and detecting health changes in a database system | |
| CN116560945A (en) | Database performance monitoring method, device, equipment and storage medium | |
| CN115661768A (en) | Space-time prediction model robustness testing method, device, equipment and medium | |
| CN119668976B (en) | A server operation status monitoring and management system based on AI intelligence | |
| CN119276451B (en) | Method, device, equipment and medium for timing compensation of cable sensor network | |
| CN112527622B (en) | A performance test result analysis method and device | |
| CN114443407A (en) | Detection method and system of server, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |