CN113485901B

CN113485901B - System evaluation method, device, equipment and medium based on log and index

Info

Publication number: CN113485901B
Application number: CN202110763094.8A
Authority: CN
Inventors: 吴声; 李耕寅; 常杰; 茅逸斐
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2021-07-06
Filing date: 2021-07-06
Publication date: 2022-11-22
Anticipated expiration: 2041-07-06
Also published as: CN113485901A

Abstract

The invention relates to the field of artificial intelligence, in particular to a system evaluation method, device, equipment and medium based on logs and indexes, wherein the method comprises the following steps: comparing the real-time log data with a log template to determine a first abnormal value; comparing the plurality of items of real-time index data with the corresponding plurality of items of historical index data to determine a second abnormal value; judging whether the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value; if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value, determining that the evaluation result of the system is abnormal; and if the first abnormal value and the second abnormal value are not the primary alarm value, the secondary alarm value or the tertiary alarm value, determining that the evaluation result of the system is normal. This can improve the accuracy in evaluating the system.

Description

System evaluation method, device, equipment and medium based on log and index

Technical Field

The invention relates to the field of artificial intelligence, in particular to a system evaluation method, device, equipment and medium based on logs and indexes.

Background

In the prior art, most of evaluation of a system is based on log monitoring or index monitoring, wherein the log monitoring is to monitor log data generated by the system to see whether the log data contains abnormal data or not, and if the log data contains abnormal data, an alarm or other subsequent operations are generated; the index monitoring is to monitor indexes such as system response time, memory occupancy rate, CPU occupancy rate and the like, and if the indexes are abnormal, an alarm or other subsequent operations can be generated. However, log monitoring and index monitoring in the prior art are often separated, which may result in too many facets when evaluating a system, resulting in low evaluation accuracy.

Therefore, there is a need for a method, an apparatus, a device and a medium for evaluating a system based on a log and an index, which can improve the accuracy of evaluating the system.

Disclosure of Invention

An object of the embodiments herein is to provide a method, an apparatus, a device, and a medium for system evaluation based on logs and indexes, so as to improve accuracy in evaluating a system.

In order to achieve the above object, in one aspect, an embodiment herein provides a system evaluation method based on logs and indexes, including:

comparing the real-time log data with a log template to determine a first abnormal value;

comparing the plurality of items of real-time index data with the corresponding plurality of items of historical index data to determine a second abnormal value;

judging whether the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value;

if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value, determining that the evaluation result of the system is abnormal;

and if the first abnormal value and the second abnormal value are not the primary alarm value, the secondary alarm value or the tertiary alarm value, determining that the evaluation result of the system is normal.

Preferably, the comparing the real-time log data with the log template to determine the first abnormal value includes:

comparing real-time log data in unit time with log templates of different categories, and determining the category of the real-time log data in unit time;

and determining a first abnormal value according to the category of the real-time log data in unit time and a comparison result of the real-time log number in unit time under the category and a first threshold value of the corresponding log template, wherein the first threshold value is the threshold value of the log number of the corresponding log template in unit time.

Preferably, the method for determining the log template includes:

obtaining historical log data, and performing word segmentation on each piece of historical log data;

merging and extracting the historical log data after word segmentation through a clustering algorithm, and determining a log template corresponding to each type of log data.

Preferably, the method for determining the first threshold of the corresponding log template includes:

judging whether the log template is an abnormal log template or not;

if the log template is the abnormal log template; determining the set threshold value as a first threshold value of the number of logs in unit time corresponding to the log template;

and if the log template is not the abnormal log template, determining a first threshold value of the log number of the corresponding log template in unit time according to the number of the historical log data corresponding to the log template, wherein the first threshold value comprises a first upper threshold value and a first lower threshold value.

Preferably, the determining a first abnormal value according to the category to which the real-time log data belongs per unit time includes:

judging whether the category of the real-time log data in unit time is a category corresponding to any log template or not;

if the category of the real-time log data in unit time is not the category corresponding to any log template, adding the category of the real-time log data into the log template, and determining the first abnormal value as a secondary alarm value;

and if the category of the real-time log data in the unit time is the category corresponding to any log template, determining a first abnormal value according to a comparison result of the number of the real-time logs in the unit time under the category and a first threshold value of the corresponding log template.

Preferably, the determining a first abnormal value according to a comparison result between the real-time log number in a unit time under the category to which the log template belongs and a first threshold of the corresponding log template includes:

judging that the first threshold value of the corresponding log template is the set threshold value, or the first upper threshold value and the first lower threshold value;

if the first threshold value of the corresponding log template is the set threshold value, determining the first abnormal value as a primary alarm value when the number of the real-time logs in unit time under the category of the log template is greater than the first threshold value of the corresponding log template;

and if the first threshold value of the corresponding log template is the first upper threshold value and the first lower threshold value, determining the first abnormal value as a third-level alarm value when the number of the real-time logs in the unit time under the category is greater than the first upper threshold value or less than the first lower threshold value.

Preferably, the comparing the plurality of items of real-time index data with the corresponding plurality of items of historical index data to determine a second abnormal value includes:

determining a second threshold corresponding to each index data according to a plurality of items of historical index data;

and comparing the plurality of items of real-time index data in the unit time with the corresponding second threshold respectively to determine a second abnormal value.

Preferably, the determining, according to multiple items of historical index data, a second threshold corresponding to each item of index data includes:

rejecting abnormal data in each item of historical index data;

and determining a second upper threshold and a second lower threshold of each item of index data according to the distribution condition of each item of historical index data.

Preferably, the comparing the plurality of items of real-time index data in the unit time with the corresponding second threshold values respectively to determine second abnormal values includes:

and comparing multiple items of real-time index data in unit time with the corresponding second threshold respectively, and determining the second abnormal value as a three-level alarm value when each item of real-time index data is greater than the corresponding second upper threshold or smaller than the second lower threshold.

Preferably, if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value, determining that the evaluation result of the system is a system abnormality includes:

if the first abnormal value or the second abnormal value is a primary alarm value, determining that the evaluation result of the system is important abnormality in system abnormality;

if the first abnormal value or the second abnormal value is a secondary alarm value, determining that the evaluation result of the system is a secondary abnormality in system abnormalities;

and if the first abnormal value or the second abnormal value is a three-level alarm value, determining that the evaluation result of the system is the system abnormality corresponding to the abnormal type according to the size relation between the times of the three-level alarm value and the first and second times.

Preferably, the determining that the evaluation result of the system is the system abnormality corresponding to the abnormality type according to the magnitude relationship between the number of times of the three-level alarm value and the first number of times and the second number of times includes:

judging the relationship between the times of the three-level alarm values and the first times and the second times;

if the times of the three-level alarm values are less than or equal to the second times, determining that the evaluation result of the system is suspected abnormality in system abnormality;

if the three-level alarm value is greater than the second time and less than the first time, determining that the evaluation result of the system is a secondary abnormality in system abnormalities;

and if the three-level alarm value is greater than or equal to the first time, determining that the evaluation result of the system is an important exception in system exceptions.

In another aspect, embodiments herein provide a system evaluation apparatus based on logs and indicators, the apparatus including:

a first outlier determination module: comparing the real-time log data with a log template to determine a first abnormal value;

a second outlier determination module: comparing the plurality of items of real-time index data with the corresponding plurality of items of historical index data to determine a second abnormal value;

a system evaluation module: judging whether the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value;

and if the first abnormal value and the second abnormal value are not the primary alarm value, the secondary alarm value or the tertiary alarm value, determining that the system is normal in the evaluation result.

In yet another aspect, embodiments herein also provide a computer device comprising a memory, a processor, and a computer program stored on the memory, the computer program, when executed by the processor, performing the instructions of any one of the methods described above.

In yet another aspect, embodiments herein also provide a computer-readable storage medium having stored thereon a computer program, which when executed by a processor of a computer device, performs the instructions of any one of the methods described above.

As can be seen from the above technical solutions provided in the embodiments herein, by determining the first abnormal value and the second abnormal value, the relationship between the first abnormal value and the second abnormal value and the first level alarm value, the second level alarm value, and the third level alarm value is determined, if the first abnormal value or the second abnormal value is the first level alarm value, the second level alarm value, or the third level alarm value, it represents that the system is abnormal, if any one of the above abnormal values does not exist, the system is normal, if the first level alarm value exists, the system is abnormal and the degree of abnormality is high, and if the third level alarm value exists, the system is abnormal and the degree of abnormality is low. Therefore, the system can be evaluated by associating the log data and the index data, and compared with the traditional method for evaluating the system only by means of the log or only by means of the index, the evaluation system is more comprehensive, the evaluation standard is more sound, and the evaluation accuracy is higher.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the embodiments or technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart diagram illustrating a method for evaluating a system based on logs and indexes provided in an embodiment of the present disclosure;

FIG. 2 illustrates a flow diagram of a method for determining a first outlier provided by embodiments herein;

fig. 3 is a flowchart illustrating a method for determining a log template provided by an embodiment herein;

fig. 4 is a flowchart illustrating a method for determining a first threshold of a log template according to an embodiment of the present disclosure;

FIG. 5 illustrates another schematic flow diagram of a method for determining a first outlier provided by embodiments herein;

FIG. 6 illustrates yet another schematic flow diagram of a method for determining a first outlier provided by embodiments herein;

FIG. 7 illustrates a flow diagram of a method for determining a second outlier provided by embodiments herein;

FIG. 8 is a flow chart illustrating a process for determining a second threshold corresponding to each index data according to a plurality of historical index data provided by an embodiment of the present disclosure;

fig. 9 shows a schematic flow chart of an evaluation result of a determination system provided by an embodiment herein;

FIG. 10 is a schematic block diagram illustrating a system evaluation apparatus based on logs and indexes according to an embodiment of the present disclosure;

fig. 11 shows a schematic structural diagram of a computer device provided in an embodiment herein.

Description of the figures the symbols:

100. a first outlier determination module;

200. a second outlier determination module;

300. a system evaluation module;

1102. a computer device;

1104. a processor;

1106. a memory;

1108. a drive mechanism;

1110. an input/output module;

1112. an input device;

1114. an output device;

1116. a presentation device;

1118. a graphical user interface;

1120. a network interface;

1122. a communication link;

1124. a communication bus.

Detailed Description

The technical solutions in the embodiments of the present invention will be described below clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the embodiments described are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the scope of protection given herein.

In order to solve the above problem, embodiments herein provide a system evaluation method based on a log and an index. Fig. 1 is a schematic diagram of steps of a system evaluation method based on logs and indexes provided in an embodiment of the present disclosure, and the present disclosure provides the method operation steps as described in an embodiment or a flowchart, but may include more or less operation steps based on conventional or non-creative labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In the actual implementation of the system or the device product, the method according to the embodiments or shown in the drawings can be executed in sequence or in parallel.

Referring to fig. 1, a system evaluation method based on logs and indexes may include:

s101: the real-time log data is compared to a log template to determine a first outlier.

S102: and comparing the plurality of items of real-time index data with the corresponding plurality of items of historical index data to determine a second abnormal value.

S103: and judging whether the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value.

S104: and if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value, determining that the evaluation result of the system is abnormal.

S105: and if the first abnormal value and the second abnormal value are not the primary alarm value, the secondary alarm value or the tertiary alarm value, determining that the evaluation result of the system is normal.

The log data is records generated in the running process of the system, and mainly records problems generated by hardware, software and the like in the system. The real-time log data may be log data obtained in real time, and the log data within a set time obtained in real time may be used as the real-time log data on the basis of any set time such as one minute or two minutes.

The multiple index data are some index values indicating the performance of the computer, and the main technical performance indexes of the computer include response time, memory capacity, access cycle, operation speed and the like. The plurality of index data may be, but is not limited to, corresponding time index data, memory capacity index data, access cycle index data, and operation speed index data. The multiple items of real-time index data may be multiple items of index data obtained in real time, and any set time such as one minute, two minutes and the like may be used as a reference, and the multiple items of index data within the set time obtained in real time are used as the multiple items of real-time index data. Since the system is often evaluated in units of one day, the plurality of items of historical index data are often a plurality of items of index data before the previous day, including the previous day.

The system abnormal degree represented by the first-level alarm value is the highest, and the system abnormal degree represented by the third-level alarm value is the lowest. And after the first abnormal value and the second abnormal value are determined through comparison, if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value, the system is abnormal, if any abnormal value does not exist, the system is normal, if the primary alarm value exists, the system is abnormal and the abnormal degree is high, and if the tertiary alarm value exists, the system is abnormal and the abnormal degree is low.

Therefore, the log data and the index data can be associated to evaluate the system, and compared with the traditional method for evaluating the system only by means of logs or indexes, the evaluation system is more comprehensive, the evaluation standard is more sound, and the evaluation accuracy is higher.

Referring to fig. 2, in this embodiment, the comparing the real-time log data with the log template to determine the first outlier may include:

s1011: comparing the real-time log data in unit time with log templates of different categories, and determining the category of the real-time log data in unit time.

S1012: and determining a first abnormal value according to the category of the real-time log data in unit time and a comparison result of the real-time log number in unit time under the category and a first threshold value of the corresponding log template, wherein the first threshold value is the threshold value of the log number of the corresponding log template in unit time.

Referring to fig. 3, further, the method for determining the log template may include:

s201: and acquiring historical log data, and performing word segmentation on each piece of historical log data.

S202: merging and extracting the historical log data after word segmentation through a clustering algorithm, and determining a log template corresponding to each type of log data.

Specifically, since the system is often evaluated on a one-day basis, the historical log data is often log data including the previous day before the previous day. In this context, historical log data over any period of time may be taken.

Because the log data is recorded and is a problem existing in the system, when each piece of historical log data is participated, firstly, the participation is to divide a complete piece of historical log data into a plurality of word segments. For example, a piece of history log data: the CPU occupancy rate is 76% ", and three word segments of CPU, occupancy rate and 76%" can be obtained after word segmentation. When each piece of historical log data is segmented, a word list can be set in advance according to actual work needs, the word list is used for storing words to be segmented, word segmentation can be performed through word segmenters such as jieba word segmentation, and for a plurality of pieces of historical log data after word segmentation, a plurality of word segments of the historical log data are obtained. Then, comparing a plurality of word segments of the segmented historical log data, combining and extracting a plurality of word segments corresponding to the historical log data of the same type after comparison, and further determining a log template corresponding to each type of log data, wherein the clustering algorithm can be any one of K-means and the like.

For example, the word segments of the two segmented historical log data are respectively: "CPU", "occupancy is", "76%"; "CPU", "occupancy is", "70%"; comparing two pieces of historical log data after being divided, extracting historical log data which are found to be the same type, and combining and extracting log templates corresponding to the log data as follows: "CPU", "occupancy" and "percentage". Therefore, log templates corresponding to different types of log data can be obtained, and each type of log template represents an error type of one type of system.

After determining the log templates, the real-time log data per unit time is compared to log templates of different categories. The unit time can be set to one minute or two minutes according to actual requirements, and the category of the real-time log data in the unit time and the number of the real-time logs in each category can be determined.

Furthermore, each type of log template has a corresponding first threshold, and the first threshold is a threshold of the number of logs in a unit time of the corresponding log template. Historical log data in a period of time can be taken, the historical log data are classified according to preset classification rules, or the historical log data are clustered as the historical log data to obtain log templates of different categories, and then the historical log data of a plurality of categories in unit time are obtained. The log number of each type of log data in a unit time may fluctuate, for example, 10 log data of type a in 0-1 minute, 9 log data of type a in 1-2 minutes, and 11 log data of type a in 2-3 minutes, and the first threshold is a reasonable number of log data of the corresponding type.

After real-time log data in unit time are obtained, the number of the real-time log data of a plurality of categories in unit time is further obtained according to a classification method of historical log data, if the system is not abnormal, the number of the real-time log data of a certain category in unit time is not more than a first threshold value, but if the system is abnormal, the number of the real-time log data of a certain category in unit time is possibly more than or less than the first threshold value, so that a first abnormal value can be determined according to a comparison result of the number of the real-time log data in unit time under the category of the real-time log data and the corresponding first threshold value, and whether the system is abnormal or not is judged through the log data.

Referring to fig. 4, in this embodiment, the method for determining the first threshold of the corresponding log template may include:

s301: and judging whether the log template is an abnormal log template.

S302: if the log template is the abnormal log template; the set threshold is determined as a first threshold corresponding to the number of logs of the log template in a unit time.

S303: and if the log template is not the abnormal log template, determining a first threshold value of the log number of the corresponding log template in unit time according to the number of the historical log data corresponding to the log template, wherein the first threshold value comprises a first upper threshold value and a first lower threshold value.

Specifically, for each type of log template, it needs to sequentially determine whether the log template is an abnormal log template, where the abnormal log template is some faults that may have an important influence on the system, and it is known to those skilled in the art that some faults that may have an important influence on the system exist in all log data, for example, faults such as failure of hard disk drive, and may be set as an abnormal log template: "hard disk", "drive", "fail". And setting an abnormal log template according to the actual production requirement, wherein all the abnormal log templates are stored in the abnormal log template, and comparing each type of log template with the log templates stored in the abnormal log template to judge whether the corresponding log template is the abnormal log template. If the log template is an abnormal log template, the set threshold can be determined as a first threshold corresponding to the log template, and the set threshold can be set according to actual working requirements, because the first threshold is a threshold corresponding to the number of logs in the log template in unit time, the first threshold (set threshold) of the abnormal log template can be usually set to 0, that is, if one (greater than 0) real-time log data corresponding to the abnormal log template exists, the real-time log data is considered as an abnormal log, and it can be determined that the system has an abnormality.

If the log template is not an abnormal log template, determining a first threshold of the log number of the corresponding log template in unit time according to the number of the historical log data corresponding to the log template, wherein the log number of each type of log template in unit time may fluctuate, for example, 10 log data in 0-1 minute A, 9 log data in 1-2 minutes A, 11 log data in 2-3 minutes A, the first upper threshold may be 11, and the first lower threshold may be 9. The first upper threshold is the number which the number of the logs in the unit time cannot exceed, the first lower threshold is the number which the number of the logs in the unit time cannot be less, and the number of the logs in the corresponding unit time of the log template needs to be within a reasonable interval range from the first lower threshold to the first upper threshold.

In addition, if the abnormal log template existing in the abnormal table is not in the abnormal log template of the system, the abnormal log template in the abnormal table can be added into the abnormal log template of the system, so that the log template of the system is enriched, and the monitoring efficiency and the monitoring effect on the abnormal log are improved.

Referring to fig. 5, further, the determining a first abnormal value according to the category to which the real-time log data belongs per unit time may include:

s1012a: and judging whether the category of the real-time log data in unit time is a category corresponding to any log template.

S1012b: and if the category of the real-time log data in unit time is not the category corresponding to any log template, newly adding the category of the real-time log data into the log template, and determining the first abnormal value as a secondary alarm value.

S1012c: and if the category of the real-time log data in the unit time is the category corresponding to any log template, determining a first abnormal value according to a comparison result of the number of the real-time logs in the unit time under the category and a first threshold value of the corresponding log template.

Specifically, for the real-time log data in a unit time, because there are a plurality of real-time log data, a plurality of real-time log data may correspond to different log templates, and each corresponding log template may generate a first abnormal value. For example, real-time log data in 0-1 minute corresponds to 5 log templates, each of which corresponds to a first outlier.

The method comprises the steps of acquiring real-time log data in unit time, for example, acquiring the real-time log data every other minute or every other two minutes, comparing the acquired real-time log data with a log template, and determining which log template each piece of real-time log data belongs to. During comparison, on the basis of the clustering algorithm, each piece of real-time log data is subjected to word segmentation to obtain a plurality of word segments, and then the log template to which the corresponding piece of real-time log data belongs is judged according to the distance between the word segments and the clustering center of each log template.

If the category to which a certain piece of real-time log data belongs in unit time is not the category corresponding to any log template, the category to which the real-time log data belongs needs to be added to the log template, and because the real-time log data is segmented and a plurality of word segments are obtained in the front, the word segments can be directly extracted to obtain the corresponding log template. Because the piece of real-time log data belongs to the newly added category, the system may have an abnormality at this time, and the first abnormal value is determined as a secondary alarm value.

Referring to fig. 6, further, the determining a first abnormal value according to a comparison result between the real-time log number in a unit time under the category and a first threshold of a corresponding log template may include:

s401: and judging that the first threshold value of the corresponding log template is the set threshold value, or the first upper threshold value and the first lower threshold value.

S402: and if the first threshold value of the corresponding log template is the set threshold value, determining the first abnormal value as a primary alarm value when the number of the real-time logs in the unit time under the category is greater than the first threshold value of the corresponding log template.

S403: if the first threshold of the corresponding log template is the first upper threshold and the first lower threshold, when the number of the real-time logs in the unit time under the category is greater than the first upper threshold or less than the first lower threshold, the first abnormal value may be determined as a third-level alarm value.

Specifically, if the category to which all real-time log data belong in a unit time corresponds to the category corresponding to any log template, the real-time log number of all log templates in different categories in the unit time needs to be determined, for example, the real-time log number in the unit time of a type a template is 1, the log number in the unit time of a type b template is 12, and the like.

In the process of determining the first abnormal value, whether the first threshold of the template A is a set threshold or a first upper threshold and a first lower threshold is judged, and if the first threshold of a certain type of template is the set threshold, the type of template is an abnormal log template. Assume that the first threshold of the class a template is a set threshold, and the first threshold of the class B template is a first upper threshold and a first lower threshold.

The first threshold value of the class A template is 0, the first upper threshold value of the class B template is 15, and the first lower threshold value is 5. Then 1 is compared to 0 and 12 is compared to 5-15, because 1 is greater than 0, the first outlier corresponding to the a template is determined to be the primary alarm value because 12 is in the range of 5-15 and the B template has no anomalies.

If the first threshold of the type A template is 0, the first upper threshold of the type B template is 10, and the first lower threshold is 5, then 1 is compared with 0, 12 is compared with 5-10, because 1 is greater than 0, the first abnormal value corresponding to the type A template is determined as a primary alarm value, 12 is greater than 10, and the first abnormal value corresponding to the type B template can be determined as a tertiary alarm value.

Referring to fig. 7, in this embodiment, the comparing the plurality of real-time indicator data with the corresponding plurality of historical indicator data to determine a second abnormal value may include:

s1021: and determining a second threshold corresponding to each index data according to the plurality of historical index data.

S1022: and comparing the multiple items of real-time index data in the unit time with the corresponding second threshold values respectively to determine second abnormal values.

Referring to fig. 8, further, the determining a second threshold corresponding to each index data according to multiple items of historical index data may include:

s1021a: and rejecting abnormal data in each item of historical index data.

S1021b: and determining a second upper threshold and a second lower threshold of each item of index data according to the distribution condition of each item of historical index data.

Specifically, abnormal data in each item of historical index data can be removed through an isolated forest algorithm, and the isolated forest algorithm can quickly find the abnormal data in the whole area. The plurality of index data may be, but is not limited to, corresponding time index data, memory capacity index data, access cycle index data, and operation speed index data. After the abnormal data is eliminated, a normal distribution method may be used, for example, for the memory capacity index data, historical memory capacity index data within a period of time may be taken, the mathematical expectation μ and the variance σ in the historical memory capacity index data within the period of time are determined, then according to the actual production needs, the second upper threshold is determined as μ +3 σ, the second lower threshold is determined as μ -3 σ, or the second upper threshold is determined as μ +2 σ, the second lower threshold is determined as μ -2 σ, and so on.

After the second upper threshold and the second lower threshold of each item of index data are determined, the plurality of items of real-time index data obtained in the unit time are respectively compared with the corresponding second upper threshold and second lower threshold.

In this embodiment, the comparing the plurality of items of real-time index data in unit time with the corresponding second threshold values respectively to determine second abnormal values may include:

and comparing multiple items of real-time index data in unit time with the corresponding second threshold respectively, and determining the second abnormal value as a three-level alarm value when each item of real-time index data is greater than the corresponding second upper threshold or less than the second lower threshold.

Through the method, the index data can be accurately analyzed.

In this embodiment, the determining that the evaluation result of the system is a system anomaly if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value, or a tertiary alarm value includes:

s1041: and if the first abnormal value or the second abnormal value is a primary alarm value, determining that the evaluation result of the system is an important one of system abnormalities.

S1042: and if the first abnormal value or the second abnormal value is a secondary alarm value, determining that the evaluation result of the system is a secondary one of system abnormalities.

S1043: and if the first abnormal value or the second abnormal value is a three-level alarm value, determining that the evaluation result of the system is the system abnormality corresponding to the abnormal type according to the magnitude relation between the times of the three-level alarm value and the first times and the second times.

The three steps of S1041, S1042 and S1043 are parallel steps, and have no precedence relationship. Because the types of the log templates in the system are numerous, real-time log data under each type of log template can generate a first abnormal value in unit time, so that a plurality of first abnormal values exist in unit time, and because the number of index data items in the system is numerous, each real-time index data can generate a second abnormal value, so that a plurality of second abnormal values exist in unit time. And the degree of abnormality is decreased from the primary alarm value to the tertiary alarm value, so that for each unit time, the evaluation result is determined as an important abnormality as long as one of the primary alarm values exists, and the evaluation result is determined as a secondary abnormality as long as one of the secondary alarm values exists, if no primary alarm value exists, corresponding to the first and second abnormal values. If neither the first level alarm value nor the second level alarm value exists, further judgment is needed.

Referring to fig. 9, specifically, the determining, according to a size relationship between the number of times of the three-level alarm value and the first number of times and the second number of times, that the evaluation result of the system is a system anomaly of a corresponding anomaly type may include:

s1043a: and judging the relationship between the times of the three-level alarm values and the first times and the second times.

S1043b: and if the times of the three-level alarm values are less than or equal to the second times, determining that the evaluation result of the system is suspected abnormality in system abnormality.

S1043c: and if the three-level alarm value is greater than the second time and less than the first time, determining that the evaluation result of the system is a secondary abnormality in system abnormalities.

S1043d: and if the three-level alarm value is greater than or equal to the first time, determining that the evaluation result of the system is an important exception in system exceptions.

And determining the type of the system abnormality according to the comparison between the number of times of the three-level alarm value and the first number of times and the second number of times. The abnormal degree of the important abnormality, the secondary abnormality and the suspected abnormality is from high to low, and for the abnormal with different degrees, response mechanisms with different degrees can be provided, for example, the important abnormality can be alarmed in time, the secondary abnormality is alarmed once every 1 hour, the suspected abnormality is alarmed once every 1 day, and the like. By the method, log monitoring and index monitoring can be associated, and evaluation accuracy is improved.

Based on the system evaluation method based on the log and the indexes, the embodiment of the text also provides a system evaluation device based on the log and the indexes. The apparatus may include systems (including distributed systems), software (applications), modules, components, servers, clients, etc. that employ the methods described herein in embodiments, in conjunction with any necessary apparatus to implement the hardware. Based on the same innovative concepts, embodiments herein provide an apparatus as described in the following embodiments. Since the implementation scheme of the apparatus for solving the problem is similar to that of the method, the specific apparatus implementation in the embodiment herein may refer to the implementation of the foregoing method, and repeated details are not described herein. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.

Specifically, fig. 10 is a schematic block diagram of an embodiment of a system evaluation apparatus based on logs and indexes provided in an embodiment of the present disclosure, and referring to fig. 10, the system evaluation apparatus based on logs and indexes provided in an embodiment of the present disclosure includes: a first outlier determination module 100, a second outlier determination module 200, a system evaluation module 300.

The first outlier determination module 100: the real-time log data is compared with the log template to determine a first outlier.

The second outlier determination module 200: and comparing the plurality of items of real-time index data with the corresponding plurality of items of historical index data to determine a second abnormal value.

The system evaluation module 300: and judging whether the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value.

And if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value or a tertiary alarm value, determining that the evaluation result of the system is abnormal.

Referring to fig. 11, based on the above-described system evaluation method based on logs and indexes, an embodiment herein further provides a computer device 1102, where the above-described method runs on the computer device 1102. Computer device 1102 may include one or more processors 1104, such as one or more Central Processing Units (CPUs) or Graphics Processors (GPUs), each of which may implement one or more hardware threads. The computer device 1102 may also include any memory 1106 for storing any kind of information, such as code, settings, data, etc., and in a particular embodiment a computer program that is run on the memory 1106 and on the processor 1104, which computer program, when executed by the processor 1104, may perform instructions according to the above-described method. For example, and without limitation, memory 1106 may include any one or combination of the following: any type of RAM, any type of ROM, flash memory devices, hard disks, optical disks, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent fixed or removable components of computer device 1102. In one case, when the processor 1104 executes the associated instructions, which are stored in any memory or combination of memories, the computer device 1102 can perform any of the operations of the associated instructions. The computer device 1102 also includes one or more drive mechanisms 1108, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any memory.

Computer device 1102 can also include an input/output module 1110 (I/O) for receiving various inputs (via input device 1112) and for providing various outputs (via output device 1114). One particular output mechanism may include a presentation device 1116 and an associated graphical user interface 1118 (GUI). In other embodiments, input/output module 1110 (I/O), input device 1112, and output device 1114 may also be excluded as just one computer device in a network. Computer device 1102 can also include one or more network interfaces 1120 for exchanging data with other devices via one or more communication links 1122. One or more communication buses 1124 couple the above-described components together.

Communication link 1122 may be implemented in any manner, e.g., through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. Communication link 422 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

Corresponding to the methods in fig. 1-9, the embodiments herein also provide a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, performs the steps of the above-described method.

Embodiments herein also provide computer readable instructions, wherein when executed by a processor, a program thereof causes the processor to perform the method as shown in fig. 1-9.

It should be understood that, in various embodiments herein, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments herein.

It should also be understood that, in the embodiments herein, the term "and/or" is only one kind of association relation describing an associated object, and means that there may be three kinds of relations. For example, a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter associated objects are in an "or" relationship.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided herein, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the elements may be selected according to actual needs to achieve the objectives of the embodiments herein.

In addition, functional units in the embodiments herein may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions in the present invention substantially or partially contribute to the prior art, or all or part of the technical solutions may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments herein. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

The principles and embodiments of the present disclosure are explained in detail by using specific embodiments, and the above description of the embodiments is only used to help understanding the method and its core idea; meanwhile, for a person skilled in the art, according to the idea of the present disclosure, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present disclosure should not be construed as a limitation to the present disclosure.

Claims

1. A system evaluation method based on logs and indexes is characterized by comprising the following steps:

comparing the real-time log data with the log template, and determining a first abnormal value according to whether the real-time log data belongs to any log template; the log data records the problems existing in the system:

comparing real-time log data in unit time with log templates of different categories, and determining the category of the real-time log data in unit time; wherein each type of log template represents an error type of one type of system;

determining a first abnormal value according to the category of the real-time log data in unit time and a comparison result of the real-time log number in unit time under the category and a first threshold value of a corresponding log template, wherein the first threshold value is the threshold value of the log number of the corresponding log template in unit time;

if the first abnormal value and the second abnormal value are not the primary alarm value, the secondary alarm value or the tertiary alarm value, determining that the system is normal according to the evaluation result of the system;

the log template determination method comprises the following steps:

2. The method of claim 1, wherein the determining the first threshold of the corresponding log template comprises:

judging whether the log template is an abnormal log template or not;

if the log template is the abnormal log template; determining the set threshold as a first threshold of the number of logs in unit time corresponding to the log template;

3. The method of claim 2, wherein determining a first outlier based on the category to which the real-time log data belongs per unit time comprises:

judging whether the type of the real-time log data in unit time is a type corresponding to any log template or not;

4. The method of claim 3, wherein determining the first outlier based on the comparison of the real-time log number per unit time in the category to the first threshold of the corresponding log template comprises:

5. The method of claim 1, wherein comparing the plurality of real-time metric data to the corresponding plurality of historical metric data to determine a second outlier comprises:

6. The method of claim 5, wherein determining the second threshold value corresponding to each index data according to a plurality of historical index data comprises:

rejecting abnormal data in each item of historical index data;

and determining a second upper threshold and a second lower threshold of each index data according to the distribution condition of each historical index data.

7. The method of claim 6, wherein the comparing the plurality of items of real-time index data in unit time with the corresponding second threshold values respectively to determine second abnormal values comprises:

8. The method of claim 1, wherein determining that the system is evaluated as abnormal if the first abnormal value or the second abnormal value is a primary alarm value, a secondary alarm value, or a tertiary alarm value comprises:

if the first abnormal value or the second abnormal value is a primary alarm value, determining that the evaluation result of the system is an important abnormality in system abnormalities;

and if the first abnormal value or the second abnormal value is a three-level alarm value, determining that the evaluation result of the system is the system abnormality corresponding to the abnormal type according to the magnitude relation between the times of the three-level alarm value and the first times and the second times.

9. The method according to claim 8, wherein the determining that the evaluation result of the system is the system anomaly of the corresponding anomaly type according to the magnitude relation between the number of times of the three-level alarm value and the first number of times and the second number of times comprises:

and if the three-level alarm value is greater than or equal to the first time, determining that the evaluation result of the system is important abnormality in system abnormality.

10. A log and index based system evaluation apparatus, the apparatus comprising:

a first outlier determination module: comparing the real-time log data with a log template, and determining a first abnormal value according to whether the real-time log data belongs to any log template; the log data records the problems existing in the system:

determining a first abnormal value according to the category of the real-time log data in unit time and a comparison result of the real-time log number in unit time under the category and a first threshold of a corresponding log template, wherein the first threshold is the threshold of the log number of the corresponding log template in unit time;

the determining method of the log template comprises the following steps:

11. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, performs the instructions of the method of any one of claims 1-9.

12. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor of a computer device, is adapted to carry out the instructions of the method according to any one of claims 1-9.