WO2025008219A1 - Computer-implemented method for reducing error logfiles of a system - Google Patents

Abstract

The invention relates to a computer-implemented method (100) for reducing error logfiles (10) of at least one system (1), and to an associated computer program product (8), a reducing system (5) and a system environment (200) comprising the reducing system (5) and at least one system (1).

Description

Title: Computer-implemented method for reducing error-indicating log files from a system

Description

The invention relates to a computer-implemented method for reducing error-indicating log files from at least one system and an associated computer program product, reduction system and environmental system with the reduction system and a system.

For many technical products, the use of computer programs or software is essential because it has become an integral part of these products. This is the case, for example, of machines used in industry, in particular machine tools, which require one or more computer programs to perform their functions. These may, for example, be machine tools that use laser technology for metalworking tasks and chip lithography, which often use particularly complex software applications. The technical products, including their computer programs, can be faulty and issue corresponding error messages. Error messages can also be false alarms, ie without product defects or real errors when using the product, e.g. the machine tool, or in the computer program. In order to provide the products with as few errors as possible, the products are tested in advance.

Testing is the process of interacting with the product to check that it meets the requirements and is free from product defects that could affect its functionality. The goal is to decide whether the product can be released or not. For small products, testing can be successfully performed manually, but as the size and complexity of the product increases, especially with the aforementioned machine tools, this task becomes impossible to perform manually. In this case, it is convenient to perform automatic testing of the product using a computer-based testing system.

However, automated tests themselves are prone to errors, and each failed test requires manual review to verify its result, i.e. to classify its cause as either a product defect or a false positive. False positives are due to test and infrastructure problems and not to defects in the product itself. Developers must examine the test reports, which consist of error messages, stack traces and log files, for example, to perform this verification. It is known from the state of the art to automate the test procedure described at the beginning. In US 2021/0287109 Al, a computer-implemented method for analyzing test errors using an AI model is described. The method first carries out clustering in which test errors are clustered into individual error clusters. The clustering result is then used to train the AI model in order to automatically identify the cause of new, unclassified errors.

Although the test procedures known from the state of the art can be used for troubleshooting and identification, they are not as suitable for preventing product defects as would be desirable. For example, it has been found that the known test procedures require a very large amount of data to be analyzed in log files, which places a very heavy load on the processor and data storage. In addition, because of the large amount of data or number of entries, log files can hardly be processed manually by service engineers when required, particularly if this is necessary in addition to computer-aided analysis. This means that defects can remain in the product which only appear later at the customer's site and lead to complications which can only be resolved with great effort through a new test phase. If, for example, a machine tool with various product defects is delivered to a customer after a supposedly successful test phase, downtimes caused by product defects can cause major production losses at the customer. The invention is therefore based on the object of increasing the reliability of the products, in particular to propose a particularly robust computer-implemented method for reducing error-indicating log files.

The object is achieved by a computer-implemented method for reducing error-indicating log files from at least one system. The method comprises the following steps:

- Reading the error-reporting log files, each of which contains several entries distributed over several lines,

- Removing predefined, error-insensitive characters from the error-indicating log files so that truncated log files are obtained, and

- Generate the reduced log files from the truncated log files.

As a result, the reduction method according to the invention thus provides a reduced log file which has been cleaned of error-free characters or, in other words, tokens, so that the log file has a smaller amount of data and file size. It can therefore be processed more easily both on a computer basis, i.e. by means of a processor, and manually, e.g. by a test engineer, in order to carry out error detection, analysis and correction.

Preferably, the reduction process according to the invention can comprise a plurality of further steps, as explained in more detail below. The reduction process according to the invention Reduction process can also be in the form of a

The data can be stored in a memory using a grouping method, as will be explained in more detail later. In such a grouping method, a database is created from different log file groups of different situations, in particular error situations. By assigning log files to the log file groups that include the same error situation, a robust database can be created by the inventive extraction of predefined, error-inconspicuous characters, which can be used for troubleshooting, analysis and correction.

The log files can be read in by a system, in particular a test system, which carries out one or more processes, for example system tests, service processes, etc., whereby several systems can also be used. The log files are error-indicating, i.e. they indicate an error, in particular that the corresponding system test was interrupted by an error message, which is noted in the log file over one or more lines as an entry or entries. The processes, for example system tests, can be carried out in particular on the products with the computer programs to be tested, in particular machines, for example machine tools. The system tests can be represented by a computer program that can be carried out by a corresponding processor, in particular on the product. A corresponding test sequence or a test phase of the test system can be carried out by a computer, in particular a processor, of the respective machine. The system, in particular a test system, thus enables a process, in particular a test of the Functionality of the machine, in particular in its functionality or with regard to possible errors at least in relation to one or more computer programs that can be executed on the machine. Such a test can in particular be designed as a UI (user interface) test, in particular GUI (graphical user interface) test, which can run on a corresponding system screen. The result of the test sequence can be a test report, whereby the test report can include the log file and possibly other attachments as a test report appendix. The method according to the invention can of course also comprise the step of the process, in particular testing or test sequence itself, in particular in the form of that described above.

The predefined, error-free characters can be, for example, N-grams, syllables, letters. Error-free means in particular that characters do not contain any information, at least no relevant information about errors, or, in other words, are inconspicuous with regard to possible errors. In particular, error-free can mean error-irrelevant. Such error-free characters can have been identified in log files that do not contain any errors. The predefined, error-free characters can in particular come from a created database. This database or the characters contained therein, for example N-grams, can be compared with the content or the entries, in particular by transforming the entries in the log files into characters, in particular N-grams, as explained in more detail later. To feed this database, in particular the data from the previously created The log file groups obtained from the grouping of the log files mentioned above can be used. The N-grams can be, for example, monograms, bigrams, trigrams, etc. and can be written in any language, e.g. German or English.

To generate the reduced log files, the predefined, error-free characters can be removed from the log files if they match, in particular deleted, and/or the log files shortened by the error-free characters can be output, in particular for further use.

In particular, before removing the predefined, error-free characters, the method can further comprise the step of transforming the entries of the log files into characters, in particular n-grams. This enables a quick and accurate comparison of the predefined, error-free characters with the log files.

The method can also further comprise the step of generating sequences from the error-free characters of the shortened log files that remain after removal in order to obtain the reduced log files. In this case, the log files transformed with characters are converted back into a character string of sequences. In particular, this makes it possible to obtain the reduced log files, which can then be used for further purposes, for example in other methods of processors for error investigation or by service technicians. Furthermore, the method can further comprise the step of merging the entries or sequences into blocks, wherein the blocks each comprise one or more lines. This is advantageous in order to carry out a subsequent similarity analysis between individual log files and thus to identify log files with similar error situations. The merging step can be carried out by calculating the longest substring or the longest repeated substring.

The method may further comprise the step of calculating similarity values between the different log files. The similarity value is a value relating to a similarity between two log files, which should in particular correspond to the human intuitive concept of similarity. This is in particular a real value, which can for example be restricted to an interval such as the unit interval, i.e. [0, 1], or a Boolean value. The similarity value can in particular be symmetrical in order to ensure that the similarity between two log files is always the same, regardless of the order in which they are determined using a similarity operator. It has been shown that the similarity calculation with blocks is particularly robust against events that are broken into several lines and can contain many words. Otherwise, the robustness is increased by the prior extraction of the predefined, error-free characters.

In particular, it can be provided that Jaccard similarities are calculated for the calculation of the similarity values. This metric quantifies the similarity between two sample sets. It is equal to the ratio between the number of elements in the intersection and the union of these sets. That is, the more elements they have in common, the greater the similarity. To calculate the similarity score, the presence or number of matching elements, such as lines, blocks, words, etc., between the compared log files can be compared.

It can be provided that the calculation of the similarity values of grouped log files is carried out with read-in, not yet grouped log files. These grouped log files have already undergone a similarity comparison and have been assigned to certain error situations. By accessing the log file groups created in this way, it is quick and easy to assign log files to a certain error situation. In this way, new error situations can also be quickly identified if a log file cannot be grouped into any previously created log file group, so that a new log file group can be created.

In particular, the method may further comprise the step of grouping the different reduced log files into at least two log file groups of different error situations based on the previously calculated similarity values. When the similarity values of the log files have been determined, it can be stated for the individual log files which log files they are most similar to or similar to, in order to assign or summarize log files with the same or similar error situations. An error situation is understood to mean at least one error or a combination of several errors that are the same or similar to one another. This refers in particular to the cause of the error, for example incorrect programming in a certain part of the computer program executed by the system, a certain category of hardware errors, etc. In order to group the log files or, in other words, cluster them, the similarity values determined in each case are used, in particular by comparing the similarity values with one another. Various grouping operations are possible here.

For example, it is possible for the similarity values to be compared with a grouping logic in order to group the different log files into at least two log file groups with different error situations. For example, to compare two log files each, the similarity values of the log files can be compared with individual grouping limits of the grouping logic and for each similarity value, the defined grouping limit can be used to determine whether there is sufficient similarity to assign the log files to the same error situation. The grouping logic can also be designed as a maximum value or a maximum requirement in such a way that it requires the maximum of individual similarity values of a log file with its compared log files. This can ensure, if required, that each log file is only grouped into one log file group and the sensitivity of a selected grouping limit is not chosen to be too small or too large. In particular, an artificial intelligence (AI) model, particularly one based on machine learning, can be used to continuously improve the grouping. Through machine learning, the AI model can select ever more optimized grouping limits, in particular different grouping limits for different error situations.

The result of the log file groups can be used in a variety of ways, and this use can be a further component of the method. For example, the result can be transmitted to the test system(s) carrying out the system test to improve testing, in particular in the form of a machine learning algorithm, to the product(s) being tested, for example to output or display the specific error situation on a corresponding screen or a control element of the product, to a service technician for rectifying product errors, etc. and used by this or that technician.

It can also be provided that at least one group-based metric is determined for each log file group, in particular a ratio of a number of log files of one of the error situations to a total number of log files. This metric can be used to very easily determine the most frequent error situation. This metric is preferably applied to the log files of a predetermined period of time, e.g. the log files of the last week. An error situation can describe both a product error and a false alarm. The metric can be used to Therefore, the most common product errors and false alarms can be determined. This allows the developers' work to be prioritized.

The method can also further comprise the step of combining the log files of at least one of the log file groups into a group error log file that is aggregated in particular with regard to the error situation. This means that only one aggregated group error log file is available, which can, for example, only contain error-relevant information about error causes, error images, etc. The group error log file can then be used to resolve errors more quickly.

In addition, the method can further comprise the step of determining an error criticality of the log file groups and in particular a prioritization of the log file groups according to the determined error criticality. In other words, an error severity of the log file groups is determined. The error criticality can be determined, for example, on the basis of one or more of the previously mentioned metrics. This allows prioritization during error correction in order to make the product, for example the machine tool, as fully functional or error-free as possible as quickly as possible.

Finally, it can also be provided that machine learning is used in one, several or all of the steps explained above. A corresponding Kl model can thus increase the robustness of the process, for example by better determining predefined, error-free characters, grouping limits, etc. The object mentioned at the outset is further achieved by a computer program product according to claim 12. The computer program product comprises instructions which, when the computer program product is executed by a computer, cause the computer to carry out the method according to the invention.

The computer program product may be a computer program as such or a computer-readable storage medium on which the computer program or its instructions are stored. The storage medium may be a tangible device that can store and save instructions for use by an instruction execution device, i.e. a computer, in particular a processor. The computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the aforementioned devices, but is not limited thereto. A non-exhaustive list of more specific examples of a computer-readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch cards or raised structures in a groove with recorded data commands, and any suitable combination of the foregoing. A computer-readable storage medium, as used herein, should not be understood to mean transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., pulses of light traveling through a fiber optic cable), or electrical signals transmitted through a wire.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to the respective computing/processing devices or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing/processing device.

The computer-readable program instructions for carrying out operations within the scope of the present invention may be assembler instructions, ISA instructions (instruction set architecture), machine instructions, machine-dependent instructions, microcode, firmware instructions, state data, configuration data for integrated circuits, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++, or similar, and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter case, the remote computer may be connected to the user's computer over any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g. over the Internet using an Internet service provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field programmable gate arrays (FPGA), or programmable logic arrays (PLA), may execute the computer readable program instructions by using state information of the computer readable program instructions to personalize the electronic circuits to perform aspects of the present invention.

The object mentioned at the outset is also achieved by a reduction system according to claim 13. The reduction system according to the invention comprises a memory and a processor connected to the memory, wherein the computer program product according to the invention is stored on the memory and the processor is configured to execute the instructions of the computer program product.

The reduction system may have at least one interface to at least one system, wherein the at least one system is configured to generate log files.

Finally, the object mentioned at the outset is achieved by a system environment according to claim 15, comprising the reduction system according to the invention and the at least one system.

In this case, features which are described herein with respect to the method according to the invention also apply with respect to the computer program product according to the invention, the reduction system and the system environment, and vice versa.

Further details and advantageous embodiments of the invention can be found in the following description, on the basis of which embodiments of the invention are described and explained in more detail.

They show:

Figure 1 is a schematic view of a method according to an embodiment of the invention;

Fig. 2 - 7 schematic views of the steps of the

method of Fig. 1 ; and Figure 8 is a schematic view of a system environment according to an embodiment of the invention, comprising a reduction system according to an embodiment of the invention and several systems.

In the following description and in the figures, the same reference symbols are used for identical or corresponding features.

Figure 1 shows purely schematically the steps 102 to 118 of a computer-implemented method 100 for reducing and grouping error-identifying log files 10 (see Figs. 2 to 7).

A step 102 that can be attributed to the method 100 or is separate from it is the testing or system test of a system 1, in particular a test system (see Fig. 8). Alternatively, other processes of the system are also possible, for example a service process, whereby only testing is discussed below as an example and the method 100 can be carried out and used analogously for such other processes. A product is tested by such a system 1, in particular a computer program product on a machine, for example a machine tool. Each of the systems 1 from Fig. 8 can be used on one or more products (not shown), e.g. computer programs for machine tools. The testing in step 102 can otherwise be carried out according to known principles, in particular as an automated testing of the respective system 1. For example, GUI tests can be used. The result of the test is in each case the output of a log file 10. Other system processes, such as a

Service process, such a log file 10 is output accordingly.

During a system test, a large number of events and operations take place. Although most of them are routine in the system, some can be used to track down the cause of a particular problem. Log files 10 are machine-generated files that record all these operations. They can also be described as containers of extensive and varied information that can be valuable for troubleshooting. Although there are no standards for their format and size, they are typically structured in entries EI, where each entry EI comprises one or more lines ZE (see Figs. 2 to 6) and reports a specific event together with a timestamp indicating the exact time of the event and, if necessary, even identification numbers and paths (not shown here).

The length of the log files 10 in the dataset can vary greatly. The log file length depends directly on the type of test. Some tests are very short and simply start and stop the test sequence, while others are longer and perform several consecutive subtests to test the endurance of the product. Among the large number of existing log files 10, client and server logs are preferred for analysis, as they have proven to be particularly informative for machines, especially machine tools. These files are similar in format, but differ in the The client protocol is responsible for capturing the events on the user interface of the machine under test or its processor, while the server protocol collects the information about the process in the backend of the same processor.

The log files 10 generated during the system test(s) in step 102 are read in step 104. In particular, only the log files 10 with errors can be read in, since only these contain an error message and are relevant for error analysis in order to increase the robustness of the products.

Protocol files 10 are a subformat of a text file that is attached to the test report. They can also be referred to as log files. In Figs. 2 to 6, the schematically illustrated protocol files 10 are also referred to as "Log 1" and "Log 2" and are thus distinguished from one another.

Figure 2 shows an example of two log files 10 placed side by side that are to be reduced and grouped. They each have a large number of entries EI that are distributed over different lines ZE.

In step 106 of the method 100, the entries EI of the two log files 10 are now transformed into characters 20, in this case particularly N-grams, for example monograms. The result of the transformation for the two log files 10 can be seen in Fig. 3. For the transformation into the characters 20 in the form of N-grams, a corresponding lexicon with various characters 20, in particular N-grams, e.g. in German and/or English, e.g. monograms, Bigrams, trigrams, etc. N-grams are understood to mean at least monograms, bigrams and trigrams.

In step 108 of the method 100, predefined characters 20 are now removed from the log files 10, in particular deleted, as illustrated in Fig. 4, so that shortened log files 10 are obtained. The predefined characters 20 can, for example, come from a corresponding negative list. The negative list can include characters 20 that are error-free or, in other words, not error-relevant or error-irrelevant. The negative list can, for example, be generated by using the log file groups 40 explained in more detail later.

In step 110 of the method 100, sequences are generated from the remaining characters 20 of the log files 10, as shown in Fig. 5. To do this, the existing characters 20 of each line ZE are each concatenated to form a sequence. Reduced log files 10 can now be output, which have been cleaned of error-irrelevant content and thus offer a better overview of the error pattern of the log file 10. In particular, the file size of the log files 10 is significantly reduced, which is gentle on the processor and data memory and thus allows faster error detection.

In step 112 of the method 100, the entries EI or sequences are combined into blocks BL, wherein the blocks BL each comprise one or more rows ZE, as illustrated in Fig. 6. The combination in step 112 can for example by calculating the longest substring or the longest repeated substring.

Following step 112, in step 114 of the method 100, the two log files 10 are compared with each other to calculate their similarity. For the sake of robustness of the method 100, the similarity is carried out by using the blocks BL. To be precise, in step 114 of the method 100, a similarity value 30 is calculated based on the blocks BL of each of two different log files 10. This is illustrated schematically in Fig. 7.

In Figure 7, the two log files 10 read in and processed according to steps 104 to 112 are compared with each other by a similarity operator 20. Alternatively or additionally, the log files 10 can also be compared with already grouped log files 10 from log file groups 40 with different error situations 41, 42 in order to enable the log files 10 to be grouped based on their error pattern, as illustrated in Figure 7 by the arrow reference of the log file groups 40 to the similarity operator 20.

The similarity operator 20 can be designed in particular as a Jaccard similarity operator of the presence of the blocks BL of the log files 10 compared with each other. In this case, the number of blocks of both log files 10 is compared with each other. This provides the respective similarity value 30 for each compared pair of log files 10. In order to improve the results, the respective blocks BL can optionally be weighted, e.g. based on their time of occurrence in the log files 10 or by determining a Tf-idf measure (statistical measure with frequency of occurrence and inverse document frequency).

For grouping the log files 10 in step 116, the similarity values 30 are compared using a grouping logic 35, which can in particular have one or more grouping limit values. The grouping logic 35 can, for example, be a real value or a requirement, e.g. for the highest similarity value 30 from the comparisons with different log files 10. If the grouping limit value required by the grouping logic 35 is met by the respective similarity value 30, the grouping takes place in the corresponding log file group 40 of different error situations 41, 42.

The result of the log file groups 40 can now be used in a variety of ways, with the optional step 118 being shown as an example in the further course of the method 100 in order to increase the possible uses.

For example, the result can be sent to the system(s) 1 to improve testing, in particular in the form of a machine learning algorithm, to the product(s) tested, for example to output or display the specific error situation on a corresponding screen or a control element of the product, to a service technician for rectifying product errors, etc. transmitted to and used by this or that person.

For example, in step 118 of the method 100, at least one group-based metric can be determined for each of the log file groups 40, in particular a ratio of a number of log files 10 of one of the error situations 41, 42 to a total number of log files 10.

Finally, Figure 8 schematically shows a system environment 200 which shows a reduction system 5 (and grouping system) according to an embodiment of the invention. The reduction system 5 here comprises a processor 6 and a memory 7 on which a computer program product 8 is stored, which comprises instructions which, when the computer program product 8 is executed by the processor 6, cause the processor 6 to carry out the method 100 according to Figure 1.

In order to carry out the method 100, the computer program product 8 requires the log files 10 of the system(s) 1. The system(s) 1 are part of the comprehensive system environment 200 shown, which also includes the reduction system 5. The reduction system 5 has an interface 9 through which it can receive the log files 10 from the systems 1. Of course, the systems 1 can include a corresponding interface (not shown) for data communication. The transmission can be wired or wireless in the case of physical interfaces, for example. The systems 1 each have a Test processor 2 and a test memory 3 on which a test computer program 4 is stored for carrying out step 102 of the method 100. The system or systems 1 can each be the products themselves, with the test computer program 4 being executed thereon in order to test the products or their computer programs.

As an alternative to the representation in Fig. 5, the system 1 and the reduction system 5 can also be implemented on one of the two systems 1, 5, for which only one memory 7 and one

Processor 6 can be present, whereby both computer programs 4, 8 can be stored in the memory 7. The interface 9 can be implemented by computer program code.

Claims

patent claims 1. Computer-implemented method (100) for reducing error-indicating log files (10) from at least one system (1), the method (100) comprising the following steps (104, 108): - Reading the error-reporting log files (10), each of which contains several entries (EI) distributed over several lines (ZE), - removing predefined, error-inconspicuous characters (20) from the error-indicating log files (10) so that shortened log files (10) are obtained, and - Generating the reduced log files (10) from the truncated log files (10) . 2. The method (100) according to claim 1, wherein the method (100) further comprises the step (106) of transforming the entries (EI) of the log files (10) into characters (20), in particular N-grams, prior to removing the predefined, error-free characters (20). 3. The method (100) of claim 2, wherein the method (100) further comprises the step (110) of generating sequences from the characters (20) of the shortened log files (10) remaining after the removal to obtain the reduced log files (10). 4. The method (100) according to any one of the preceding claims, wherein the method (100) further comprises the step (112) of assembling the entries (EI) or the sequences into blocks (BL), wherein the blocks (BL) each comprise one or more rows (ZE). 5. The method (100) of claim 4, wherein the step (112) of merging is performed by calculating the longest substring or the longest repeated substring. 6. The method (100) of claim 4 or 5, wherein the method (100) further comprises the step (114) of calculating similarity values (30) between the different log files (10). 7. The method (100) according to claim 6, wherein Jaccard similarities are calculated for calculating the similarity values (30). 8. The method (100) according to claim 6 or 7, wherein the calculation of the similarity values (30) of grouped log files (10) is carried out with read-in, not yet grouped log files (10). 9. The method (100) according to any one of claims 6 to 8, wherein the method (100) further comprises the step (116) of grouping the different reduced log files (10) into at least two log file groups (40) of different error situations (41, 42) based on the previously calculated similarity values (30). 10. The method (100) according to claim 9, wherein the similarity values (30) are compared with a grouping logic (35) in order to group the different log files (10) into the at least two log file groups (40) of different error situations (41, 42). 11. The method (100) according to claim 9 or 10, wherein at least one group-based metric is determined for each of the log file groups (40), in particular a ratio of a number of log files (10) of one of the error situations (41, 42) to a total number of log files (10). 12. Computer program product (8) comprising instructions which, when the computer program product (8) is executed by a computer, cause the computer to carry out the method (100) according to one of the preceding claims. 13. Reduction system (5) comprising a memory (7) and a processor (6) connected to the memory (7), wherein the computer program product (8) according to claim 12 is stored on the memory (7) and the processor (6) is configured to execute the instructions of the computer program product (8). 14. Reduction system (5) according to claim 13, further comprising at least one interface (9) to at least one system (1), wherein the at least one system (1) is configured to generate log files (10). 15. System environment (200) comprising the reduction system (5) according to claim 14 and the at least one system (1).