WO2025246269A1 - Test code repairing method and related device - Google Patents

Abstract

The present application discloses a test code repairing method applied to a code development platform. The method comprises: obtaining a test code corresponding to a source code, and then executing the test code to obtain execution information of the test code, wherein the execution information comprises abnormal stack tracking information output during execution of the test code; on the basis of the execution information, performing fault localization on the test code to obtain location information of a faulty code; on the basis of the location information of the faulty code, extracting context information, wherein the context information comprises at least one of a faulty test case, a fault description, or the faulty content; and inputting a prompt constructed on the basis of the context information into a language model and performing inference, so as to obtain a first repaired code. The method is used to perform fault localization in light of the execution information, extract the context information on the basis of the location information of the faulty code, and, on the basis of the context information, generate accurate and complex repaired code via an advanced language model, thereby adapting to various complex fault scenarios and improving repair efficiency and accuracy.

Description

A test code repair method and related equipment

This application claims priority to Chinese Patent Application No. 202410683284.2, filed on May 29, 2024, entitled "A Unit Test Repair Method and Related Equipment", and to Chinese Patent Application No. 202410961426.7, filed on July 17, 2024, entitled "A Test Code Repair Method and Related Equipment", the entire contents of which are incorporated herein by reference.

Technical Field

This application relates to the field of artificial intelligence (AI) technology, and in particular to a test code repair method, a code development platform, a computing device cluster, a computer-readable storage medium, and a computer program product.

Background Technology

As computer technology continues to evolve, software is becoming increasingly complex, and the requirements for software quality are also rising. As a fundamental means of ensuring software quality, software testing, especially unit testing, is crucial for early detection and fix of errors. Although software testing is extremely valuable, test code may contain errors, which could be caused by logical flaws, coding mistakes, or environment configuration issues. Incorrect test code not only fails to accurately detect problems in the software but may also lead developers to make incorrect code modifications, resulting in decreased software quality or even introducing new errors.

As software grows in size and complexity, manually writing and maintaining test code requires developers to invest significant time in checking and fixing errors. This not only reduces development efficiency but also increases the overall cost of the software. Developing automated test code fixing techniques can help alleviate the burden on developers and improve the efficiency and accuracy of testing.

Currently, most automated test code repair methods rely on manual intervention or heuristic algorithms, resulting in low repair efficiency and insufficient accuracy.

Summary of the Invention

This application provides a test code repair method. This method locates errors by combining execution information, extracts context information based on the location of the error code, and generates more accurate and complex repair code using an advanced language model based on this context information. This adapts to various complex error scenarios, is not limited to specific error types, and thus improves repair efficiency and accuracy. This application also provides a code development platform, computing device cluster, computer-readable storage medium, and computer program product corresponding to the above method.

Firstly, this application provides a method for repairing test code. This method can be executed by a code development platform. Code development platforms are typically used to develop and maintain test code. Code development platforms can be provided to users in different forms. For example, a code development platform can be a standalone software system, including but not limited to an integrated development environment (IDE) or a code editor. In this form, the developer can provide the code development platform software package to the user, who can then deploy it themselves, for example, on a local computing device, to achieve comprehensive repair of the test code. Alternatively, the code development platform can be integrated into other software or platforms, for example, as a plugin or component. This allows for a seamless repair experience for users through an embedded solution. Furthermore, the code development platform can be deployed in a cloud environment and provided to users for subscription as a cloud service. The cloud environment can be a cloud computing cluster formed by multiple computing devices, including but not limited to a central cloud (a cloud computing cluster formed by central computing devices) and an edge cloud (a cloud computing cluster formed by edge computing devices). The cloud service subscription model allows users to access the corresponding cloud service via the Internet to execute the test code repair method of this application, without the need for local software package installation, offering high flexibility and scalability. In some possible implementations, the code development platform can also be a hardware system with test code development capabilities, such as a cluster of computing devices with test code development capabilities. When the computing device cluster is running, it executes the test code repair method of this application.

Specifically, the code development platform can obtain the test code corresponding to the source code. This test code is used to test the functionality and/or performance of the source code. The platform then executes the test code and obtains its execution information, including stack traces of any exceptions output during execution. The platform can then locate errors in the test code based on this information, obtaining the location of the erroneous code. Based on this location, it extracts contextual information, including at least one of the following: erroneous test cases, error descriptions, or error content. The platform can then use a language model built based on this contextual information to reason and obtain a first corrective code. This first corrective code is obtained by the language model correcting the erroneous code in the test code.

This method first locates errors by combining the execution information of the test code, then extracts contextual information based on the location of the error code. Based on this contextual information, it generates more accurate and complex repair code using an advanced language model to adapt to various complex error scenarios, not limited to specific error types. This reduces manual intervention, improves the automation level of test code repair, and thus improves repair efficiency and accuracy, accelerates the iteration speed of software development and testing, and enhances the overall quality of the software. Furthermore, this method can repair different error types, demonstrating strong versatility and adaptability.

In some possible implementations, the code development platform can also execute the first fix code or test code fixed using the first fix code. Specifically, if the first fix code is the full code, the code development platform can execute it. If the first fix code is incremental code, the code development platform can execute the test code fixed using the first fix code, such as inserting a code block containing the first fix code into the test code, or replacing an erroneous code block with the first fix code in the test code. If execution fails, the code development platform presents the first fix code to the user, receives user feedback on the first fix code, and obtains an updated first fix code.

In addition to automatically repairing test code, this method also introduces a manual repair channel, allowing users to provide feedback such as modifying the first repair code, thereby further improving the accuracy and efficiency of the repair.

In some possible implementations, the code development platform can also determine the error type of the error code, fix the error code according to the corresponding repair template, and obtain a second repair code. Correspondingly, if the second repair code fails to fix the error, the code development platform can extract context information based on the location information of the error code.

This method combines template repair and language model repair techniques to achieve comprehensive repair of test code. Template repair relies on pre-defined repair templates to quickly locate and fix common error types, while language model repair can generate more precise and complex repair solutions to handle complex error scenarios.

In some possible implementations, the error type of the error code includes assertion failure. Accordingly, the code development platform can first determine the assertion type of the erroneous assertion in the error code. When the assertion type is an equality type, the code development platform extracts the expected value of the erroneous assertion from the error code, extracts the actual value of the erroneous assertion from the test report, and then replaces the expected value in the assertion parameter of the erroneous assertion with the actual value according to the assertion repair template corresponding to the assertion type, thus obtaining the second repair code.

In this method, for erroneous assertions of the equality type, the code development platform can use the original assertion template as the assertion repair template to replace the assertion parameters of the erroneous assertion, such as replacing the expected value with the actual value, thereby achieving rapid repair of erroneous assertions of the equality type.

In some possible implementations, when the assertion type is Boolean, the code development platform can generate second repair code based on the assertion parameters in the error assertion and the second assertion repair template corresponding to the first assertion repair template used by the error assertion, wherein the Boolean logic of the second assertion repair template is opposite to that of the first assertion repair template.

In this method, for Boolean type error assertions, the code development platform can quickly fix Boolean type error assertions by changing the template (for example, changing the assertion repair template used by the error assertion to an assertion repair template with the opposite Boolean logic) while keeping the assertion parameters unchanged.

In some possible implementations, the error type of the error code includes an exception, and the second fix code may include the exception handling code block corresponding to the exception. The code development platform can determine the exception type from the exception's stack trace information, and then fill the exception type into the exception fix template to obtain the exception handling code block. When the second fix code is incremental code, it may include the aforementioned exception handling code block. When the second fix code is full code, it may include test code and the exception handling code block; for example, the code development platform inserts an exception handling code block after the error code in the test code to obtain the second fix code.

This method addresses error codes related to exception types. By filling the exception type into an exception repair template, exception handling is achieved, thus enabling rapid repair of error codes of exception types. It should be noted that the exception repair template of this method supports rapid repair of various exception types, and is not limited to specific exception types such as null pointer exceptions, thus possessing high usability.

In some possible implementations, the code development platform can also compile test code to obtain compilation information. This compilation information includes error logs output by the compiler, which record the error type of the erroneous code. When the error type includes a missing class reference, the second fix can include import statements. The code development platform can determine the missing class name and construct a symbol index based on project dependencies, development tool libraries, or third-party libraries. This symbol index includes the fully qualified names of accessible classes. The platform then matches the missing class name with the symbol index to obtain the fully qualified name corresponding to the missing class name. The code development platform can insert this fully qualified name into the import fix template to obtain the import statement. It should be noted that the second fix can be incremental code, in which case it can include the aforementioned import statements. Alternatively, the second fix can be full code, in which case it can include both test code and import statements. For example, the code development platform can insert import statements into the test code to obtain the second fix.

This method also supports generating import statements using import repair templates for compilation error codes such as missing class references, thereby enabling rapid repair of error codes with missing class references.

In some possible implementations, if the first code fix fails, the code development platform can re-execute the code fix for the test code until the test code is successfully fixed or the number of retries reaches the target value.

This method supports setting a maximum number of retries, and the code development platform can perform multiple repairs on the test code based on the language model, thereby improving the repair success rate and the degree of automation.

Secondly, this application provides a code development platform. The code development platform includes:

The test code selection module is used to obtain the test code corresponding to the source code, and the test code is used to test the functionality and/or performance of the source code;

The execution module is used to execute the test code and obtain the execution information of the test code, including the stack trace information of the exceptions output when the test code is executed;

The error location module is used to locate errors in the test code based on the execution information and obtain the location information of the error code.

The model repair module is used to extract context information based on the location information of the error code. The context information includes at least one of the error test case, error description, or error content. The module then uses a prompt input language model built based on the context information to perform reasoning to obtain a first repair code. The first repair code is obtained by the language model repairing the error code in the test code.

In some possible implementations, the code development platform also includes a feedback module:

The running module is also used to execute the first repair code or the test code repaired using the first repair code;

The feedback module is used to present the first repair code to the user when the execution fails, receive feedback from the user on the first repair code, and obtain the updated first repair code.

In some possible implementations, the code development platform also includes:

The template matching and repair module is used to determine the error type of the error code, and repair the error code according to the repair template corresponding to the error type to obtain a second repair code;

The model repair module is specifically used for:

If the second repair code fails to repair, the context information is extracted based on the location information of the error code.

In some possible implementations, the error type of the error code includes assertion failure;

The template matching and repair module is specifically used for:

Determine the assertion type of the error assertion in the error code;

When the assertion type is an equality type, extract the expected value of the error assertion from the error code and extract the actual value of the error assertion from the test report;

In the assertion repair template corresponding to the assertion type, the expected value in the assertion parameter of the erroneous assertion is replaced with the actual value to obtain the second repair code.

In some possible implementations, the template matching repair module is further used for:

When the assertion type is Boolean, a second repair code is generated based on the assertion parameters in the error assertion and the second assertion repair template corresponding to the first assertion repair template used by the error assertion. The Boolean logic of the second assertion repair template is the opposite of the Boolean logic of the first assertion repair template.

In some possible implementations, the error type of the error code includes an exception, and the second repair code includes an exception handling code block corresponding to the exception;

The template matching and repair module is specifically used for:

Determine the exception type from the stack trace information of the exception;

Fill the exception type into the exception repair template to obtain the exception handling code block.

In some possible implementations, the code development platform also includes:

A compilation module is used to compile the test code and obtain compilation information of the test code. The compilation information includes error logs output by the compiler when compiling the test code, and the error logs record the error types of the error codes.

When the error code includes a missing class reference, the second repair code includes an import statement;

The template matching and repair module is specifically used for:

Determine the missing class name;

Based on project dependency libraries, development tool libraries, or third-party libraries, construct a symbol index, which includes the fully qualified names of accessible classes;

The missing class name is matched with the symbol index to obtain the fully qualified name corresponding to the missing class name;

Insert the fully qualified name into the import package repair template to obtain the import statement.

In some possible implementations, the model repair module is also used for:

If the first repair code fails to repair, the code repair is re-executed for the test code until the test code is successfully repaired or the number of retries reaches the target value.

Thirdly, this application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory to cause the computing device or the computing device cluster to perform the test code repair method as described in the first aspect or any implementation thereof.

Fourthly, this application provides a computer-readable storage medium storing instructions that instruct a computing device or a cluster of computing devices to execute the test code repair method described in the first aspect or any implementation thereof.

Fifthly, this application provides a computer program product containing instructions that, when run on a computing device or a cluster of computing devices, causes the computing device or cluster of computing devices to execute the test code repair method described in the first aspect or any implementation thereof.

Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods.

Attached Figure Description

To more clearly illustrate the technical methods of this application, the accompanying drawings used will be briefly described below.

Figure 1 is a schematic diagram of the architecture of a code development platform provided in this application;

Figure 2 is a flowchart of a test code repair method provided in this application;

Figure 3 is a schematic diagram of a code editing interface provided in this application;

Figure 4 is a schematic diagram of a code editing interface provided in this application;

Figure 5 is a schematic diagram of a prompt template provided in this application;

Figure 6 is a schematic diagram of test code repair based on assertion repair template provided in this application;

Figure 7 is a schematic diagram of test code repair based on assertion repair template provided in this application;

Figure 8 is a schematic diagram of test code repair based on an exception repair template provided in this application;

Figure 9 is a schematic diagram of test code repair based on an exception repair template provided in this application;

Figure 10 is a schematic diagram of the structure of a computing device provided in this application;

Figure 11 is a schematic diagram of the structure of a computing device cluster provided in this application;

Figure 12 is a schematic diagram of another computing device cluster provided in this application;

Figure 13 is a schematic diagram of another computing device cluster provided in this application.

Detailed Implementation

The terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

First, some technical terms involved in the embodiments of this application will be introduced.

Software testing is a technique that uses test code to check whether software (or a software system) meets user requirements. Software testing can be divided into several phases, such as unit testing, integration testing, system testing, and acceptance testing. Unit testing refers to testing the smallest testable unit in the software, which can be a function, a method, or a class. Unit testing is usually performed during the development process and aims to verify whether the module's functionality is correct. Integration testing is the process of verifying whether the interaction between multiple modules works as expected. Integration testing aims to discover dependencies and communication problems between modules. System testing is performed on the entire software system and aims to verify whether the entire software system works as required and expected by the user. Acceptance testing, also known as delivery testing, is usually the last testing operation before deploying the software. The purpose of acceptance testing is to ensure that the software is ready and can be used by end users to perform the software's intended functions and tasks.

Test code includes test classes and test methods to implement testing functionality. To improve code development efficiency, test code can be developed based on a testing framework. Taking Java as an example, Java supports the development of unit test code using multiple testing frameworks, with JUnit being a mainstream example. Therefore, when developing test code, you can import the JUnit package, create test classes, and write test methods within those classes. Test methods can use methods such as `assertEquals()`, `assertTrue()`, or `assertFalse()` to check whether the tested software meets expectations. Test classes and test methods can be annotated with appropriate tags.

For unit testing, integration testing, or system testing, test code can include test case code. A test case describes a specific software testing task and typically includes the test objective, test environment, test inputs, test steps, and expected results. Test cases are used to verify whether the software under test meets a specific software requirement. For example, executing the test case code in the test environment obtains the actual output corresponding to the test inputs. Based on the actual output and expected results, the test case verifies whether the software under test meets the test objective (e.g., a specific software requirement).

As software grows in size and complexity, manually writing and maintaining test code becomes increasingly difficult. Developers need to invest significant time and effort in checking and fixing errors in the test code. This reduces development efficiency and increases the overall cost of the software. To address this, the industry has proposed several automated test fixing solutions to reduce the workload of developers in creating and maintaining test code, thereby improving the efficiency and accuracy of software testing.

Automated test remediation solutions typically rely on manual intervention or heuristic methods. An example of an automated unit test remediation method for Null Pointer Exception (NPE) is provided. In this example, after the test verification module starts the test case, it verifies whether the test passes. If no execution exception is detected during the test case execution, the test passes and the process ends. If the test fails, the process jumps to the exception handling module. The exception handling module captures the type of the exception detected during the test, determines whether the exception is an NPE, and if it is, the process jumps to the NPE handling module; otherwise, it outputs the exception information for developers to manually fix. The NPE handling module automatically handles the captured NPE exception, and after processing and fixing it, it executes the test verification again. If no exception is found, the test passes; if an exception is found, the above steps are repeated until the exception is resolved or thrown.

The methods described above typically only detect and fix a limited number of error types. For example, automated unit test remediation methods for Non-Proof-of-Specification (NPE) errors focus on resolving NPE issues in unit tests. However, software testing can encounter various types of errors, such as assertion failures, type mismatches, and array out-of-bounds errors. Test remediation schemes based on manual intervention or heuristic methods lack versatility and flexibility, limiting their application scope. Furthermore, because these schemes primarily detect specific error types and perform targeted remediation, their remediation efficiency is low, and their accuracy is not high.

In view of this, this application provides a method for repairing test code. This method can be executed by a code development platform. A code development platform is typically used to develop and maintain test code. The code development platform can be a standalone software system, such as an integrated development environment (IDE) or a code editor. The code development platform can be provided to users as a software package, which users can deploy themselves, for example, on a local computing device, to achieve comprehensive repair of the test code. The local computing device is a computing device directly controlled by the user, including but not limited to personal computers (PCs), laptops, and personal workstations. The code development platform can also be integrated into other software or platforms, for example, as a plugin or component. This allows for a seamless repair experience for users through an embedded solution. The code development platform can also be deployed in a cloud environment and provided to users as a cloud service for subscription. The cloud environment can be a cloud computing cluster formed by multiple computing devices, including but not limited to a central cloud (a cloud computing cluster formed by central computing devices) and an edge cloud (a cloud computing cluster formed by edge computing devices). The cloud service subscription model allows users to access corresponding cloud services via the internet to execute the test code repair method of this application, without the need for local software package installation, offering high flexibility and scalability. In some possible implementations, the code development platform can also be a hardware system with test code development capabilities, such as a cluster of computing devices capable of test code development. When the computing device cluster is running, it executes the test code repair method of this application.

Specifically, the code development platform obtains the test code corresponding to the source code. This test code is used to test the functionality and/or performance of the source code. The platform then executes the test code to obtain execution information, including stack trace information of any exceptions output during the execution of the test code. Stack trace information, also known as stack backtracking information, describes the active stack frames at a specific point in time during the program's (e.g., the test code's) execution. Stack frame information can include the exception type, the location of the exception (e.g., class name, method name, line number), and the method call chain that caused the exception. The code development platform can then locate the error in the test code based on the execution information, obtaining the location information of the erroneous code. Next, based on the location information of the erroneous code, the code development platform extracts context information, which includes at least one of the following: the erroneous test case, the error description, or the error content. The code development platform inputs a prompt constructed based on the context information into a language model (LM) for inference, obtaining a first corrective code. This first corrective code can be obtained by the language model correcting the erroneous code in the test code.

This method first locates errors by combining execution information from the test code, then extracts rich contextual information based on the error code's location. Using this contextual information, it generates more precise and complex repair code through an advanced language model to adapt to various complex error scenarios, not limited to specific error types. This reduces manual intervention, improves the automation level of test code repair, thereby increasing repair efficiency and accuracy, accelerating software development and testing iterations, and enhancing overall software quality. Furthermore, this method can repair different error types, demonstrating strong versatility and adaptability.

To make the technical solution of this application clearer and easier to understand, the system architecture of the code development platform of this application will be introduced below with reference to the accompanying drawings.

Referring to Figure 1, which shows a schematic diagram of the architecture of a code development platform, the code development platform 10 includes an interactive device 100 and a test code repair device 200. The interactive device 100 can be deployed as a plugin on a client, which can be an IDE or a code editor; therefore, the interactive device 100 can also be called a plugin. The test code repair device 200 can be deployed on a server; therefore, the test code repair device 200 can also be called a server.

The test code repair device 200 may include an error location module (not shown in Figure 1) and a model repair module 201. Further, the test code repair device 200 may also include a template matching repair module 203. The interaction device 100 coordinates the workflow of the functional modules (such as the model repair module 201 and the template matching repair module 203) in the test code repair device 200, interacts with other hardware devices, and collects user information in real time. The interaction device 100 can also run the test code to obtain its execution information (or running information) and synchronize and interact with the server (including the central server and edge servers), so that the server-side test code repair device 200 can repair the test code based on the execution information. The interaction device 100 may include a test code selection module 102 and a running module 104. In some possible implementations, the test code may be compiled before running, and the interaction device 100 may also include a compilation module 103. The server-side test code repair device 200 can then repair the test code based on the compiled information. Furthermore, the interactive device 100 may also include a repair strategy selection module 106. The interactive device 100 can coordinate the workflow of the repair module in the test code repair device 200 according to the repair strategy. In addition, the interactive device 100 may also include a feedback module 108. The feedback module 108 supports the introduction of a mechanism for manually repairing test code on top of automated test code repair.

The functions of each module and the collaboration process between modules are described below.

The test code selection module 102 is used to obtain the test code corresponding to the source code. The source code can be the source code of functions, methods, or classes of the software under test, or it can be the project-level code of the software under test. Specifically, the test code selection module 102 can respond to the user's selection operation and select the test code to be tested and fixed. The test code can differ depending on the stage of software testing. Taking unit testing as an example, the test code can be the test case code in the unit test. This test code can be manually developed or automatically generated. For example, the user can input the source code of the software under test, preprocess the source code (e.g., extract key information from the source code to construct prompts), and then input the prompts into a language model, especially a large language model (LLM). Through the text understanding and generation capabilities of the LLM, test cases are generated.

The compilation module 103 is used to perform compilation operations on the test code and obtain compilation information. This compilation information includes error logs output by the compiler during the compilation of the test code. The error logs record the error types of the error codes. For example, error types may include missing class references, which is a compilation error. The error logs also record the location information of the error codes, thereby enabling error localization.

The execution module 104 is used to perform execution operations on the test code. Specifically, the execution module 104 can call the executor to execute the test code. The execution module 104 can run the test code when it has been successfully compiled. This embodiment decouples the compilation and execution of the test code, enabling more accurate problem localization and allowing for categorized handling of different types of problems. In some possible implementations, the compilation module 103 and the execution module 104 can also be coupled together; the coupled module is called the compilation and execution module, used to implement compilation, execution, and other functions.

The repair strategy selection module 106 is used to select a repair strategy and provide it to the test code repair device 200. The repair strategy may include, but is not limited to, directly using the language model for repair, directly using template matching for repair, or prioritizing template matching repair followed by model repair. Prioritizing template matching repair means using the repair template first; if that fails, the language model is then used. Prioritizing model repair means using the language model first. Different repair strategies provided by the repair strategy selection module 106 can lead to different workflows for the functional modules in the test code repair device 200.

The error localization module is used to obtain execution information of the test code, including stack trace information of exceptions output during test code execution. Based on this execution information, the module locates errors in the test code to obtain the position information of the erroneous code. This error code position information is then input into the model repair module 201 or the template matching repair module 203 for test code repair.

The model repair module 201 is used to extract context information based on the location information of the error code. This context information includes at least one of the following: the erroneous test case, the error description, or the error content. The prompt constructed based on the context information is then input into a language model for inference to obtain a first repair code. This first repair code can be obtained by the language model repairing the error code in the test code.

The template matching and repair module 203 is used to determine the error type of the error code, and repair the error code according to the repair template corresponding to the error type to obtain the second repair code. The first and second repair codes can be incremental code for repairing the error code, or full code (the complete test code after repair).

It should be noted that when the repair strategy prioritizes template matching repair, the template matching repair module 203 can also return the second repair code to the interactive device 100. The interactive device 100 determines whether the second repair code was successfully repaired by executing a test operation on the test code (e.g., the second repair code, or both the test code and the second repair code) repaired by the template matching repair module 203. If the execution passes, the repair is successful; if the execution fails, the repair fails.

Accordingly, the model repair module 201 is specifically used to extract context information based on the location information of the error code when the second repair code fails, and to perform reasoning on the prompt input language model constructed based on the context information to obtain the first repair code.

In some possible implementations, the model repair module 201 is also used to return the first repair code to the interactive device 100. The execution module 104 in the interactive device 100 is also used to execute the first repair code or test code repaired using the first repair code. The feedback module 108 in the interactive device 100 is used to present the first repair code to the user when execution fails, receive feedback from the user on the first repair code, and obtain the updated first repair code.

Based on the code development platform 10 shown in Figure 1, this application provides a test code repair method. The test code repair method of this application will be described below with reference to the accompanying drawings.

Referring to Figure 2, a flowchart of a test code repair method is shown. This method is executed by a code development platform 10, which includes an interactive device 100 (such as a plugin) and a test code repair device 200 (such as a server). The method includes the following steps:

S201, Interactive device 100 obtains the test code corresponding to the source code.

The source code can be the code of the software under test (such as project-level code), or the code of one or more modules (or the smallest testable unit) of the software under test. Test code can be code used to test the software or one or more modules of the software. Test code is used to test the functionality and/or performance of the source code. Functional testing focuses on verifying whether the behavior of the software under test meets design and user requirements, mainly checking that each function of the software under test works correctly as required. Performance testing evaluates the response speed, stability, reliability, resource consumption, and scalability of the software under test under specific conditions, involving the efficiency of the software under test running under high load or stress conditions. Depending on the testing phase, test code can be unit test code, integration test code, or system test code.

For a software project, the interactive device 100 can support user selection of test code. This test code can be manually written by the user or automatically generated. For ease of description, an example of automatically generated test code is used. Specifically, referring to a schematic diagram of a code editing interface shown in Figure 3, the code editing interface 300 includes a code editing window 302 and a toolbar 304. The code editing window 302 can include the code 3022 of the software under test, and the toolbar 304 can include a test code generation control 3042 and a test code repair control. The interactive device 100 can present the code of the software under test to the user through the code editing window 302 of the code editing interface 300. The user can select a code block to be used to generate test code from the code presented in the code editing window 302, such as a function code block or a method class code block, and then click the test code generation control 3042 to trigger the test code generation operation. In response to the test code generation operation, the interactive device 100 generates test code based on the code block to be used. The interactive device 100 can construct prompts based on code blocks, input the prompts into a trained LLM used to generate test code, and obtain the test code.

For ease of understanding, Figure 3 uses a shopping system as an example of the software under test. The shopping system includes a product class named Product, which has the following attributes: product name, price, and inventory. After the test code is generated, the interactive device 100 can display the test code 3024 of the user-selected code block in the code 3022 of the software under test through the code editing window 302. Again using the shopping system example, the test code can be generated based on a testing framework (such as the JUnit testing framework). The test code includes a Product object, which has attributes such as name, price, and inventory. Assert statements (such as assert statements) are used in the test code to determine whether the actual result matches the expected result.

In the example of Figure 4, the test code could be:

Specifically, `testProductName()` tests whether the `getName()` method of the `Product` object can correctly return the name of the product; `testProductPrice()` tests whether the `getPrice()` method of the `Product` object can correctly return the price of the product; `testProductStock()` tests whether the `getStock()` method of the `Product` object can correctly return the inventory of the product; and `testProductOutOfStock()` tests whether the `isOutOfStock()` method of the `Product` object can correctly determine whether the inventory of the product is 0.

S202, Compile test code for interactive device 100.

Specifically, the interactive device 100 can invoke a compiler to compile the test code. The compiler can be a local compiler or a remote compiler (e.g., a cloud compiler). During compilation, the interactive device 100 can select a compiler that matches the language of the test code (e.g., a programming language or program language). For example, if the test code is developed in Java, the interactive device 100 can select a Java compiler and then invoke the Java compiler to compile the test code.

S204. Interactive device 100 determines whether a compilation error has occurred. If yes, proceed to S209; otherwise, proceed to S206.

The interactive device 100 can determine whether a compilation error has occurred by interacting with the compiler. For example, the compiler can output an error log when a compilation error occurs. The error log can include the error type and error location. The interactive device 100 can determine whether a compilation error has occurred by reading the error log. If the error log is empty, or no error log is read, it means that compilation is successful and there is no compilation error. If the error log is not empty, it means that compilation fails and there is a compilation error. Alternatively, the interactive device 100 can listen for events reported by the compiler. If a warning event is detected, it means that compilation has failed; if no warning event is detected, it means that compilation is successful.

The steps S202 and S204 described above are optional steps in this application. The test code repair method of this application may be implemented without executing the above steps. For example, when the test code is developed using a non-compiled language, the interactive device 100 may not execute S202 and S204.

S206, Interactive device 100 executes test code.

Specifically, the interactive device 100 can invoke an executor to execute test code. When the test code is developed in a compiled language, the interactive device 100 can invoke the executor to execute the compiled test code (usually executable machine code) and obtain the execution result. When the test code is developed in a non-compiled language, such as machine code, the interactive device 100 can directly invoke the executor to execute the test code.

An executor is a tool for executing test code. It is typically a virtual machine; for example, if the test code is Java code, the executor could be a Java Virtual Machine (JVM). In practical applications, the interactive device 100 can use a local compiler or a remote compiler (such as a cloud compiler) to execute the test code (e.g., compiled test code). When executing the test code, the interactive device 100 can select an executor that matches the language of the test code (e.g., programming language or program language) to execute the test code.

S208. Interactive device 100 determines whether an execution error has occurred. If yes, then execute S210; otherwise, then execute S224.

The interactive device 100 can determine whether an execution error has occurred by interacting with the executor. For example, the interactive device 100 can listen for events reported by the executor. If an error event (such as an exception event) is detected, it indicates that an execution error exists. If no error event is detected, it indicates that no execution error exists and the test code executes successfully.

S206 and S208 described above are optional steps in the embodiments of this application. The test code repair method of this application may also be performed without performing the above steps. For example, when the software under test is software that has been launched in the production environment, the user may also download the above test code and its compilation information or execution information from the production environment and input it into the test code repair device 200 to perform test code repair.

S209, the test code repair device 200 obtains the compilation information of the test code, locates errors in the test code based on the compilation information, and obtains the location information of the error code. Then, S212 is executed.

S210, the test code repair device 200 obtains the execution information of the test code, locates errors in the test code based on the execution information, and obtains the location information of the error code. Then, S212 is executed.

Compilation information includes error logs output by the compiler when compiling the test code. The error log records the error type of the error code, and may also record at least one of the following: the location of the error code, and related warning information. The location of the error code can be represented by at least one of the following: class name, method name, or line number. Warning information may include an error description and suggested fixes.

Execution information includes stack traces of exceptions output during the execution of test code. Exceptions are a different type of error from errors such as missing class references (compile-time errors) and assertion failures (or assertion errors). Stack traces describe the active stack frames at a specific point in time during the program's (e.g., test code) execution. Stack frame information can include the exception type, its location (e.g., class name, method name, line number), and the method call chain that caused the exception.

When a compilation error occurs, the test code repair device 200 can parse the compilation output error log to locate the error in the test code and obtain the location information of the error code. When an execution error occurs, the test code repair device 200 can parse the stack trace information and obtain the location information of the error code through the location of the occurrence in the stack frame information of the stack trace information. The location information of the error code can be represented by class name, method name, or line number.

It should be noted that S209 is an optional step in this application, and the test code repair method of this application may not require S209. For example, when the test code is executable code, including but not limited to machine code, the test code repair device 200 may not need to execute S209. Furthermore, S201 to S210 described above are specific implementations in this application's embodiments of obtaining the test code corresponding to the source code, executing the test code to obtain the execution information of the test code, and performing error location based on the execution information to obtain the location information of the error code. In practical applications, error location can also be performed in other ways to obtain the location information of the error code. For example, when the code development platform 10 provides the code to users through a cloud service subscription, the user can pass in the test code through the interface exposed by the cloud service. The cloud service can execute the test code, obtain the execution information of the test code, and perform error location based on the execution information to obtain the location information of the error code.

S212, The test code repair device 200 determines the error type of the error code, repairs the error code according to the repair template corresponding to the error type, and obtains the second repair code.

Specifically, the test code repair device 200 can obtain the error type by parsing error logs or stack trace information. For some error types, the test code repair device 200 supports using the corresponding repair template to repair the erroneous code. The error types supported by the template repair device 200 can form an error type set, which includes at least one error type. For example, the error type set may include missing class references, assertion failures, or exceptions. Exceptions may include runtime exceptions, also known as unchecked exceptions. Runtime exceptions are usually caused by design flaws or code logic errors. Common runtime exceptions include null pointer exceptions (NPE), array index out of bounds exceptions, type conversion exceptions, and data storage exceptions (inconsistent types when operating on arrays).

Based on this, the test code repair device 200 can determine whether the error type of the erroneous code matches the error type set. Specifically, the test code repair device 200 can compare the error type of the erroneous code with the error type set. If the error type of the erroneous code matches any error type in the error type set, it means that the error type matches the error type set, and the test code repair device 200 can repair the erroneous code according to the repair template corresponding to the error type to obtain a second repair code. If the error type of the erroneous code does not match any error type in the error type set, it means that the error type does not match the error type set, and the test code repair device 200 can use other methods to repair the erroneous code, such as using a language model to repair the erroneous code.

S214. The test code repair device 200 determines whether the second repair code has been successfully repaired. If yes, then execute S224; if no, then execute S216.

The test code repair device 200 can return second repair code to the interactive device 100. The second repair code can be either the full code after repairing the test code or incremental code that repairs the test code. When the second repair code is the full code, the repaired test code can be the second repair code; when the second repair code is incremental code, the repaired test code can be either the test code or the second repair code. The interactive device 100 can determine whether the second repair code was successfully repaired by executing the repaired test code, such as the second repair code or test code repaired using the second repair code. In some examples, the interactive device 100 can compile the repaired test code (such as the second repair code or test code repaired using the second repair code). If the compilation is successful, the repaired test code continues to be executed. The interactive device 100 can determine whether the second repair code was successfully repaired based on the execution result and return the repair result to the test code repair device 200. If the repair is successful, the test code repair device 200 can stop repairing, and the interactive device 100 can save the second repair code. If the repair is unsuccessful, the test code repair can continue.

In some possible implementations, the code development platform 10 can also be configured with a maximum number of retries, for example, the maximum number of retries can be configured as a target value. Based on this, the test code repair device 200 can obtain the compilation information or execution information of the second repair code. When the compilation information or execution information indicates that the second repair code has failed to repair, the test code repair device 200 can re-execute the code repair for the test code until the test code is successfully repaired or the number of retries reaches the target value.

S216. The test code repair device 200 extracts context information based on the location information of the error code.

The context information includes at least one of the following: erroneous test cases, error descriptions, or error content. The error description may include the error type and the location information of the error code. Specifically, the test code repair device 200 can obtain the error content based on the location information of the error code. The error content can be content related to the error code in the compilation or execution information, such as an error record in the compilation or execution information that includes the aforementioned location information. This error record not only records the location information of the error code but may also record the error type (such as exception type) and the method call chain that caused the exception. The test code repair device 200 can also generate an error description based on the compilation or execution information of the test code. Furthermore, the test code repair device 200 obtains the erroneous test cases based on the code file where the error code is located. The test cases may include the test objective, test environment, test input, and expected result.

To provide sufficient contextual information for language models such as LLM, the test code repair device 200 can construct a contextual information set, denoted as "C". "C" includes the erroneous test case T{error}, the error description D{e}, and the error content C_e, as shown below:

C = {T{error}, D{e}, C_{e}}.

S218, Test code repair device 200 constructs a prompt based on context information.

The test code repair device 200 can construct prompts based on context information and a prompt template. The context information can include different types, and the test code repair device 200 can fill the corresponding fields of the prompt template with different pieces of context information to obtain the prompts.

In some examples, the prompt template may include error messages and error information. Error messages indicate that errors exist in the language model test code and need to be fixed. Error information provides information that can be used to fix the erroneous code, such as context information. Error information may include multiple fields, such as test case, error description, and error content. The test code repair device 200 fills the test case field with the erroneous test case T{error} from the context information, fills the error description field with the error type, error code location information, etc., and fills the error content field with the error from the compilation or execution information. This provides clues for error localization and analysis in the language model.

In some possible implementations, the prompt template may also include a fix requirement field. The fix requirement field is used to populate a fix requirement for the test code that caused the error. For example, the fix requirement could be to require the language model (such as LLM) to modify the test code and output the complete test code as a code block. Figure 5 shows a schematic diagram of a prompt template. As shown in Figure 5, test template 500 can include error prompt 502, error message 504, and fix requirement 506. Error prompt 502 is used to indicate that the test code has generated an error. In some examples, error prompt 502 can also indicate the error type. In the example in Figure 5, error prompt 502 could be “There may be assertion errors./Your unit test has encountered an error,” where “There may be assertion errors” indicates that there may be assertion errors (assertion failure), and “Your unit test has encountered an error” indicates that the unit test code may have an error. The error type “assertion” in error prompt 502 can be filled in based on context information. Error message 504 can include a padding description and a region to be filled (as shown in "{error_message}" in section 5). The padding description indicates the location to fill in the error message, and the region to be filled can be filled with context information. Repair request 506 can be "Modify your test code accordingly and output completed test code in a code cell," indicating a request for the language model to repair the test code and output the complete test code as a code block.

S220, the test code repair device 200 requests the LLM to repair according to the prompt and obtains the first repair code.

Specifically, the test code repair device 200 can input a prompt into the LLM to request the LLM to repair the test code based on the prompt. The test code repair device 200 can use the prompt as an application programming interface (API) parameter to generate an API call request, and then invoke the LLM through the API call request to receive the first repair code returned by the LLM. The first repair code is obtained by the LLM repairing the errors in the test code.

LLM is just one example of a language model. In practical applications, the test code repair device 200 can also use a high-precision, high-performance small language model (SLM) for inference to obtain the first repair code.

The steps S216 to S220 described above are a specific implementation of extracting context information based on the location information of the error code, and then using the prompt input language model constructed based on the context information for reasoning to obtain the first repair code. In practical applications, the first repair code can also be obtained through other methods. For example, in some cases, the code development platform 10 (e.g., the test code repair device 200) can obtain the error code based on its location information, directly input the error code into the language model for reasoning, and obtain the first repair code.

S222, The test code repair device 200 determines whether the first repair code was successfully repaired. If yes, then execute S224; if no, then execute S226.

The test code repair device 200 can return first repair code to the interaction device 100. The first repair code can be either the full code after repairing the test code or incremental code that repairs the test code. When the first repair code is the full code, the repaired test code can be the first repair code. When the second repair code is incremental code, the repaired test code can be the test code repaired using the first repair code, such as a code block formed by inserting the first repair code into the test code. The interaction device 100 can determine whether the first repair code was successfully repaired by executing the repaired test code, such as the first repair code or the test code repaired using the first repair code.

Specifically, the interactive device 100 compiles the repaired test code (such as the first repaired code, or the test code repaired using the first repaired code). If the compilation succeeds, the repaired test code continues to be executed. The interactive device 100 can determine whether the first repaired code was successfully repaired based on the execution result and return the repair result to the test code repair device 200. If the repair is successful, the test code repair device 200 can stop the repair, and the interactive device 100 can save the first repaired code. If the repair is unsuccessful, the test code repair can continue.

In some possible implementations, the code development platform 10 can also configure a maximum number of retries (upper limit on the number of retries) for the language model. For example, the maximum number of retries can be configured as a target value. Based on this, when the first code repair fails, the test code repair device 200 can re-execute code repair based on the LLM or other language models for the test code until the test code is successfully repaired or the number of retries reaches the target value.

Furthermore, the interactive device 100 can also present the first repair code to the user when the first repair code or the test code repaired using the first repair code fails to execute, receive feedback from the user on the first repair code, and obtain an updated first repair code. For example, the interactive device 100 can present the first repair code to the user when the number of retries reaches the target number and the repaired test code (such as the first repair code or the test code repaired using the first repair code) still fails to execute. The user can then manually modify the first repair code, and the interactive device 100 can receive feedback from the user on the first repair code and obtain an updated first repair code.

S224. The interactive device 100 saves the test code that has passed execution.

In S208, if the test code executes successfully, the interactive device 100 can save the test code. Alternatively, in S214, if the second repair code is determined to be successfully repaired, the interactive device 100 can save the successfully repaired test code. If the second repair code is a full code, the successfully repaired test code can be the second repair code itself; if the second repair code is an incremental code, the successfully repaired test code can be both the second repair code and the test code. Alternatively, in S222, if the first repair code is determined to be successfully repaired, the interactive device 100 can save the successfully repaired test code. If the first repair code is a full code, the successfully repaired test code can be the first repair code itself; if the first repair code is an incremental code, the successfully repaired test code can be both the first repair code and the test code.

S226, Interactive device 100 end test code repair.

The above steps S224 and S226 are optional steps in the embodiments of this application. The test code repair method of this application may also omit the above steps S224 and S226.

Based on the above description, this application provides a test code repair method. This method combines template matching repair technology and language model repair technology to achieve comprehensive repair of test code. The template repair technology relies on preset repair templates and can quickly locate and repair common error types. The language model repair technology utilizes advanced language models to generate more accurate and complex repair solutions by analyzing error information and code context. Therefore, the test code repair method of this application is not only suitable for repairing specific types of errors but can also handle various complex error scenarios, providing more intelligent and flexible repair strategies, and has strong versatility and adaptability. This method can significantly improve the automation level of test code repair, reduce manual intervention, accelerate the iteration speed of software development and testing, and improve the overall quality of software.

It should be noted that the embodiment shown in Figure 2 illustrates how language model repair is used as a supplementary means to further improve the success rate of test code repair when template matching repair fails. To further improve the repair success rate, the code development platform can also provide template matching repair-related information, such as second repair code or intermediate template matching results, as part of the context information to the language model, thereby generating more accurate first repair code. In some possible implementations, the code development platform 10 can also directly use the language model for test code repair, or directly use the repair template for test code repair; this application does not impose any limitations on this.

Next, we will use different error types as examples to illustrate test code repair based on repair templates. Error types can include missing class references (or compilation errors), assertion failures (assertion errors), or exceptions. Correspondingly, repair templates can include import repair templates, assertion repair templates, or exception repair templates. These will be explained in turn below.

In some possible implementations, the error type of the error code includes missing class references, and correspondingly, the second repair code may include import statements. The code development platform 10 can determine the missing class name and construct a symbol index based on project dependency libraries, development tool libraries, or third-party libraries. The symbol index includes the fully qualified name of the accessible class. The fully qualified name, also called the fully qualified class name, usually refers to the fully qualified class name and typically includes the package path. For example, the fully qualified name of a Java class could be java.lang.String. The missing class name refers to the class name of the missing class reference. This class name is usually an unqualified class name, also called a short name, and typically does not include the package path or package name. For example, the unqualified class name of a Java class could be String. The code development platform 10 matches the missing class name with the symbol index to obtain the fully qualified name corresponding to the missing class name. Then, the code development platform 10 inserts this fully qualified name into the import repair template to obtain the import statement. The code development platform 10 can repair the test code using either incremental updates or full updates. In some examples, the second fix code can be incremental code, in which case it includes the aforementioned import statements. In other examples, the second fix code can be full code, in which case it can include test code and import statements. For example, code development platform 10 can insert import statements into test code to obtain the second fix code.

The import repair template is a strategy to fix "symbol not found" errors caused by missing class or interface imports during compilation. The core of this template is to identify missing class references and automatically insert the corresponding import statements. An example is provided below to illustrate this.

In this example, the code development platform 10 parses the compiler's error log to obtain the error type T and the location information L{c} = (p, N{row}, N{col}). Here, p represents the file path, N{row} and N{col} represent the line number and column number where the error occurred, respectively, and the missing class name or interface name S. Then, the code development platform 10 performs the following steps to implement test code repair based on the imported package repair template.

Step 1: The code development platform 10 constructs a symbol index (Index, I) based on the project dependency library, development tool library, and third-party library.

The development toolkit (DMP) is related to the programming language used in development. For example, when using Java, the DMP could be the Java Development Kit (JDK) library. This index contains the fully qualified names of all accessible Java classes, implementing a mapping from S to R, where R represents the fully qualified name of the symbol S.

Step 2: The code development platform 10 uses regular expressions to extract the missing class name S from the error message, matches it in the symbol index I, and finds the corresponding fully qualified name R.

Step 3: The code development platform 10 fills the fully qualified name R of the matching import repair template, generates a complete import statement, and inserts the import statement after the existing import statement in file p.

In some other possible implementations, the error type of the error code includes assertion failure. The code development platform 10 can determine the assertion type of the erroneous assertion in the error code, and then, based on the assertion type, choose to update the assertion repair template or update the assertion parameters, thereby repairing the erroneous assertion.

For example, if the assertion type is equality, the code development platform 10 can extract the expected value of the erroneous assertion from the error code and the actual value of the erroneous assertion from the test report. Then, based on the assertion repair template corresponding to the assertion type, it replaces the expected value in the assertion parameters of the erroneous assertion with the actual value. In other words, for equality-type erroneous assertions, the code development platform 10 uses the existing assertion template as the assertion repair template and replaces the assertion parameters of the erroneous assertion, such as replacing the expected value with the actual value, thereby achieving rapid repair of equality-type erroneous assertions.

For example, if the assertion type is Boolean, the code development platform 10 can generate second repair code based on the assertion parameters in the erroneous assertion and the second assertion repair template corresponding to the first assertion repair template used by the erroneous assertion. The Boolean logic of the second assertion repair template is the opposite of that of the first assertion repair template. In other words, for Boolean type erroneous assertions, the code development platform 10 achieves rapid repair of Boolean type erroneous assertions by changing the template while keeping the assertion parameters unchanged. Using a unit test example, the assertion repair template is a strategy for repairing assertion failures in unit tests. The core of the assertion repair template is to identify the cause of the assertion failure and automatically adjust the expected result of the assertion to match the actual test result. The expected result can be an expected value, and the actual result can be an actual value.

After determining the location information L{f}＝(p,N{row},N{col}) of the test code line containing the erroneous assertion through error localization, the following steps can be performed to repair the test code:

Step 1: The code development platform 10 determines the assertion type of the error assertion. When the assertion type is an equality type, the expected value V{expected} of the error assertion is extracted from the error assertion, and the corresponding actual value V{actual} is extracted from the test report, and then Step 2 is executed; when the assertion type is a boolean type, Step 3 is executed.

The assertion type can include, but is not limited to, boolean type and equality type. Boolean type can include assertTrue, assertFalse, assertNull, and assertNotNull, and equality type can include assertEquals.

Step 2: In the assertion repair template corresponding to the assertion type, the code development platform 10 replaces V{expected} in the assertion parameter of the erroneous assertion with V{actual}.

Step 3: The code development platform 10 generates the second repair code based on the assertion parameters in the error assertion and the second assertion repair template corresponding to the first assertion repair template used by the error assertion.

The Boolean logic of the second assertion repair template is the opposite of that of the first assertion repair template.

As shown in Figure 6, when using the assertEquals method for assertion in the test code, it is usually necessary to compare whether two values are equal. The code development platform 10 uses regular expressions to extract the actual value in the test report, processes the extracted actual value, such as removing redundant parentheses or special characters, and then constructs a replacement string according to the type of the actual value. The replacement string can be represented as actualValue. The replacement string is used to replace the first parameter expectedValue in the assertEquals method with actualValue.

As shown in Figure 7, when using the `assertNull` or `assertTrue` method for assertions in test code, if the test fails, it means that the object being checked contradicts expectations. In this case, the code development platform 10 can correct `assertNull` to `assertNotNull` and `assertTrue` to `assertFalse`, thus achieving quick repair. Conversely, if the test fails when using `assertNotNull` or `assertFalse` in the assertions, the code development platform 10 can correct `assertNotNull` to `assertNull` and `assertFalse` to `assertTrue`.

In some possible implementations, the error type of the error code includes exceptions, the repair template can include an exception repair template, and correspondingly, the second repair code can include the exception handling code block corresponding to the exception. The exception repair template is a repair strategy for exception error types. It can catch and handle these exceptions by inserting or improving exception handling mechanisms around code blocks that may throw exceptions, thereby preventing the program from terminating due to unhandled exceptions. The code development platform 10 can determine the exception type from the exception's stack trace information, fill the exception type into the exception repair template, and obtain the exception handling code block (e.g., an exception handling statement). When the second repair code is incremental code, it can include the aforementioned exception handling code block. When the second repair code is full code, it can include test code and the exception handling code block. For example, the code development platform 10 can insert an exception handling code block after the error code in the test code to obtain the second repair code.

After determining the exception type E and the location information L{e}＝(p,N{row},N{col}) of the code block where the exception occurred through error localization, the following steps can be performed to repair the test code:

Step 1: The code development platform 10 fills the exception type E into the exception repair template, generates a complete exception handling code block, and inserts the exception handling code block into the file p.

The exception handling code block can be located before code block C.

Step 2: After the insertion is completed, the code development platform 10 performs a syntax check to ensure that the new code snippet conforms to the Java syntax specification and avoids introducing new compilation errors.

After the repair is complete, the code development platform 10 can recompile and execute the repaired test code to verify whether the repair was successful. If the compilation and execution are successful, the repair process ends; if there are still errors, the code development platform 10 can return to the exception information extraction stage and continue to process the next error.

As shown in Figure 8, when an exception is thrown in a line of code in the test code, the exception can be caught and repaired by inserting exception handling statements. Specifically, the code development platform 10 can use try-catch statements to include the line of code that may throw an exception and handle the expected exception type in the catch block.

As shown in Figure 9, in some cases, existing exception handling statements may not fully cover all exception types actually thrown during code execution. To further improve the robustness of test cases, adding catch statements can ensure that the original exception handling logic remains intact, while avoiding the introduction of new errors through modifications, expanding the coverage of exception handling, and enabling test cases to handle more error scenarios.

The aforementioned repair templates are examples of preset static repair templates. In practical applications, the code development platform 10 can generate dynamic repair templates in real time based on the error type, without relying on preset static repair templates, thus improving the flexibility and accuracy of repair. For example, the code development platform 10 can use the interactive device 100 to allow developers to interact more deeply with the backend repair templates or models, propose modifications, or select multiple possible repair solutions, thereby enabling more precise control over the test code repair process.

It should be noted that the test code repair method of this application is not only applicable to the repair of Java unit test code, but can also be widely applied to other technical fields, such as embedded system development, mobile application development, and network security. In embedded systems, code often needs to be optimized to meet memory and processing power limitations. Therefore, the test code repair method of this application can not only fix errors, but also optimize the code to reduce resource consumption. Thus, the repair template can include preset templates for resource optimization, while language model repair can be trained to identify and propose solutions to reduce the resource requirements of code execution. For mobile applications, to ensure that the application's user interface and user experience are not affected by code errors, template repair based on user interface (UI) templates/user experience (UX) templates can be used, specifically for repairing code defects that may affect the user interface. Language model repair can optimize repair suggestions by analyzing error information related to the user interface, ensuring smooth application operation. In the field of network security, code repair needs to address not only functional errors, but also issues that may lead to security vulnerabilities. Template repair can introduce repair templates for common security vulnerabilities, such as defense against SQL injection or cross-site scripting (XSS) attacks. Language model repair can be further developed to generate repair code based on specific information about security vulnerabilities, thereby improving system security.

Based on the aforementioned code generation method, this application also provides a code development platform 10. The code development platform 10 is used to integrate functions into applications. As shown in Figure 1, the code development platform 10 includes:

The test code selection module 102 is used to obtain the test code corresponding to the source code, and the test code is used to test the functionality and/or performance of the source code;

The execution module 104 is used to execute the test code and obtain the execution information of the test code, the execution information including the stack trace information of the exceptions output when the test code is executed;

The error location module is used to locate errors in the test code based on the execution information and obtain the location information of the error code.

The model repair module 201 is used to extract context information based on the location information of the error code. The context information includes at least one of the error test case, error description, or error content. The module then uses a prompt input language model built based on the context information to perform reasoning to obtain a first repair code. The first repair code is obtained by the language model repairing the error code in the test code.

For example, the above-mentioned test code selection module 102, execution module 104, error location module, and model repair module 201 can be implemented in hardware or software.

When implemented in software, the test code selection module 102, execution module 104, error location module, and model repair module 201 can be applications running on computing devices, such as computing engines. These applications can also be virtualized and provided to users as virtualization services. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. VM services can be services that use virtualization technology to create virtual machine (VM) resource pools on multiple physical hosts to provide VMs for users to use on demand. BMS services are services that use virtualization technology to create BMS resource pools on multiple physical hosts to provide BMS for users to use on demand. Container services are services that use virtualization technology to create container resource pools on multiple physical hosts to provide containers for users to use on demand. A VM is a simulated virtual computer, that is, a logical computer. A BMS is a scalable, high-performance computing service with computing performance indistinguishable from traditional physical machines and features secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely specific examples. In practical applications, virtualization services can also include other lightweight or heavyweight virtualization services, which are not specifically limited here.

When implemented in hardware, the test code selection module 102, the execution module 104, the error location module, and the model repair module 201 may include at least one computing device, such as a server. Alternatively, the test code selection module 102, the execution module 104, the error location module, and the model repair module 201 may also be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

In some possible implementations, the code development platform 10 further includes a feedback module 108:

The running module 104 is further configured to execute the first repair code or the test code repaired using the first repair code;

The feedback module 108 is used to present the first repair code to the user when the execution fails, receive feedback from the user on the first repair code, and obtain the updated first repair code.

The feedback module 108 can be implemented in software or hardware. When implemented in software, the feedback module 108 can be an application running on a computing device, such as a computing engine. This application can also be virtualized and provided to users as a virtualization service via VMs, containers, etc. When implemented in hardware, the feedback module 108 can include at least one computing device, such as a server.

In some possible implementations, the code development platform 10 also includes:

The template matching and repair module 203 is used to determine the error type of the error code, repair the error code according to the repair template corresponding to the error type, and obtain a second repair code;

The model repair module 201 is specifically used for:

If the second repair code fails to repair, the context information is extracted based on the location information of the error code.

In some possible implementations, the error type of the error code includes assertion failure;

The template matching and repair module 203 is specifically used for:

Determine the assertion type of the error assertion in the error code;

When the assertion type is an equality type, extract the expected value of the error assertion from the error code and extract the actual value of the error assertion from the test report;

Based on the assertion repair template corresponding to the assertion type, the expected value in the assertion parameter of the erroneous assertion is replaced with the actual value to obtain the second repair code.

The template matching and repair module 203 can be implemented in software or hardware. When implemented in software, the template matching and repair module 203 can be an application running on a computing device, such as a computing engine. This application can also be virtualized and provided to users as a virtualization service via VMs, containers, etc. When implemented in hardware, the template matching and repair module 203 can include at least one computing device, such as a server.

In some possible implementations, the template matching and repair module 203 is further configured to:

When the assertion type is Boolean, a second repair code is generated based on the assertion parameters in the error assertion and the second assertion repair template corresponding to the first assertion repair template used by the error assertion. The Boolean logic of the second assertion repair template is the opposite of the Boolean logic of the first assertion repair template.

In some possible implementations, the error type of the error code includes an exception, and the second repair code includes an exception handling code block corresponding to the exception;

The template matching and repair module is specifically used for:

Determine the exception type from the stack trace information of the exception;

Fill the exception type into the exception repair template to obtain the exception handling code block.

In some possible implementations, the code development platform 10 also includes:

The compilation module 103 is used to compile the test code and obtain the compilation information of the test code. The compilation information includes the error log output by the compiler when compiling the test code. The error log records the error type of the error code.

When the error code includes a missing class reference, the second repair code includes an import statement;

The template matching and repair module 203 is specifically used for:

Determine the missing class name;

Based on project dependency libraries, development tool libraries, or third-party libraries, construct a symbol index, which includes the fully qualified names of accessible classes;

The missing class name is matched with the symbol index to obtain the fully qualified name corresponding to the missing class name;

Insert the fully qualified name into the import package repair template to obtain the import statement.

The compilation module 103 can be implemented in software or hardware. When implemented in software, the compilation module 103 can be an application running on a computing device, such as a computing engine. This application can also be virtualized and provided to users as a virtualization service via VMs, containers, etc. When implemented in hardware, the compilation module 103 can include at least one computing device, such as a server.

It should be noted that the compilation module 103 and the runtime module 104 in Figure 1 can also be coupled into a single module, for example, coupled into a compilation and runtime module. Figure 1 is merely one specific implementation of the code development platform 10. In other possible implementations of this application, the compilation and execution of test code can also be performed in other ways.

In some possible implementations, the model repair module 201 is further configured to:

If the first repair code fails to repair, the code repair is re-executed for the test code until the test code is successfully repaired or the number of retries reaches the target value.

This application also provides a computing device 1000. As shown in FIG10, the computing device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate with each other via the bus 1002. The computing device 1000 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1000.

Bus 1002 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 10, but this does not imply that there is only one bus or one type of bus. Bus 1002 can include pathways for transmitting information between various components of computing device 1000 (e.g., memory 1006, processor 1004, communication interface 1008).

The processor 1004 may include any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

The memory 1006 may include volatile memory, such as random access memory (RAM). The memory 1006 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD). The memory 1006 stores executable program code, which the processor 1004 executes to implement the aforementioned code generation method. Specifically, the memory 1006 stores instructions from the code development platform 10 for executing the code generation method.

The communication interface 1008 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1000 and other devices or communication networks.

This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

As shown in Figure 11, the computing device cluster includes at least one computing device 1000. The memory 1006 of one or more computing devices 1000 in the computing device cluster may store instructions from the same code development platform 10 for executing code generation methods.

In some possible implementations, one or more computing devices 1000 in the computing device cluster can also be used to execute some of the instructions used by the code development platform 10 to execute the code generation method. In other words, a combination of one or more computing devices 1000 can jointly execute the instructions used by the code development platform 10 to execute the code generation method.

It should be noted that the memory 1006 in different computing devices 1000 in the computing device cluster can store different instructions for executing some functions of the code development platform 10.

Figure 12 illustrates one possible implementation. As shown in Figure 12, two computing devices 1000A and 1000B are connected via a communication interface 1008. The memory in computing device 1000A stores instructions for executing the functions of the test code selection module 102 and the execution module 104. The memory in computing device 1000B stores instructions for executing the functions of the error location module and the model repair module 201. Furthermore, the memory in computing device 1000A also stores instructions for executing the functions of the compilation module 103, the repair strategy selection module 106, and the feedback module 108, while the memory in computing device 1000B also stores instructions for executing the function of the template matching repair module 203. In other words, the memory 1006 of computing devices 1000A and 1000B jointly stores the instructions used by the code development platform 10 to execute the code generation method.

The connection method between the computing device clusters shown in Figure 12 can be considered because the code generation method provided in this application requires a lot of resources to run the language model, thereby realizing the generation of the first repair code based on language model reasoning. Therefore, it is considered to dedicate the functions implemented by the error location module and the model repair module 201 in the test code repair device 200 to independent computing devices. For example, the functions implemented by the relevant modules in the interaction device 100 are executed by computing device 1000A, and the functions implemented by the relevant modules in the test code repair device 200 are executed by computing device 1000B.

It should be understood that the functions of computing device 1000A shown in Figure 12 can also be performed by multiple computing devices 1000. Similarly, the functions of computing device 1000B can also be performed by multiple computing devices 1000.

In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 13 illustrates one possible implementation. As shown in Figure 13, two computing devices 1000C and 1000D are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1006 in computing device 1000C stores instructions for executing the functions of relevant modules in the interactive device 100, such as instructions for executing the functions of the test code selection module 102 and the execution module 104. Furthermore, the memory 1006 in computing device 1000C also stores instructions for executing the functions of the compilation module 103, the repair strategy selection module 106, and the feedback module 108. Meanwhile, the memory 1006 in the computing device 1000D stores instructions for executing the functions of relevant modules in the test code repair device 200. For example, the memory of the computing device 1000D stores instructions for executing the functions of the error location module and the model repair module 201. Furthermore, the memory of the computing device 1000D also stores instructions for executing the functions of the template matching repair module 203.

The connection method between the computing device clusters shown in Figure 13 can be considered as follows: taking into account that the code generation method provided in this application requires a lot of resources for reasoning to generate the first repair code, it is considered that the functions implemented by the relevant modules in the test code repair device 200 are handed over to the computing device 1000D for execution.

It should be understood that the functions of computing device 1000C shown in Figure 13 can also be performed by multiple computing devices 1000. Similarly, the functions of computing device 1000D can also be performed by multiple computing devices 1000.

This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the code generation method described above applied to the code development platform 10.

This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the above-described code generation method.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims (19)

A method for fixing test code, characterized in that it is applied to a code development platform, the method comprising: Obtain the test code corresponding to the source code, and the test code is used to test the functionality and/or performance of the source code; Execute the test code to obtain the execution information of the test code, the execution information including the stack trace information of the exceptions output when the test code is executed; Based on the execution information, the test code is used to locate errors and obtain the location information of the error code. Based on the location information of the error code, extract the context information, which includes at least one of the error test case, error description, or error content. The prompt input language model constructed based on the context information is used for reasoning to obtain a first repair code, which is obtained by the language model to repair the erroneous code in the test code. The method according to claim 1, characterized in that the method further comprises: Execute the first fix code or the test code that is fixed using the first fix code; If the execution fails, the first fix code is presented to the user, feedback from the user on the first fix code is received, and the updated first fix code is obtained. The method according to claim 1 or 2, characterized in that the method further comprises: Determine the error type of the error code; The error code is repaired according to the repair template corresponding to the error type to obtain the second repair code; The step of extracting context information based on the location information of the error code includes: If the second repair code fails to repair, the context information is extracted based on the location information of the error code. The method according to claim 3, wherein the error type of the error code includes assertion failure; The step of repairing the error code according to the repair template corresponding to the error type to obtain the second repair code includes: Determine the assertion type of the error assertion in the error code; When the assertion type is an equality type, extract the expected value of the error assertion from the error code and extract the actual value of the error assertion from the test report; Based on the assertion repair template corresponding to the assertion type, the expected value in the assertion parameter of the erroneous assertion is replaced with the actual value to obtain the second repair code. The method according to claim 4, characterized in that the method further comprises: When the assertion type is Boolean, a second repair code is generated based on the assertion parameters in the error assertion and the second assertion repair template corresponding to the first assertion repair template used by the error assertion. The Boolean logic of the second assertion repair template is the opposite of the Boolean logic of the first assertion repair template. According to the method of claim 3, the error type of the error code includes an exception, and the second repair code includes an exception handling code block corresponding to the exception; The step of repairing the error code according to the repair template corresponding to the error type to obtain the second repair code includes: Determine the exception type from the stack trace information of the exception; Fill the exception type into the exception repair template to obtain the exception handling code block. The method according to claim 3, characterized in that the method further comprises: Compile the test code to obtain the compilation information of the test code. The compilation information includes the error log output by the compiler when compiling the test code. The error log records the error type of the error code. When the error code includes a missing class reference, the second repair code includes an import statement; The step of repairing the error code according to the repair template corresponding to the error type to obtain the second repair code includes: Determine the missing class name; Based on project dependency libraries, development tool libraries, or third-party libraries, construct a symbol index, which includes the fully qualified names of accessible classes; The missing class name is matched with the symbol index to obtain the fully qualified name corresponding to the missing class name; Insert the fully qualified name into the import package repair template to obtain the import statement. The method according to any one of claims 1 to 7, characterized in that the method further comprises: If the first repair code fails to repair, the code repair is re-executed for the test code until the test code is successfully repaired or the number of retries reaches the target value. A code development platform, characterized in that the code development platform comprises: The test code selection module is used to obtain the test code corresponding to the source code, and the test code is used to test the functionality and/or performance of the source code; The execution module is used to execute the test code and obtain the execution information of the test code, including the stack trace information of the exceptions output when the test code is executed; The error location module is used to locate errors in the test code based on the execution information and obtain the location information of the error code. The model repair module is used to extract context information based on the location information of the error code. The context information includes at least one of the error test case, error description, or error content. The module then uses a prompt input language model built based on the context information to perform reasoning to obtain a first repair code. The first repair code is obtained by the language model repairing the error code in the test code. The code development platform according to claim 9, characterized in that the code development platform further includes a feedback module: The running module is also used to execute the first repair code or the test code repaired using the first repair code; The feedback module is used to present the first repair code to the user when the execution fails, receive feedback from the user on the first repair code, and obtain the updated first repair code. The code development platform according to claim 8 or 9, characterized in that the code development platform further includes: The template matching and repair module is used to determine the error type of the error code, and repair the error code according to the repair template corresponding to the error type to obtain a second repair code; The model repair module is specifically used for: If the second repair code fails to repair, the context information is extracted based on the location information of the error code. The code development platform according to claim 11 is characterized in that the error type of the error code includes assertion failure; The template matching and repair module is specifically used for: Determine the assertion type of the error assertion in the error code; When the assertion type is an equality type, extract the expected value of the error assertion from the error code and extract the actual value of the error assertion from the test report; Based on the assertion repair template corresponding to the assertion type, the expected value in the assertion parameter of the erroneous assertion is replaced with the actual value to obtain the second repair code. According to claim 12, the code development platform is characterized in that the template matching and repair module is further used for: When the assertion type is Boolean, a second repair code is generated based on the assertion parameters in the error assertion and the second assertion repair template corresponding to the first assertion repair template used by the error assertion. The Boolean logic of the second assertion repair template is the opposite of the Boolean logic of the first assertion repair template. According to the code development platform of claim 11, the error type of the error code includes an exception, and the second repair code includes an exception handling code block corresponding to the exception; The template matching and repair module is specifically used for: Determine the exception type from the stack trace information of the exception; Fill the exception type into the exception repair template to obtain the exception handling code block. The code development platform according to claim 11, characterized in that the code development platform further includes: A compilation module is used to compile the test code and obtain compilation information of the test code. The compilation information includes error logs output by the compiler when compiling the test code, and the error logs record the error types of the error codes. When the error code includes a missing class reference, the second repair code includes an import statement; The template matching and repair module is specifically used for: Determine the missing class name; Based on project dependency libraries, development tool libraries, or third-party libraries, construct a symbol index, which includes the fully qualified names of accessible classes; The missing class name is matched with the symbol index to obtain the fully qualified name corresponding to the missing class name; Insert the fully qualified name into the import package repair template to obtain the import statement. The code development platform according to any one of claims 9 to 15, wherein the model repair module is further configured to: If the first repair code fails to repair, the code repair is re-executed for the test code until the test code is successfully repaired or the number of retries reaches the target value. A computing device cluster, characterized in that the computing device cluster includes at least one computing device, the at least one computing device includes at least one processor and at least one memory, the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions to cause the computing device cluster to perform the code generation method as described in any one of claims 1 to 8. A computer-readable storage medium, characterized in that it includes computer-readable instructions; the computer-readable instructions are used to implement the code generation method according to any one of claims 1 to 8. A computer program product, characterized in that it includes computer-readable instructions; the computer-readable instructions are used to implement the code generation method according to any one of claims 1 to 8.