CN115470104B

CN115470104B - A simulation software testing method based on equivalent module mutation

Info

Publication number: CN115470104B
Application number: CN202210983410.7A
Authority: CN
Inventors: 郭世凯; 徐志豪
Original assignee: Dalian Maritime University
Current assignee: Dalian Maritime University
Priority date: 2022-08-16
Filing date: 2022-08-16
Publication date: 2025-06-24
Anticipated expiration: 2042-08-16
Also published as: CN115470104A

Abstract

The invention discloses a simulation software testing method based on equivalent mode variation, which comprises the steps of carrying out availability check on a generated test case, judging whether the test case is normally compiled, collecting variable values and time delays of the test case according to path coverage, generating a graphical preprocessing result graph according to the ratio of normal compiling, inserting two branch selection fragments into each test case, filling non-executed branches of the two branch selection fragments to generate a new variant program, screening out a variant program with high complexity in the new variant by adopting a Markov chain Monte Carlo method as an equivalent variant, and comparing differences between an original test program and the equivalent variant by adopting an equivalent mode input method so as to find defects of simulation software. According to the method, a plurality of effective simulation software bugs are tested, stability of the simulation verification tool is maintained to a certain extent, and the simulation verification tool is more reliable.

Description

Simulation software testing method based on equivalent mode variation

Technical Field

The invention relates to the field of software testing, in particular to a simulation software testing method based on equivalent mode variation.

Background

In digital circuit design, register-transfer level (RTL, english: register-TRANSFER LEVEL) is an abstract model of synchronous digital circuits, which is used in hardware description languages such as Verilog and VHDL to create high-level descriptions of actual circuits. In modern digital designs, designs on the register transfer level are the most typical workflow.

The simulation software is used as a tool for verifying the transmission level codes of the register, and the waveform chart output by the simulation software can help a developer to know the operation effect of the whole model in an intuitive description. This way the sender can locate possible flaws in the abstract model at a smaller cost. Therefore, it is important to ensure the reliability and stability of the simulation software.

The current test method for simulation software mainly uses cases of fuzzy test to generate class test, wherein the class test is typically VERISMITH. It generates random Verilog code based on AST fuzzy generation mode and applies it to the test of comprehensive tool or simulation software.

In the fields of chip design and high-end manufacturing, high requirements are placed on the stability and correctness of development tools. The existing test technology has two defects, firstly, the test tool cannot ensure the stability of the test tool, and the Verilog code generated based on AST has certain unavailability. This also results in the problem of false positives for defects found in the tool. Secondly, verilog codes generated based on AST are insufficient in diversity, the realized functions are single, and the space for exploring defects by tools is limited.

Disclosure of Invention

According to the problems existing in the prior art, the invention discloses a simulation software testing method based on equivalent mode variation, which specifically comprises the following steps:

Carrying out availability check on the generated test case, judging whether the test case is normally compiled or not, collecting variable values and time delays of the test case according to path coverage, and generating a graphical preprocessing result diagram according to the normal compiling ratio;

inserting two branch selection fragments into each test case;

filling non-executing branches of the two branch selection fragments to generate a new variant program;

screening out a variant program with high complexity in the new variants by adopting a Markov chain Monte Carlo method to serve as an equivalent variant;

the method adopts an equivalent mode input method to compare the differences between the original test program and the equivalent variants, thereby finding out the defects of the simulation software.

The test case generation tool VERISIMTH is adopted to generate test cases, the generated test cases are put into a test case pool and subjected to static standardization check, the test cases are compiled one by using simulation software so as to check the usability of the test cases, and the variable values of the test cases are stored in a table of a maintenance case pool.

When inserting the binary selection code segment, obtaining all variable values of the variable information table in the preprocessing process of the test case, randomly selecting a program point, selecting a variable according to the variable information of the variable information table, generating a discriminant which is far greater than the variable, synthesizing binary conditional statements with contradictory predicates according to the discriminant, inserting the synthesized binary conditional statements at the selected program point, and placing the code segment after the original program point in an execution branch.

When a new variant program is generated, firstly, generating a filling sentence, filling the filling sentence in a non-execution branch, carrying out static grammar check on the filling sentence, judging whether the filling sentence meets the grammar requirement, exporting codes meeting the grammar requirement as a new variant file, deleting codes not meeting the grammar requirement, and re-executing the two-branch sentence insertion program.

When the equivalent variant is obtained, constructing a state transition matrix by using the program distance between the new variant and the original test case, inputting the state transition matrix as a parameter into a selection program, obtaining the acceptance probability according to the result calculated by the selection program, judging whether the acceptance probability is greater than 1, accepting the variant if the acceptance probability is greater than 1, and if the acceptance probability is greater than 1, continuing to generate the new variant, simplifying the time sequence for updating the acceptance variant and saving the time sequence as the equivalent variant program.

Compiling operation and checking an equivalent variant program, if the compiling operation has errors, writing error information into a bug table, if no errors occur, compiling the program and generating a waveform diagram, comparing the generated waveform diagram with the waveform diagram compiled by the original test case, if differences are generated, writing difference information into the bug table, and if no differences are generated, ending the test process.

By adopting the technical scheme, the simulation software testing method based on the equivalent mode variation provided by the invention adopts the Markov Monte Carlo method to carry out variant program screening, so that the complexity of the generated equivalent variants is higher, a plurality of effective simulation software bug are tested according to the method, the stability of a simulation verification tool is maintained to a certain extent, the simulation verification tool is more reliable, and the usability of the generated variants is ensured by the variant method inserted with the two-branch condition selection statement, so that the success rate of the variants is higher.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to the drawings without inventive effort to those skilled in the art.

FIG. 1 is a flow chart of the method of the present invention

Detailed Description

In order to make the technical scheme and advantages of the present invention more clear, the technical scheme in the embodiment of the present invention is clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention:

The simulation software testing method based on the equivalent mode variation shown in fig. 1 specifically comprises the following steps:

s1 pretreatment stage

S11, generating test cases by adopting a test case generation tool VERISIMTH

S12, placing the generated test cases into a test case pool, and carrying out static standardization check before preprocessing

S13, starting preprocessing of the test program, and compiling the test cases one by using simulation software to check the usability of the test cases

S14, detecting whether the model can be compiled normally, if so, putting the model into a test pool, otherwise, deleting the use case and printing out errors

S15, collecting variable values and time delays of test cases according to path coverage

S16, storing the collected variable information into a table of a maintenance use case pool, and generating a graphical preprocessing result graph according to the rate of compiling normally

S17, finishing preprocessing and transferring to a test case mutation stage

S2, inserting a binary branch selection code segment into each test case, wherein the steps are as follows:

S21, acquiring values of all variables of the variable information table in the preprocessing process of the test case

S22, randomly selecting a program point

S23, randomly selecting a variable according to the variable information of the variable information table

S24, generating discriminant which is far larger than the variable

S25, synthesizing two branch conditional sentences with contradictory predicates according to the discriminant

S26, inserting the synthesized two-branch conditional statement at the selected program point

S27, placing the code segment after the original program point in an execution branch

S3, filling non-execution branches of the two-branch conditional statement, wherein the specific steps are as follows:

S31, filling pretreatment sampling program

S32, generating a filling sentence by using a use case generating tool

S33, filling the filling sentence in the non-execution branch

S34, executing static grammar check on the filling sentence to judge whether the grammar requirement is met

S35, exporting codes meeting grammar requirements into a new variant file, deleting codes not meeting grammar requirements and re-executing a two-branch statement inserting program

S4 Another innovation of our procedure is in the variant selection process.

The Markov chain Monte Carlo method is selected as a sampling selection strategy, and the variant with higher acceptance complexity is selected according to the defined variant distance, so that the stability and the robustness of the whole program are ensured, and the whole program is more stable in the execution time sequence. The method comprises the following specific steps:

s41, constructing a state transition matrix by using the program distance between the new variant and the original test case.

S42, inputting the state transition matrix as a parameter into the selection program

S43, obtaining the acceptance probability according to the result calculated by the selection program

S44, if the judging acceptance probability is larger than 1, accepting the variant, otherwise, continuing to generate a new variant to judge

S45, simplifying the time sequence of updating the receiving variant and saving the time sequence as an equivalent variant program

And S5, filling is completed through the steps. Next enter save test cases and report test procedure.

The invention uses a new technology named Equivalent Mode Input (EMI) test. This is a new development of differential testing of process language compilers. The idea is to systematically mutate a source program, whose output should be equivalent as long as its semantics remain equivalent for a given input data. As a special case, if some variables of the successful simulation model are not compiled, or the signal data after the simulation of the variables are compared with the original program variable simulation data, a comparison error is generated. These errors are not due to human factors and are a problem of the compiler itself. Thus we can say that this is the occurrence of a bug by the compiler. Differential testing with equivalent mode input can then effectively discover compiler problems that are easily ignored. The method comprises the following steps of

S51, compiling operation and checking equivalent variant program

S52, if the compiling operation has errors, writing error information into the bug table

S53, if there is no compiling error, compiling the program and generating a waveform diagram

S54, comparing the generated waveform diagram with the waveform diagram compiled by the original test case

S55, if the difference is generated, writing the difference information into the bug list

S56, if not, ending the comparison procedure

The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.

Claims

1. A simulation software testing method based on equivalent module variation, characterized by comprising:

Perform usability checks on the generated test cases and determine whether the test cases are compiled normally. Collect the variable values and delays of the test cases based on path coverage, and generate a graphical preprocessing result diagram based on the normal compilation ratio.

Insert a binary selection snippet for each test case;

Filling the unexecuted branches of the two-branch selection fragment generates a new variant program;

The Markov Chain Monte Carlo method is used to screen out the variant programs with high complexity among the new variants as equivalent variants;

The equivalent model input method is used to compare the differences between the original test program and the equivalent variant, so as to find the defects in the simulation software;

When inserting a binary selection code snippet: obtain all variable values of the variable information table of the test case during preprocessing, randomly select a program point, select a variable based on the variable information in the variable information table, generate a discriminant greater than the variable, synthesize a two-branch conditional statement with contradictory predicates based on the discriminant, insert the synthesized two-branch conditional statement at the selected program point, and place the code segment after the original program point in the execution branch.

2. The simulation software testing method based on equivalent module variation according to claim 1 is characterized in that: a test case generation tool VeriSimth is used to generate test cases, the generated test cases are placed into a test case pool and static normalization checks are performed, the test cases are compiled one by one using simulation software to check the availability of the test cases, and the variable values of the test cases are saved in a table that maintains the test case pool.

3. The simulation software testing method based on equivalent module variation according to claim 1 is characterized in that: when generating a new variant program: first generate a fill statement, fill the fill statement in the non-executed branch, perform a static syntax check on the fill statement to determine whether it meets the syntax requirements, export the code that meets the syntax requirements as a new variant file, delete the code that does not meet the syntax requirements and re-execute the two-branch statement insertion program.

4. The simulation software testing method based on equivalent module variation according to claim 3 is characterized in that: when obtaining an equivalent variant: a state transfer matrix is constructed using the program distance between the new variant and the original test case, the state transfer matrix is input as a parameter into a selection program, and the acceptance probability is obtained according to the result calculated by the selection program, and it is determined whether the acceptance probability is greater than 1. If so, the variant is accepted, otherwise new variants are continued to be generated, the timing of updating the accepted variants is simplified, and saved as an equivalent variant program.

5. The simulation software testing method based on equivalent module variation according to claim 4 is characterized in that: the equivalent variant program is compiled, run and checked; if an error occurs during the compilation and running, the error information is written into a bug table; if no error occurs, the program is compiled and a waveform diagram is generated; the generated waveform diagram is compared with the waveform diagram compiled from the original test case; if a difference occurs, the difference information is written into the bug table; if no difference occurs, the test process is terminated.