Disclosure of Invention
The invention provides a software test case automatic execution method based on data and keyword driving, aiming at solving the problems of low test accuracy and poor test effect of a test case of embedded software caused by the complexity of the embedded software.
The technical scheme adopted by the invention is as follows:
it comprises the following steps:
s1, establishing a model project;
s2, initializing system resources of the model engineering, and generating specific test data according to the initialized system resources;
s3, establishing a test model of the software according to the established model engineering and the generated specific test data;
s4, carrying out constraint design on the test model;
s5, binding keywords for driving the test case to automatically execute with the test model;
s6, configuring a coverage strategy of the test case, and generating the test case with a fixed format according to specific test data by using the test model obtained in the S5 after the keywords are bound;
s7, screening and confirming the test cases with fixed formats generated in the S6, and integrating the test cases obtained after screening and confirming to be used as a test case execution set;
s8, binding and connecting the test case execution set with the test environments, and traversing the test case execution set by using each test environment after the binding is successful;
and S9, driving each test case in the test case execution set to be automatically executed by using the keywords bound in the S5.
Preferably, the system resources of the model engineering are initialized in S2, and specific test data is generated according to the initialized system resources; the specific process is as follows:
s21, the system resources comprise data, time constraint and test environment of test cases;
setting parameters of data, wherein the parameters comprise data format, data type, data precision and data range;
the data format comprises single data, structural data and aggregate data;
s22, generating specific test data according to the parameters of the data;
the generation of the specific test data comprises the following four conditions:
setting a value taking domain of data, and respectively setting a valid class of the data and an invalid class of the data;
II, automatically sampling and generating a data effective value and a data invalid value according to an algorithm of an equivalence class and a boundary value through a set data value taking domain or a set identification logic operation expression;
III, generating continuous data;
and IV, generating time constraint by adopting a five-value method, wherein the five-value method comprises a left boundary-precision, a left boundary, a middle value, a right boundary and a precision value.
Preferably, the test model established in S3 is a nestable activity graph;
the graphic elements of the activity graph comprise action nodes, action streams, starting nodes, ending nodes, judging nodes, timer nodes, concurrency starting nodes and concurrency ending nodes.
Preferably, the test path of the test model established in S3 includes:
(1) Common path: no loop, no judgment node and no concurrent path exist;
(2) Circulation path: a looped path;
(3) Concurrent path: the method comprises the following steps that two or more activities are executed in the same time interval;
(4) The constraint function expression: and performing logic operation, arithmetic operation, time constraint and data operation and different types of time constraint operation on the judgment nodes of the test model.
Preferably, the constraint design in S4 includes a data constraint design, preset conditions, time constraints, and a sequential logic relationship.
Preferably, the keywords in S5 include a keyword function name, parameters of the keyword function, and a type of the keyword function; the types of the keyword functions comprise service layer keywords, logic layer keywords and execution layer keywords.
Preferably, in S5, the keyword for driving the test case to automatically execute is bound to the test model, and the specific process is as follows:
and binding each primitive of the test model activity diagram with one or more keywords, wherein the keywords bound by the same primitive are separated by half-angle semicolons.
Preferably, in S9, each test case in the test case execution set is driven to be automatically executed by using the keywords bound in S5, and the specific process is as follows:
s91, driving each test case in the test case execution set to generate a corresponding test script by using the keywords bound in the S5, wherein the test script is used for calling a function of the keywords corresponding to the test case and displaying parameter values of the function;
s92, sending the test script to a corresponding test environment in the test system;
s93, automatically executing a test script by a test environment;
s94, the test system records and dynamically displays the execution condition of the test script in real time, and the execution condition of the corresponding test case can be obtained;
s95, after the execution is finished, the test environment feeds back the execution result to the test system;
and S96, analyzing and updating the execution result of the test script by the test system, and outputting the execution result.
Preferably, in S91, each test case in the test case execution set is driven by using the keywords bound in S5 to generate a corresponding test script, and the specific process is as follows:
s911, analyzing the keyword function;
s912, outputting the analyzed keywords according to a front-back sequence to test cases;
s913, assigning a plurality of values to the output parameters of the test data of the continuous frame type;
and S914, dereferencing the parameter values of the keywords as dereferences of corresponding test data, and outputting the keywords without dereferences in an original state to obtain the test script.
Preferably, the parsing of the keyword function in S911 includes the following three parts:
(1) Testing key operations to be executed, namely some key actions of actual operations;
(2) An operated object to be triggered by the key action, namely an operated control element;
(3) Data that must be provided for testing, i.e., test data.
Has the advantages that:
firstly, establishing a test model for embedded software, carrying out constraint design on the test model, and binding keywords for driving a test case to automatically execute with the test model, wherein the test model is a nestable active graph, each primitive of the active graph is bound with one or more keywords, and the keywords bound by the same primitive are separated by a half-angle semicolon; the keywords comprise keyword function names, parameters of the keyword functions and types of the keyword functions; the types of the key word functions comprise service layer key words, logic layer key words and execution layer key words. Generating test cases based on the data and various coverage strategies, screening and confirming that the generated test cases are bound with the test environments, and traversing the test case execution set by each test environment after the binding is successful; finally, the keywords are used for driving the test cases to generate the test scripts, each test case corresponds to one test script, and the test system executes the test scripts and feeds the result back to the test system in real time, so that automatic test execution of the test cases is achieved, the test accuracy of the test cases is improved, and a good test effect is obtained. The test is driven by the keywords, the test cases are separated from the actual test data, the requirement on the code capability of a tester is not high, and the operation is simple; the testing personnel only need to select the test case, and the distributed execution of the test case can be realized by binding the corresponding testing environment, so that the working efficiency of the testing personnel is improved, and the testing cost and risk are reduced.
In the invention, when the application to be tested changes, a tester only needs to modify the corresponding key words and the specific logic thereof to realize the test, and the test data and the test logic are completely separated, so that the test case and the test script thereof become easy to maintain, and the test becomes more efficient.
Detailed Description
The first embodiment is as follows: the embodiment is described with reference to fig. 1, and the method for automatically executing software test cases based on data and keyword driving in the embodiment includes the following steps:
s1, establishing a model project;
the model project comprises a file of the model and data of the model;
the model engineering is actually established by establishing a working space for a test project, when the embedded software model engineering is established, firstly naming the test project, determining the actual storage position of the project, and then establishing a model file and model data of the project by a system. The model files comprise physical files such as a test document folder, a picture folder and a test model file, wherein the test model file is an entry file for a tester to open a model editor and read and write the contents of the model. The model data refers to the actual content of the model and the unified storage data of information possibly used in the project, and can be realized in the form of a database or XML and the like.
S2, initializing system resources of the model engineering, and generating specific test data according to the initialized system resources; the specific process is as follows:
the system resources include data, time constraints, and test environments for test cases.
Initializing all system resources of the model engineering, and generating specific test data about the embedded software according to the initialized system resources; the time constraint is a constraint with a given time range.
S21, setting parameters of data, wherein the parameters comprise a data format, a data type, data precision and a data range, and the data format comprises single data, structural data and aggregate data;
individual data: a single physical meaning of a datum; structural data: the system consists of a plurality of data with different attributes, and the data are usually collected, calculated and processed simultaneously; collecting data: the method is characterized by comprising a plurality of physical data, wherein each physical data does not have too much association, is simply combined, and can be processed together or separately. Data needs to be able to express single data, structured data and aggregate data.
S22, generating specific test data according to the parameters of the data;
the generation of the specific test data comprises the following four conditions:
setting a value taking domain of data, and respectively setting a valid class of the data and an invalid class of the data;
II, automatically sampling and generating a data effective value and a data invalid value according to an algorithm of an equivalence class and a boundary value through a set data value taking domain or a simple identified logical operation expression;
the specific algorithm description is shown as algorithm a.
The algorithm name is: algorithm a
Inputting: data of
And (3) outputting: specific test data
The process is as follows: acquiring parameters of data, namely a data structure, and judging whether the data is single data or multiple data; for a single datum, covering each equivalence class of the datum and boundary values (7-value method, 9-value method); for multiple data, firstly determining the value of single data, and then carrying out combined test (EC, 2-wise,3-wise, n-wise and the like) on the multiple data; and finally, obtaining specific test data and storing the specific test data.
III, generating continuous data; continuously receiving certain data, wherein the data can be unchanged, but most of the cases are that the data is changed in real time;
and IV, generating time constraint by adopting a five-value method, wherein the five-value method comprises a left boundary-precision, a left boundary, a middle value, a right boundary and a precision value.
S3, establishing a test model of the software according to the established model engineering and the generated specific test data; the software is embedded software.
The established test model is a nestable activity diagram and supports software service coverage of basic service, cyclic processing and concurrent service.
The primitives of the activity graph comprise action nodes, action streams, a start node, an end node, a judgment node, a timer node, a concurrency start node and a concurrency end node, and the action nodes support subgraph nested modeling.
The embedded software system is subjected to test requirement modeling by adopting a formal expression mode of an activity diagram, and then the model is subjected to constraint design and binding of the driving test automatic execution keywords.
The test path of the test model comprises:
common path: there is no loop, no judgment node, no concurrent path.
Circulation path: a path with a loop (loop).
Concurrent path: meaning that two or more activities are performed within the same time interval. For some complex large systems, objects often have more than one control flow at runtime, but two or more concurrently running control flows.
The constraint function expression: logic operation, arithmetic operation, time constraint and data operation and different types of time constraint operation can be carried out on the judgment nodes of the model.
And S4, carrying out constraint design on the test model.
The constraint design comprises a data constraint design, preset conditions, time constraints and a time sequence logic relation.
Sub-graphs of the same type can be nested in an activity graph of the test model, namely the activity graph can be nested in a sub-activity graph, and the constraint design can be carried out based on the test model besides the software functional logic description during modeling, wherein the constraint design is as follows:
designing data constraint: the input data in the data configuration is referred in the activity nodes in the activity graph, and data constraint conditions are added according to the defined data, namely, the constraint is represented by a logic expression, and the branch can be determined under what conditions. The reference definition data is referenced with a $ symbol, all defined data is automatically referenced quickly, with the logical operators { >, <, =, &, | |, |! And the like, and taking the value in the data variable value set once when the data variable is quoted once.
Presetting conditions: and adding preset conditions on the graphic elements of the active graph. The preset conditions are divided into two types: one is the preset condition of the whole test model, which is equivalent to the setting of a precondition before the execution of the function or scene expressed by the test model, and is mostly used for the execution of case sets; another is a preset condition for a certain activity, corresponding to a precondition that must be fulfilled before the activity is performed. The preset condition can be either a literal expression or a logical expression referring to the data.
And (3) time constraint: a time constraint may be expressed on an activity node of an activity graph. The time constraint is divided into absolute time and relative time, and specifically divided into time point constraint, duration constraint and cycle time constraint. The point-in-time constraint is a time that limits the time at which time is generated or ended; the duration constraint is to limit the task execution duration or wait time before starting; the cycle time constraint is to limit the time interval for the task to be repeatedly executed;
the sequential logical relationship is as follows: the following sequential logical relationships of 4 cases can be expressed on the activity diagram. Timing relationships between time points, timing relationships between time points and durations, timing relationships between durations, and timing relationships between time points and cycle times.
S5, binding keywords for driving the automatic execution of the test case with the test model, wherein the specific process is as follows:
and binding each primitive of the test model activity diagram with one or more keywords, wherein the keywords bound by the same primitive are separated by half-angle semicolons.
In order to realize the automatic execution of the keywords in the drive test, firstly, the definition of the keywords is needed, and the keywords mainly comprise the following three parts:
(1) A keyword function name;
(2) Parameters of the keyword function: derived from defined test data;
(3) Type of keyword function: business layer keywords, logic layer keywords and execution layer keywords;
s6, configuring a coverage strategy of the test case, and generating the test case with a fixed format according to specific test data by using the test model obtained in the S5 after the keywords are bound;
the coverage strategy comprises a data coverage strategy, a condition judgment coverage strategy, a path coverage strategy and a sub-graph coverage strategy.
And configuring a coverage strategy for each layer of active graph of the test model according to different test risk levels of the software to determine a test target, combining specific test data according to the data coverage strategy, and selecting path depth, a sub-graph bringing-in strategy and a condition combination based on the test model under the drive of the specific test data until a test case finally meeting the data coverage strategy, the condition judgment coverage strategy, the path coverage strategy and the sub-graph coverage strategy is generated.
Data coverage strategy: the invention provides data coverage strategy selection for each layer of graph, which comprises EC, 2-wise,3-wise, N-wise and non-coverage selection. Wherein EC is a single selection combination, and each factor appears at least once; n is the interaction strength among different data factors, under two special conditions, the single factor covers the standard, and the number of test cases generated by combination is C = max { A {
i When the data factors are completely combined and covered, the number of test cases generated by combination is
Wherein A is
i And the number of the selectable values in the data field for the ith factor after the test data generation processing.
Conditional decision override policy: the present invention provides conditional combinatorial coverage selection, default branch coverage when not hooked, similar to logical coverage in white-box testing.
Path coverage policy: the invention provides 1,2,3, n depth drop-down frame selection, adopts a depth-based path coverage strategy, takes a judgment node as a cross junction of two adjacent paths, calculates the coverage depth of the adjacent paths, and generates more test paths with higher set depth and higher coverage degree, finally generates test cases which are classified by the test paths, and each path may have a plurality of groups of data execution.
Sub-graph coverage strategy: including four relationships, i.e., an expansion relationship, a penetration relationship, and a combination relationship. The unfolding relationship is as follows: in order to display visually or conveniently, the subgraph is used as a part of a parent graph to form a graph independently, and the coverage strategy is consistent with that of the parent graph; the through relation is as follows: any through path in the subgraph is merged into the father graph, and the test thought is layered; the combination relation is as follows: all paths in the subgraph are fully combined with the connection paths of the parent graph, and the condition that a plurality of incoming paths or outgoing paths exist in the parent graph is considered.
S7, screening and confirming the test cases with the fixed format generated in the S6, and integrating the test cases obtained after screening and confirming to be used as a test case execution set;
and screening and confirming the generated test cases with fixed formats according to user-defined conditions or system default conditions, and integrating the test cases obtained after screening and confirmation into a test case execution set.
S8, binding and connecting the test case execution set with the test environments, and traversing the test case execution set by using each test environment after the binding is successful;
according to the test environment set by the system resources in the S2, whether the test cases can be successfully connected with the test environment is checked, and after the connection is successful, the system sends the test cases to each test environment one by one to carry out automatic execution on the test cases; otherwise, the operation ends. The detailed steps of the test case execution are as follows:
(1) And automatically generating a test case based on the data and the coverage strategy.
(2) And screening and confirming the final test cases, and forming a test case execution set by the obtained test cases.
(3) And binding and connecting the test case execution set with a test environment (test platform).
(5) And after the binding is successful, traversing the test case execution set by each test environment, and automatically executing the test cases one by one.
S9, driving each test case in the test case execution set to be automatically executed by using the keywords bound in the S5, wherein the specific process is as follows:
the automatic execution steps of a single test case are as follows:
s91, driving each test case in the test case execution set to generate a corresponding test script by using the keywords bound in S5, wherein the test script is used for calling a function of the keywords corresponding to the test case and displaying parameter values of the function; the specific process is as follows:
s911, analyzing the keyword function, which comprises the following three parts:
(1) Testing key operations to be executed, namely some key actions of actual operations;
(2) An operated object to be triggered by the key action, namely an operated control element;
(3) Data that must be provided for testing, i.e., test data.
S912, outputting the analyzed keywords according to a front-back sequence to test cases;
s913, assigning a plurality of values to the output parameters of the test data of the continuous frame type;
the data defined by the system resource can define data of a continuous frame type, and a specific data structure is similar to an array, for example, if data a has 3 frames, a [3] can be defined, and the parameter value is 3. (assignment array: attribute A = [1,2,3 ]).
S914, dereferencing the parameter value of the keyword as the dereferencing of corresponding test data, and outputting the original keyword without dereferencing to obtain a test script;
s92, sending the test script to a corresponding test environment in the test system;
s93, automatically executing the test script in the test environment, and feeding back the execution condition in real time;
s94, the test system records and dynamically displays the execution condition of the test script in real time, and the execution condition of the corresponding test case can be obtained;
s95, after the execution is finished, the test environment feeds back the execution result to the test system;
and S96, analyzing and updating the execution result of the test case by the test system, and outputting the execution result to a tester.
The test cases and the test scripts are in one-to-one correspondence, one test case is one test script, the test cases are traversed, the scripts corresponding to the test cases are handed to the test platform to be executed, the test platform feeds back results, and then the next test case is executed. The automatic execution of the embedded software test case can be realized, and the test of the embedded software is finally realized.
In the invention, when the application to be tested changes, the testing personnel only need to modify the corresponding key words and the specific logic thereof to realize the change, and the test data and the test logic are completely separated, so that the test case and the test script thereof become easy to maintain, and the test becomes more efficient.