WO2016206505A1 - Automated testing system and testing method - Google Patents

Abstract

An automated testing system, comprising: a version update module, set up to detect the testing software version and implement updates; a static inspection module, set up to implement static inspection of the programs in the updated testing software; a compiling module, set up to implement compiling of the programs in the testing software after static inspection is complete; a burning module, set up to write the programs in the compiled testing software into a tested device; a testing execution module, set up to execute automated testing on the tested device on the basis of a preset testing policy; and a testing feedback module, set up to feed back the results of the automated testing and testing log information. The present solution implements modularisation of each automated testing function, such that each module has an independent function, and thus the entire testing method has strong portability and scalability, reducing the problem of unscientific testing caused by human factors, and improving research and development efficiency.

Description

Automated test system and test method Technical field

This article covers, but is not limited to, the field of embedded device testing, involving automated test systems and test methods.

Background technique

With the rapid development of embedded devices, especially the rapid development of embedded communication network devices, embedded communication network devices with high transmission efficiency, large system capacity and low transmission cost are also receiving more and more attention, such as optical network transmission equipment. . However, because the optical network transmission equipment supports many functions and many types of boards, the version of the test program is also released. Therefore, the tester has to perform a large number of repeated operations every time the test is scheduled, resulting in an increase in time and labor costs, and test efficiency. reduce.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

The embodiment of the invention provides an automatic test system and a test method, which solves the technical problem that the related automatic test method cannot meet the test characteristics of the optical network transmission device, thereby increasing the time and labor cost of the test and reducing the test efficiency.

An automated test system provided by an embodiment of the present invention includes: a version update module, a static check module, a compiling module, a burning module, a test execution module, and a test feedback module;

The version update module is configured to detect a version of the test software and update the same;

The static check module is configured to perform static check on the program in the updated test software;

The compiling module is configured to compile a program in the test software after the static check is completed;

The programming module is configured to write the program in the compiled test software to the device under test;

The test execution module is configured to perform an automated test on the device under test according to a preset test strategy;

The test feedback module is configured to feed back automated test results and test log information.

Optionally, the automated test system further includes:

The jump module is set to determine whether the current module ends execution and jumps to the next module after the current module execution ends.

Optionally, the automated test system further includes:

The adaptive decision module is configured to determine whether to continue the automated test when a failure occurs during the automated test.

Optionally, the adaptive decision module is set to:

When the test case fails, it is determined whether to continue the automatic test according to the preset fault level.

Optionally, the adaptive decision module determines whether to continue to perform the automatic test according to a preset fault level when the test case fails.

When the test case fails and is the preset first type of fault level, the test of the current test case is terminated and the tester is notified;

When the test case fails and is the default second level of fault, continue with the next test case.

An embodiment of the present invention further provides a testing method, where the testing method includes:

When there is an update of the version of the test software, the automated test system obtains the updated test software, and performs static check on the program in the updated test software;

Compiling the program in the test software after the static check is completed, and writing the program in the compiled test software to the device under test;

Perform automated testing on the device under test according to a preset test strategy, and feed back automated test results and test log information.

Optionally, the method further includes: when the version of the test software is updated, the automated test system detects the updated test software and performs static check on the program in the test software, the automated test system detects Test the version of the software for updates.

Optionally, the method further includes:

When a failure occurs during an automated test, the automated test system decides whether to continue The automated test described.

Optionally, when the fault occurs in the automated testing process, the automated testing system determines whether to continue to perform the automated testing, including:

When the test case fails, the automated test system decides whether to continue the automated test based on the preset fault level.

Optionally, when the test case fails, the automated test system determines whether to continue to perform the automated test according to a preset fault level, including:

When the test case fails and is a preset first type of fault level, the automated test system terminates the test of the current test case and notifies the tester;

When the test case fails and is a preset second type of fault level, the automated test system continues to execute the next test case.

The embodiment of the invention modularizes each function of the automated test, so that each module functions independently, thereby making the whole test method have strong portability and scalability, and can be fully automated, thereby reducing human factors. The unscientific test of the test has improved the efficiency of research and development. In addition, the embodiment of the present invention also considers a situation in which the test process is faulty. By testing the interrupt adaptive mode, the test program decides whether to continue the test, thereby providing a good fault tolerance mechanism for the test interrupt condition, and improving the test efficiency. Reduced product development costs. The embodiment of the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and when the computer executable instructions are executed, the test method is implemented.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

DRAWINGS

1 is a schematic diagram of functional modules of an automated test system according to a first embodiment of the present invention;

2 is a structural block diagram of an automated test system according to a first embodiment of the present invention;

3 is a schematic diagram of a layered framework of an automated test system according to a first embodiment of the present invention;

4 is a schematic diagram of functional modules of an automated test system according to a second embodiment of the present invention;

5 is a schematic diagram of functional modules of an automated test system according to a third embodiment of the present invention;

6 is a schematic diagram of an adaptive process of the adaptive decision module of FIG. 5;

FIG. 7 is a schematic flowchart diagram of a testing method according to a fourth embodiment of the present invention; FIG.

FIG. 8 is another schematic flowchart of a testing method according to a fifth embodiment of the present invention.

detailed description

It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the invention.

First embodiment

Embodiments of the present invention provide an automated test system.

Referring to FIG. 1, FIG. 1 is a schematic diagram of functional modules of an automated test system according to an embodiment of the present invention.

In this embodiment, the automated test system includes: a version update module 10, a static check module 20, a compiling module 30, a programming module 40, a test execution module 50, and a test feedback module 60;

The version update module 10 is configured to detect and update the version of the test software;

According to the test needs, the tester needs to continuously modify the test software to achieve the test purpose. After the tester has modified and uploaded the test software, the version update module 10 will automatically detect whether there is an update of the version of the test software. If there is an update, the latest version of the test software is automatically downloaded and updated. The test software includes test data for testing, test cases, and test function sets for performing tests.

The static check module 20 is configured to perform static check on the program in the updated test software;

The compiling module 30 is configured to compile the program in the test software after the static check is completed;

The programming module 40 is configured to write the program in the compiled test software to the device under test;

In this embodiment, the device under test includes an embedded device, and the related program in the test software needs to be written into the device under test to facilitate testing. In this embodiment, the static check module 20 performs static check on the program in the updated test software, such as checking the syntax structure of the program, the array is out of bounds, the divisor is zero, and the buffer overflows; Static check The program is compiled; the program in the compiled test software is written into the device under test through the programming module 40. In addition, optionally, when an error occurs during static checking or compiling or burning, the static checking module 20 or the compiling module 30 or the programming module 40 is further configured to feed back the error information that appears to the tester.

The test execution module 50 is configured to perform an automated test on the device under test according to a preset test strategy;

The test feedback module 60 is configured to feed back automated test results and test log information.

After the relevant program of the test is written to the device under test, the test execution module 50 automatically performs an automatic test according to the preset test strategy, and completes the test execution process. The default test strategy includes the type of test and the corresponding test cases, such as regression test, stress test, and so on. At the same time, the test cases used by different test types can be the same or different. After the automated test is finished, the test feedback module 60 feeds back the automated test results and the test log information to the tester, for example, by sending the test result and the test log to the tester's mailbox by email, so that the tester can follow up and understand the test situation in time. And make subsequent modifications to the test equipment or test procedures.

The block diagram of the automated test system shown in Figure 2. The automated test system includes an automation server, a version server, and a display kanban.

The automation server is responsible for automating use case management, parameter configuration, test execution and feedback of test results, and interacts with the device under test to complete the automated test process; the version server is responsible for checking and updating the software version, compiling and programming. And pass the relevant software program to the device under test; the display kanban is set to display software version check, compile, burn, and test result display. In addition, it should be noted that, in this embodiment, the device under test includes an embedded device, which may be an optical network transmission device. In this embodiment, the version update module 10, the static check module 20, the compiling module 30, and the burning module 40 may be disposed on the version server, the test execution module 50 may be disposed on the automation server, and the test feedback module 60 may be disposed on the display. Watch on the board.

A schematic diagram of a layered framework of the automated test system of the present invention as shown in FIG. In this embodiment, a modular design idea is introduced, and the automated test system is regarded as a framework with a three-layer architecture, which is a support layer, a service layer, and an application layer. The support layer is used to support the underlying device communication protocol to send and receive message platform functions, and can ensure automatic measurement after completing the encapsulation of the underlying device communication protocol. The test system interacts with the communication information of the device under test; the service layer mainly encapsulates the message interface according to the service of the embedded device product, and completes an API (Application Programming Interface) (such as security, performance, maintenance, and alarm). Directly shield the function differences of each device and be called by the upper application layer; the application layer completes the service parameter configuration and use case management functions, and at the same time guarantees the execution result and log output.

The modular design adopted in this embodiment has strong portability and scalability, and thus can meet the testing requirements of different embedded device products and different test software versions. The entire execution process of the automated test system simulates the network management operation, thereby ensuring the consistency of the test environment steps, avoiding irregularities or differences in human operation, and automatically feeding back test results. By modularizing the functions of the automated test, the functions of each module are independent, which makes the whole test method highly portable and scalable, and can be fully automated, thereby reducing the unscientific test caused by human factors and improving R&D. effectiveness.

Second embodiment

Referring to FIG. 4, FIG. 4 is a schematic diagram of functional modules of an automated test system according to a second embodiment of the present invention. Based on the foregoing automated test system, the automated test system in this embodiment further includes:

The jump module 70 is configured to determine whether the current module ends execution and jump to the next module after the current module execution ends.

It should be noted that the current module in this embodiment refers to the running module that is being executed during the automatic testing process, and may be a version update module 10, a static check module 20, a compiling module 30, a burning module 40, a test executing module 50, and Test any of the feedback modules 60.

In this embodiment, the jump module 70 determines whether the current module is executed and automatically jumps to the next module after the execution of the current module, as the functions of the modules are independent of each other. Thereby, the entire functional modules of the automated test system are connected in series and executed in sequence. Specifically, the end tag and the calling address of the next module can be set in each module.

This embodiment is an alternative embodiment for fully automating the test execution process of an automated test system. In addition, you can automate the execution according to the calling sequence in the configuration file by setting the configuration file that is executed automatically; or each module notifies the next module by sending a message notification after the execution ends. The block is executed automatically. Through fully automated execution, the test caused by the reduction of human factors is unscientific. At the same time, each module function is independent of each other and fully automated. It can adapt to the processing of embedded devices and has many types of boards, resulting in a large number of released versions. , testing the technical problems of heavy tasks, thereby saving test time and labor costs, improving test efficiency and development process, and reducing development costs.

Third embodiment

Referring to FIG. 5, FIG. 5 is a schematic diagram of functional modules of an automated test system according to a third embodiment of the present invention. Based on the foregoing automated test system, the automated test system in this embodiment further includes:

The adaptive decision module 80 is configured to determine whether to continue the test when a failure occurs during the test.

In this embodiment, considering the strong correlation between the test of the embedded device, especially the optical network transmission device and the environment, the fault analysis and positioning of the optical network transmission device is determined to be inseparable from the actual environment of the site, and therefore, when testing When an external environmental change or a test unexpected interruption occurs in the process, the adaptive decision module 80 determines whether to continue the test. The automatic test system can determine whether to continue the test by presetting the processing method corresponding to all the faults that may occur.

In this embodiment, in order to better locate the faulty on-site environment of the device under test, and also ensure that the entire test process is automatically executed, the adaptive decision module 80 determines the fault that occurs during the test, and if the test is terminated, the test site is retained; otherwise Continue to perform the test.

Optionally, the adaptive decision module 80 is configured to: when the test case fails, determine whether to continue the test according to the preset fault level.

Executing test cases is the core part of the entire automated test, and there are many test cases that need to be executed during the entire automated test process. Generally, automated test tests are performed according to the set test sequence, and when a test case fails. The tester needs to manually modify or adjust the execution order of the test cases; or when multiple test cases fail, it will take a lot of time to manually modify or adjust, which affects the test efficiency of the entire automated test.

In this embodiment, when the test case fails, the adaptive decision module 80 determines whether to continue the test according to the preset fault level, thereby better positioning the fault environment of the device under test, and ensuring the entire test process. Automated execution.

Optionally, referring to FIG. 6, the adaptive decision module 80 determines whether to continue to perform the automatic test according to a preset fault level when the test case fails.

When the test case fails and is the preset first type of fault level, the test of the current test case is terminated and the tester is notified;

When the test case fails and is the default second level of fault, continue with the next test case.

In this embodiment, according to the termination execution and the continuation execution, the fault levels corresponding to the test cases can be correspondingly divided into two categories, for example, the test case failures that are relatively serious and need to be processed by the test personnel in time are preset as the first type of fault level. Other test case failures are preset to the second type of fault level. When the test case fails and is the preset first type of fault level, the test of the current test case is terminated to retain the field fault environment, and the tester is informed by short message or mail; and when the test case fails When the second type of fault level is preset, continue with the next test case.

Fourth embodiment

Referring to FIG. 7, FIG. 7 is a schematic flowchart of a first testing method according to a fourth embodiment of the present invention. In this embodiment, the testing method includes:

Step S10, when there is an update of the version of the test software, the automated test system obtains the updated test software and performs static check on the program in the updated test software;

According to the test needs, the tester needs to continuously modify the test software to achieve the test purpose. When the tester modifies and uploads the test software, that is, when the version of the test software is updated, the automated test system will automatically download and update the latest version of the test software, and perform static check on the program in the test software, such as checking The syntax structure of the program, the array is out of bounds, the divisor is zero, and the buffer overflows. Test software includes test data for testing, test Examples and test function sets for performing tests.

Step S20, compiling the program in the test software after the static check is completed, and writing the program in the compiled test software to the device under test;

In this embodiment, the device under test includes an embedded device. Therefore, it is necessary to write related programs in the test software to the device under test to facilitate testing. The automated test system compiles the program that passes the static check and writes the program in the compiled test software to the device under test. In addition, optionally, when an error occurs during static checking or compiling or burning, the automated test system feeds back error information to the tester.

Step S30: Perform an automated test on the device under test according to a preset test strategy, and feed back the automated test result and the test log information.

When the relevant program of the test is written to the device under test, the automated test system will automatically perform the automated test according to the preset test strategy, and complete the test execution process. The default test strategy includes the type of test and the corresponding test cases, such as regression test, stress test, and so on. At the same time, the test cases used by different test types can be the same or different. When the automated test is over, the automated test system feeds the automated test results and test log information back to the tester. For example, the test results and test logs are sent to the tester's mailbox by email, so that the tester can follow up and understand the test in time. And make subsequent modifications to the test equipment or test procedures.

The modular design adopted in this embodiment has strong portability and scalability, and thus can meet the testing requirements of different embedded device products and different test software versions. The entire execution process of the automated test system simulates the network management operation, thereby ensuring the consistency of the test environment steps, avoiding irregularities or differences in human operation, and automatically feeding back test results. By modularizing each function of the automated test, each module is functionally independent, which makes the whole test method highly portable and expandable, reducing the unscientific test caused by human factors and improving the research and development efficiency.

Fifth embodiment

Referring to FIG. 8, FIG. 8 is a schematic flowchart diagram of a testing method according to a fifth embodiment of the present invention. Based on the foregoing test method, in the embodiment, the method further includes: before the step S10:

In step S00, the automated test system detects whether there is an update of the version of the test software.

According to the test needs, the tester needs to continuously modify the test software to achieve the test purpose. In this embodiment, when the tester modifies and uploads the test software, the automated test system will automatically detect whether the version of the test software is updated. If there is an update, the latest version of the test software is automatically downloaded and updated.

Optionally, in this embodiment, performing the automated test on the device under test according to the preset test policy in step S30 includes: when a fault occurs during the test, the automated test system determines whether to continue the test.

In this embodiment, considering the strong correlation between the test of the embedded device, especially the optical network transmission device and the environment, the fault analysis and positioning of the optical network transmission device is determined to be inseparable from the actual environment of the site, and therefore, when testing When an external environmental change or a test failure is interrupted during the process, the automated test system determines whether to continue the test. The automatic test system can determine whether to continue the test by presetting the processing method corresponding to all the faults that may occur.

In this embodiment, the field environment in which the device under test is faulty is better positioned, and the entire test process is ensured to be automatically executed. The fault occurs in the test process through the automatic test system, and the test site is retained if the test is terminated; otherwise, the test is continued. test.

Optionally, based on the foregoing embodiment, when a fault occurs during the testing, determining whether to continue to perform the test includes: when the test case fails, the automated test system determines whether to continue the test according to the preset fault level.

Executing test cases is the core part of the entire automated test, and there are many test cases that need to be executed during the entire automated test process. Generally, automated test tests are performed according to the set test sequence, and when a test case fails. The tester needs to manually modify or adjust the execution order of the test cases; or when multiple test cases fail, it will take a lot of time to manually modify or adjust, which affects the test efficiency of the entire automated test.

In this embodiment, when the test case fails, the automated test system determines whether to continue the test according to the preset fault level, thereby better positioning the fault environment of the device under test, It also ensures the automated execution of the entire test process.

Optionally, based on the foregoing embodiment, when the test case fails, determining whether to continue the test according to the preset fault level includes:

When the test case fails and is a preset first type of fault level, the automated test system terminates the test of the current test case and notifies the tester;

When the test case fails and is a preset second type of fault level, the automated test system continues to execute the next test case.

In this embodiment, according to the termination execution and the continuation execution, the fault levels corresponding to the test cases can be correspondingly divided into two categories, for example, the test case failures that are relatively serious and need to be processed by the test personnel in time are preset as the first type of fault level. Other test case failures are preset to the second type of fault level. When the test case fails and is the preset first type of fault level, the automated test system terminates the test of the current test case to preserve the on-site fault environment, while notifying the tester for analysis by short message or mail; and when the test case When a fault occurs and is a preset second type of fault level, the automated test system continues to execute the next test case.

The embodiment of the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and when the computer executable instructions are executed, the test method is implemented.

One of ordinary skill in the art will appreciate that all or a portion of the above steps may be performed by a program to instruct related hardware, such as a processor, which may be stored in a computer readable storage medium, such as a read only memory, disk or optical disk. Wait. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the above embodiment may be implemented in the form of hardware, for example, by implementing an integrated circuit to implement its corresponding function, or may be implemented in the form of a software function module, for example, executing a program stored in the memory by a processor. / instruction to achieve its corresponding function. This application is not limited to any specific combination of hardware and software. A person skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application, and should be covered in the present application. Within the scope of the claims.

Industrial applicability

The above technical solutions have strong portability and scalability, can be fully automated, can reduce the unscientific testing caused by human factors, improve research and development efficiency, and reduce product development costs.

Claims (10)

An automated test system includes: a version update module, a static check module, a compiling module, a burning module, a test execution module, and a test feedback module; The version update module is configured to detect a version of the test software and update the same; The static check module is configured to perform static check on the program in the updated test software; The compiling module is configured to compile a program in the test software after the static check is completed; The programming module is configured to write the program in the compiled test software to the device under test; The test execution module is configured to perform an automated test on the device under test according to a preset test strategy; The test feedback module is configured to feed back automated test results and test log information. The automated test system of claim 1 further comprising: The jump module is set to determine whether the current module ends execution and jumps to the next module after the current module execution ends. The automated test system of claim 1 or 2, further comprising: The adaptive decision module is configured to determine whether to continue the automated test when a failure occurs during the automated test. The automated test system of claim 3 wherein said adaptive decision module is configured to: When the test case fails, it is determined whether to continue the automatic test according to the preset fault level. The automated test system of claim 4, wherein the adaptive decision module determines whether to continue to perform the automated test according to a preset fault level when the test case fails: When the test case fails and is the preset first type of fault level, the test of the current test case is terminated and the tester is notified; When the test case fails and is the default second level of fault, continue with the next test case. A test method, the test method comprising: When there is an update of the version of the test software, the automated test system obtains the updated test software, and performs static check on the program in the updated test software; Compiling the program in the test software after the static check is completed, and writing the program in the compiled test software to the device under test; Perform automated testing on the device under test according to a preset test strategy, and feed back automated test results and test log information. The test method according to claim 6, further comprising: said, when the version of the test software is updated, the automated test system acquires the updated test software and performs a static check on the program in the test software, The automated test system detects if there is an update to the version of the test software. The test method according to claim 6 or 7, further comprising: When a failure occurs during an automated test, the automated test system decides whether to proceed with the automated test. The test method according to claim 8, wherein said automatic test system decides whether to continue execution of said automated test when a failure occurs during an automated test, comprising: When the test case fails, the automated test system decides whether to continue the automated test based on the preset fault level. The testing method according to claim 9, wherein when the test case fails, the automated test system determines whether to continue executing the automated test according to a preset fault level, including: When the test case fails and is a preset first type of fault level, the automated test system terminates the test of the current test case and notifies the tester; When the test case fails and is a preset second type of fault level, the automated test system continues to execute the next test case.

Images