CN105427695B - Program class examination paper automatic assessment method and system - Google Patents

Abstract

The present invention relates to an automatic evaluation method for programming test questions, comprising: S1: the server obtains the code to be evaluated and the preset test case sent by the client; S2: determines the input, output, weight and score corresponding to the test case ; S3: Run the code to be evaluated according to the input of the test case to obtain the output result of the code to be evaluated; S4: Match the output result of the code to be evaluated with the expected output of the test case , and calculate the score of the code to be evaluated according to the weight and score of the test case. By adopting the automatic evaluation method for programming test questions provided by the present invention, the correctness of the program can be automatically judged according to the preset test cases, and the corresponding scores can be returned, which can effectively solve the cumbersome verification of the programming problem assignments submitted by the teacher, It cannot effectively and objectively score questions, so as to better serve programming language courses.

Description

Method and system for automatic evaluation of programming test questions

technical field

The invention relates to the field of educational applications, in particular to an automatic evaluation method and system for programming test questions.

Background technique

In recent years, a new network course teaching mode MOOC (Massive Open Online Course, Massive Open Online Course) has emerged in the field of education. Take edx as an example, because it is free and open, it has attracted countless young students, and its users have gradually increased. Currently, it has more than 900,000 registered users. For massive online education, "massive" means that the number of participants is unlimited; "open" means that anyone can participate in the learning, and it is free; "online" means that participants participate in the course through the network, without Geographically restricted. C language, C++ language, and Java language are widely used programming languages in computer programming at present. C, C++ and other languages are compulsory basic courses in computer and related majors in various colleges and universities.

At present, the school's assessment of programming test questions is mainly based on manual marking by teachers. Human emotions and other reasons during grading cause deviations in grading, which is unfair to students. When students learn programming, they need to install an integrated development environment in the computer room or personal computer, edit the program, compile the program, and run the program. However, when encountering common problems such as compiling and running, it is often difficult to solve them. In addition, when judging the students' programming assignments, the teacher needs to copy the program, compile the program, and run the program before giving the score, which is a heavy workload, time-consuming and labor-intensive.

To sum up, it is very important to design an automatic evaluation method for programming test questions to reduce the workload of teachers.

Contents of the invention

The technical problem to be solved by the present invention is how to provide an automatic evaluation method for programming test questions.

To this end, the present invention proposes a method for automatic evaluation of programming class examination questions, comprising the following steps:

S1: The server obtains the code to be evaluated and the preset test cases sent by the client;

S2: Determine the input, output, weight and score corresponding to the test case;

S3: Run the code to be evaluated according to the input of the test case, and obtain an output result of the code to be evaluated;

S4: Match the output result of the code to be evaluated with the expected output of the test case, and calculate the score of the code to be evaluated according to the weight and score of the test case.

Preferably, the step S1 further includes: judging whether the code to be evaluated contains a prohibited system command, and if the code to be evaluated contains a prohibited system command, stopping the running of the code to be evaluated.

Preferably, the step S1 further includes: compiling the code to be evaluated, obtaining a compiling result, and returning an error message if compiling an error.

Preferably, before step S1, it also includes:

Run the program on the server side, write the process number of the program on the server side into the running file; and start the running program monitoring thread;

Create a Socket through the main thread, bind the Socket, monitor the Socket port, and wait for the connection request of the client; wherein, after receiving the connection of a client, create a new thread to continue processing the connection request of another client.

Preferably, said step S3 also includes:

The name of the code to be evaluated is sent to the running program monitoring thread on the server side through the message queue, and the running process of the code to be evaluated is monitored through the running program monitoring thread.

Preferably, the running program monitoring thread monitoring the running process of the code to be evaluated specifically includes the following steps:

S301: Open the message queue;

S302: Waiting to receive the message queue;

S303: sleep for a first preset time;

S304: Determine whether the code to be evaluated is still running; if it has stopped running, proceed to step S302; if it is still running, continue to sleep for a second preset time;

S305: After the second preset period of dormancy, if the code to be evaluated is still running, forcibly terminate the running of the code to be evaluated; if it has stopped running, continue to step S302.

In yet another aspect, the present invention also provides an automatic evaluation system for programming test questions, including a server end and a client end, and the server end includes a receiving unit, an analyzing unit, an operating unit, and a comparing unit;

The receiving unit is used to obtain the code to be evaluated and the preset test case sent by the client;

The parsing unit is used to determine the input, output, weight and score corresponding to the test case;

The running unit is used to run the code to be evaluated according to the input of the test case, and obtain an output result of the code to be evaluated;

The comparison unit is configured to match the output of the code to be evaluated with the expected output of the test case, and calculate the score of the code to be evaluated according to the weight and score of the test case.

Preferably, the client includes a question-making module and a question-answering module;

The question making module is used to submit programming test questions and the preset test cases to the server;

The answering module is used to submit the code to be evaluated to the server.

Preferably, the client further includes an anti-plagiarism module, and the anti-plagiarism module is used to calculate the similarity between any two codes to be evaluated.

Preferably, the server end is a distributed cluster system, including evaluation management nodes, program running nodes, job management nodes, task nodes, data management nodes and data nodes; the data management nodes and data nodes constitute a distributed file system, Provide an environment for storing program codes and other data for other nodes to process data;

The evaluation management node is used to store the code to be evaluated in the distributed file system through the data management node after receiving the connection request from the client, and assign the program to the code to be evaluated according to the current load condition run node;

The program running node is used for compiling and running the code to be evaluated, and saving the output results of compiling and running in the distributed file system;

The job management node is used to submit the search query code job, and assign the job to each of the task nodes, and perform the big data search and matching of the code concurrently through each of the task nodes, and return the matching result to the job management node ;

The evaluation management node is further configured to return the running output result and the matching result to the client.

By adopting the automatic evaluation method and system for programming class examination questions provided by the present invention, it is possible to learn computer programming courses for online education platforms, such as C programming and C++ programming courses that provide online editing programs, compiling programs, and distributed clusters of running programs Test environment; according to the program use cases set by the teacher, the correctness of the program can be judged in real time and automatically, and the corresponding score can be returned; it can also conduct big data analysis on the submitted program code, analyze the similarity rate of student code, and prevent plagiarism The purpose is to effectively solve the problems of teachers’ cumbersome verification of students’ submission of programming assignments, the inability to effectively and objectively give marks, the inability to give help information for compilation and operation errors, and the inability to give program reference codes, etc., so as to better serve programming language classes. Courses provide services.

Description of drawings

The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the accompanying drawings:

Fig. 1 shows the schematic flow chart of the automatic evaluation method for programming class examination questions of the present invention;

Fig. 2 shows a schematic diagram of the server-side distributed cluster system of the present invention;

Fig. 3 shows a schematic diagram of the running process of the running program monitoring thread monitoring the code to be evaluated in the present invention.

Detailed ways

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Example 1

As shown in Figure 1, the present invention provides a method for automatic evaluation of programming class examination questions, comprising the following steps:

Run the program on the server side, write the process number of the program on the server side into the running file; and start the running program monitoring thread;

Create a Socket through the main thread, bind the Socket, monitor the Socket port, and wait for the client's connection request; wherein, after receiving a client's connection, create a new thread to continue processing another client's connection request.

S1: The server acquires the code to be evaluated and the preset test cases sent by the client; and judges whether the code to be evaluated contains prohibited system commands. For example, by parsing the json character string, it is identified whether prohibited system commands such as fdisk (hard disk partition) and shutdown (shutdown) are included, and if the prohibited system commands are included, the operation of the code to be evaluated is stopped. According to the method, select the compiler and compilation options, compile the code to be evaluated, and obtain the compilation result, and return an error message if the compilation fails.

S2: Determine the input, output, weight and score corresponding to the test case; wherein, multiple groups of test cases can be set, and each group of test cases has different inputs and expected outputs; each group of test cases can be set with different weights, resulting in different scores. In other words, even if the code program is opportunistic, it can only pass a set of test cases, and it is impossible to pass all or part of the test cases.

S3: According to the input of the test case, run the code to be evaluated to obtain the output of the code to be evaluated; in order to strengthen the robustness, stability, and security of the system program, it is necessary to monitor the operation of the evaluation service. If the service program crashes or is maliciously damaged for some reason, the system can automatically resume and start. Therefore, before running the evaluation code program, the name of the code to be evaluated is sent to the running program monitoring thread of the server through the IPC message queue, and the running process of the code to be evaluated is monitored through the running program monitoring thread. The code submitted by the client needs to have a running program monitoring thread to monitor the operation of these codes, including: performing virus and malicious code scanning checks on the program code; ) to scan and check; allocate, release, and check program code memory, and the system should automatically release and recycle the memory that the user program has not released after the program runs; if there are problems or design defects in the program code, such as program infinite loop, the running program will continue to run Blocking, empty CPU resource consumption, etc. are checked. Once an abnormal running program is detected, the running of the program will be terminated immediately. In addition, programs are not allowed to use the sleep function. If a program runs for more than 1000 milliseconds, we consider the program to enter an infinite loop, and the system background will forcibly terminate the program to ensure the safety of the evaluation system.

As shown in Figure 3, the running process of the running program monitoring thread monitoring the code to be evaluated specifically includes the following steps:

S301: Open the message queue;

S302: Waiting to receive the message queue;

S303: sleep for a first preset time;

S304: Determine whether the code to be evaluated is still running; if it has stopped running, proceed to step S302; if it is still running, continue to sleep for a second preset time;

S305: After the second preset period of dormancy, if the code to be evaluated is still running, forcibly terminate the running of the code to be evaluated; if it has stopped running, continue to step S302. Wherein, the first preset time may be 300ms (millisecond), and the second preset time may be 1s (second).

S4: Match the output result of the code to be evaluated with the expected output of the test case, and calculate the score of the code to be evaluated according to the weight and score of the test case.

Example 2

By adopting the automatic evaluation method for programming type examination questions described in Embodiment 1, the present invention also provides an automatic evaluation system for programming type examination questions, including a server end and a client end, and the server end includes a receiving unit, an analysis unit, an operating unit and a comparison unit;

The receiving unit is used to obtain the code to be evaluated and the preset test case sent by the client;

The parsing unit is used to determine the input, output, weight and score corresponding to the test case;

The running unit is used to run the code to be evaluated according to the input of the test case, and obtain an output result of the code to be evaluated;

The comparison unit is used to match the output result of the code to be evaluated with the expected output of the test case, and calculate the score of the code to be evaluated according to the weight and score of the test case.

Preferably, the client may include a questioning module and an answering module; the questioning module is used to submit programming questions and the preset test cases to the server; the answering module is used to submit The server end submits the code to be evaluated. The client also includes an anti-plagiarism module, and the anti-plagiarism module is used to calculate the similarity between any two copies of the codes to be evaluated.

As shown in Figure 2, the server side is a distributed cluster system, including evaluation management nodes, program running nodes, job management nodes, task nodes, data management nodes and data nodes; the data management nodes and data nodes constitute a distributed file The system provides an environment for storing program codes and other data for other nodes to process data;

The evaluation management node is used to store the code to be evaluated in the distributed file system through the data management node after receiving the connection request from the client, and assign the program to the code to be evaluated according to the current load condition run node;

The program running node is used for compiling and running the code to be evaluated, and saving the output results of compiling and running in the distributed file system;

The job management node is used to submit the search query code job, and assign the job to each of the task nodes, and perform the big data search and matching of the code concurrently through each of the task nodes, and return the matching result to the job management node ;

The evaluation management node is further configured to return the running output result and the matching result to the client.

Specifically, in the question making module, teachers can add programming questions, set programming question types, scores, start and end times, etc., fill in programming questions, problem descriptions, test cases, and click submit to generate a programming question. Among them, multiple groups of test cases can be set, and each group of test cases has different inputs and expected outputs; each group of test cases can be set with different weights to obtain different scores. In the answering module, students can edit the code to be evaluated in the "Edit Code" area. If the code needs to input values, they can also fill in the corresponding input values. They can click the "Run" button to compile the code. If there is a problem, a compilation error will be displayed; if the compilation fails, the "compilation result" output box will give a corresponding compilation error message prompt. If the code runs normally, only the running result is displayed, and the compilation result is not displayed. When the student clicks the "Submit" button, the code will be uploaded to the server, and then handed over to the server for automatic evaluation. According to the input and output of the test case preset by the teacher, it will be fuzzy matched with the code to be evaluated submitted by the student. If the source code After passing multiple sets of test cases set before, you can get a full score. If a set of test cases fails, the score will be calculated according to the corresponding proportion.

The anti-plagiarism module is mainly for teachers to check the similarity of programming assignments among students. Calculate the similarity of any two students A and B's homework of the same programming problem, and calculate the similarity between each line of code of student A's homework to be tested and each line of code of student B's homework (code lines with less than 15 words will not be performed) Calculation), the highest value is taken as the similarity of the line of code of two people's homework, and the average of the similarity of all lines of code is calculated as the similarity of the two students' homework of this programming problem.

By adopting the automatic evaluation method and system for programming class examination questions provided by the present invention, it is possible to learn computer programming courses for online education platforms, such as C programming and C++ programming courses that provide online editing programs, compiling programs, and distributed clusters of running programs Test environment; according to the program use cases set by the teacher, the correctness of the program can be judged in real time and automatically, and the corresponding score can be returned; it can also conduct big data analysis on the submitted program code, analyze the similarity rate of student code, and prevent plagiarism The purpose is to effectively solve the problems of teachers’ cumbersome verification of students’ submission of programming assignments, the inability to effectively and objectively give marks, the inability to give help information for compilation and operation errors, and the inability to give program reference codes, etc., so as to better serve programming language classes. Courses provide services.

Through the above description of the implementation manners, those skilled in the art can clearly understand that the present invention can be implemented by hardware, or by means of software and a necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.), including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present invention.

Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention. within the bounds of the requirements.

Claims (8)

1. a kind of programming class examination paper automatic assessment method, which is characterized in that include the following steps：

S1：Server end obtains the code to be tested and assessed and preset test case that client is sent；

S2：Determine the corresponding input of the test case, output, weight and score；

S3：According to the input of the test case, code to be tested and assessed described in operation obtains the output knot of the code to be tested and assessed Fruit；

S4：The output result of the code to be tested and assessed is matched with the desired output of the test case, and according to The weight and score of the test case, the score of code to be tested and assessed described in calculating；

Wherein, the step S3 further includes：

The title of the code to be tested and assessed is sent to the operation program monitoring thread of the server end by message queue, is led to Cross the operational process of code to be tested and assessed described in the operation program monitoring thread monitoring；

Wherein, the operational process of code to be tested and assessed described in the operation program monitoring thread monitoring specifically includes following steps：

S301：Open the message queue；

S302：Etc. the message queue to be received；

S303：The first preset time of suspend mode；

S304：Whether code to be tested and assessed described in judgement is still running；If out of service, step S302 is gone successively to； If still run, continue the second preset time of suspend mode；

S305:It is to be measured described in forced termination if the code to be tested and assessed still is being run after between suspend mode second is default Comment the operation of code；If out of service, step S302 is gone successively to.

2. a kind of programming class examination paper automatic assessment method according to claim 1, which is characterized in that the step S1 is also wrapped It includes：Whether include forbidden system command in code to be tested and assessed described in judgement, if including forbidden system command, stops The only operation of the code to be tested and assessed.

3. a kind of programming class examination paper automatic assessment method according to claim 1 or 2, which is characterized in that the step S1 Further include：Code to be tested and assessed described in compiling, and compiling is obtained as a result, returning to error message if compiling and malfunctioning.

4. a kind of programming class examination paper automatic assessment method according to claim 1 or 2, which is characterized in that step S1 it Before further include：

Operating file is written in the process number of the program of the server end by the program for running the server end；And open fortune Line program monitoring thread；

Socket is created by main thread, binding Socket, monitors the ports Socket, and the connection of the client is waited for ask It asks；Wherein, after the connection for receiving a client, the connection request that a thread continues with another client is created.

5. a kind of automatic evaluation system of programming class examination paper, which is characterized in that including server end and client, the server end Including receiving unit, resolution unit, running unit and comparing unit；

The receiving unit is for obtaining the code to be tested and assessed and preset test case that client is sent；

The resolution unit is for determining the corresponding input of the test case, output, weight and score；

The running unit is used for according to the input of the test case, code test and assess described in operation, obtain described in wait testing and assessing The output result of code；

The comparing unit be used for by the output result of code test and assess and the test case it is desired it is described export into Row matching, and according to the weight and score of the test case, the score of code to be tested and assessed described in calculating；

Wherein, the running unit is additionally operable to：

The title of the code to be tested and assessed is sent to the operation program monitoring thread of the server end by message queue, is led to Cross the operational process of code to be tested and assessed described in the operation program monitoring thread monitoring；

Wherein, the operational process of code to be tested and assessed described in the operation program monitoring thread monitoring specifically includes following steps：

S301：Open the message queue；

S302：Etc. the message queue to be received；

S303：The first preset time of suspend mode；

S304：Whether code to be tested and assessed described in judgement is still running；If out of service, step S302 is gone successively to； If still run, continue the second preset time of suspend mode；

S305:It is to be measured described in forced termination if the code to be tested and assessed still is being run after between suspend mode second is default Comment the operation of code；If out of service, step S302 is gone successively to.

6. the automatic evaluation system of programming class examination paper according to claim 5, which is characterized in that the client includes setting a question Module, answer module；

The module of setting a question to the server end for submitting programming examination question and the preset test case；

The answer module is used for code to be tested and assessed described in server end submission.

7. the automatic evaluation system of programming class examination paper according to claim 6, which is characterized in that the client further includes preventing Plagiarize module, the anti-similarity plagiarized module and be used to calculate code to be tested and assessed described in arbitrary two parts.

8. the automatic evaluation system of programming class examination paper according to claim 5 or 6, which is characterized in that

The server end is distributed cluster system, including evaluation and test management node, program operation node, job management node, Task node, data management node and back end；The data management node and back end constitute distributed file system, Save routine code and the environment of other data are provided for other each node processing data；

The evaluation and test management node is used for after receiving the connection request of client, will code be tested and assessed by data management node It is saved in the distributed file system, and according to current loading condition to the code distribution described program fortune to be tested and assessed Row node；

Described program operation node for compile, run described in code test and assess, and will compile, run output result be stored in In the distributed file system；

The job management node distributes operation to each task node for submitting search inquiry code operation, leads to It crosses each task node and concurrently returns to the working pipe into the search of the big data of line code, matching, and by matching result Manage node；

The evaluation and test management node is additionally operable to the output result and matching result of the operation returning to the client.