WO2025005339A1 - Method, program, and apparatus for generating test - Google Patents

Abstract

Disclosed are a method, a program, and an apparatus for generating a test, according to an embodiment of the present disclosure. The method may comprise the steps of: acquiring input data regarding attributes of a test intended by a user; on the basis of a user input, determining detection conditions for monitoring behaviors of test takers; and opening the test on the basis of the determined detection conditions.

Description

Test generation method, program and device

The present disclosure relates to a test generation method, and more particularly, to a method and device for setting conditions necessary for test management and supervision in a computing environment, and generating a test according to the set conditions.

In an educational environment where tests are administered, it is necessary to monitor the actions of test takers during the test period. In particular, unlike offline tests, it is difficult to effectively check the actions of test takers and their surroundings in online tests. Therefore, in an environment where online tests are administered, it is more important for administrators to accurately analyze the actions taken by test takers in real time to determine whether there was any cheating.

Meanwhile, in order to determine whether the actions taken by the examinee are cheating, it is also important to establish criteria for cheating. Although there are qualitative criteria that are generally agreed upon, since such criteria are not quantitatively defined based on offline exams, it is difficult to directly apply them to online exams. Therefore, in an environment where online exams are administered, it is necessary to appropriately establish detection criteria for monitoring behaviors such as cheating by comprehensively considering the characteristics of the exam, environment, and proctoring tendencies when opening the exam. In addition, it is necessary to supplement the detection criteria for monitoring the examinee's behavior through feedback using data obtained through the operation of the exam.

The present disclosure has been made in response to the aforementioned background technology, and aims to provide a test construction environment capable of generating and recommending detection conditions necessary for test supervision in accordance with the properties of a test, and in which user customization of detection conditions is easy.

However, the problems to be solved in the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

According to one embodiment of the present disclosure for realizing the task as described above, a test generation method is disclosed, which is performed by a computing device. The method may include a step of obtaining input data regarding properties of a test intended by a user; a step of determining a detection condition for monitoring behavior of test takers based on the user input; and a step of opening a test based on the determined detection condition.

Alternatively, the attributes of the test may include at least one of the test type, whether it is open book, whether it is searchable, or the test period.

Alternatively, the step of determining a detection condition for monitoring the behavior of test takers based on said user input may include the step of determining the detection condition based on user input via a first user interface to which a recommended ruleset is exposed.

Alternatively, the step of determining a detection condition based on a user input through the first user interface to which the recommended ruleset is exposed may include the step of generating a recommended ruleset from a rule database based on input data regarding the properties of the test; and the step of determining the detection condition based on the selected ruleset when either the recommended ruleset or a preset exposed through the first user interface is selected according to the user input.

Alternatively, the above rule database can be updated based on the results of analyzing big data accumulated as the test is opened and operated.

Alternatively, the big data may include at least one of data on detection conditions adjusted by the user's rule set customizing during the process of opening a test or data on events detected based on rules during the process of running an opened test.

Alternatively, the update may be to create a rule to be included in the rule database, or to dynamically change a detail of a rule that constitutes a ruleset included in the rule database.

Alternatively, when one of the recommended rulesets or presets exposed through the first user interface is selected according to the user input, the step of determining the detection condition based on the selected ruleset may include a step of adjusting the intensity of the detection condition according to a user input for controlling the level of the selected ruleset. In addition, the level of the ruleset may be distinguished according to the numerical value of the detailed items of the rules constituting the ruleset.

Alternatively, the step of determining detection conditions for monitoring the test takers' behavior based on said user input may include the step of determining the detection conditions based on user input via a second user interface for customizing the ruleset.

Alternatively, the step of determining the detection condition based on user input via the second user interface for customizing the ruleset may include a step of determining the detection condition based on a ruleset including the set or adjusted rule when at least one rule included in the ruleset is set or adjusted based on the user input via the second user interface.

Alternatively, when setting or adjusting at least one rule included in the ruleset is performed according to user input through the second user interface, the step of determining a detection condition based on the ruleset including the set or adjusted rule may include: when setting or adjusting at least one rule is performed, a step of analyzing a detailed item determined to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied; and a step of recommending a detailed item determined to be suitable for the set or adjusted rule based on a result of the analysis.

Alternatively, when setting or adjusting at least one rule is performed, the step of analyzing a detailed item determined to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied may include a step of analyzing weights of the rules based on big data of a test to which rules corresponding to the set or adjusted rule are applied; and a step of determining a detailed item of a rule selected according to the analyzed weights as a detailed item suitable for the set or adjusted rule.

Alternatively, the big data that influences the analysis of the above weights may include at least one of the rate at which a result detected according to a particular rule was delivered to the test taker, or the number of times a particular rule was activated by the test taker's behavior satisfying a subitem of the particular rule.

According to one embodiment of the present disclosure for achieving the above-described task, a test generation method performed by a computing device is disclosed. The method may include the steps of displaying a first user interface in which a recommended ruleset generated from a rule database based on input data regarding properties of a test intended by a user is exposed; and the steps of displaying a second user interface area for receiving user input for customizing the ruleset.

According to one embodiment of the present disclosure for achieving the above-described task, a computer program stored in a computer-readable storage medium is disclosed. When the computer program is executed on one or more processors, it causes operations for generating a test. At this time, the operations may include an operation of obtaining input data regarding properties of a test intended by a user; an operation of determining a detection condition for monitoring behavior of test takers based on a user input; and an operation of opening a test based on the determined detection condition.

According to one embodiment of the present disclosure for achieving the above-described task, a computing device for generating a test is disclosed. The device may include a processor including at least one core; a memory including program codes executable by the processor; and a network unit for obtaining input data regarding properties of a test intended by a user. In this case, the processor may determine a detection condition for monitoring the behavior of test takers based on a user input, and open a test based on the determined detection condition.

The present disclosure can generate and recommend detection conditions required for test supervision according to the properties of a test, and can provide a test construction environment in which user customization of detection conditions is easy.

FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

FIG. 2 is a conceptual diagram illustrating a process for generating a recommended ruleset according to one embodiment of the present disclosure.

FIG. 3 is a block diagram illustrating a test opening process according to one embodiment of the present disclosure.

FIG. 4 is a flowchart showing a test opening process according to one embodiment of the present disclosure.

FIG. 5 is a flowchart illustrating a test generation method according to one embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in the present disclosure are provided so that those skilled in the art can utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments below.

Throughout the specification of the present disclosure, the same or similar drawing reference numerals refer to the same or similar components. In addition, in order to clearly describe the present disclosure, drawing reference numerals of parts that are not related to the description of the present disclosure may be omitted in the drawings.

The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or the context makes clear, "X employs either A or B" should be understood to mean either one of the natural inclusive permutations. For example, unless otherwise specified herein or the context makes clear, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

The term "and/or" as used herein should be understood to refer to and include all possible combinations of one or more of the associated concepts listed.

The terms "comprises" and/or "comprising" as used herein should be understood to mean the presence of particular features and/or components. However, it should be understood that the terms "comprises" and/or "comprising" do not exclude the presence or addition of one or more other features, other components, and/or combinations thereof.

Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is intended to be referred to, the singular should generally be construed to include “one or more.”

The term "Nth (N is a natural number)" used in the present disclosure can be understood as an expression used to mutually distinguish components of the present disclosure according to a predetermined standard such as a functional viewpoint, a structural viewpoint, or convenience of explanation. For example, components performing different functional roles in the present disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of the present disclosure but should be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

The term "acquisition" as used in this disclosure may be understood to refer to generating or receiving data in an on-device form, as well as receiving data over a wireless communication network with an external device or system.

Meanwhile, the term "module" or "unit" used in the present disclosure may be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, a combination of software and hardware, etc. At this time, the "module" or "unit" may be a unit composed of a single element, or may be a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to a computing device itself that constitutes a system, or an application that is executed on a computing device, etc. However, the above-described concept is only an example, and the concept of “module” or “part” may be variously defined within a category understandable to those skilled in the art based on the contents of the present disclosure.

The term "model" used in the present disclosure may be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units to solve a specific problem, or an abstract model regarding a processing process to solve a specific problem. For example, a deep learning "model" may refer to the entire system implemented with a neural network that has a problem-solving ability through learning. In this case, the neural network may have a problem-solving ability by optimizing parameters connecting nodes or neurons through learning. A deep learning "model" may include a single neural network, or may include a set of neural networks in which multiple neural networks are combined.

The term "image" used in the present disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object that can be seen by the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

Meanwhile, the term "rule" used in the present disclosure can be understood as an action corresponding to a detection condition for monitoring a user's action. For example, a rule can correspond to an abnormal action that is not an act of cheating or an act of cheating but may be suspected of being an act of cheating defined by a user or a system. At this time, the detailed items of the rule can be composed of an action such as 'raising an arm' or 'looking away from the monitor' and a numerical value such as '0.5 seconds'. That is, a rule set as a cheating action can be composed of a combination of an action and a numerical value such as 'raising an arm for 0.5 seconds'. In addition, the term "ruleset" can be understood as a set of rules such as the examples described above.

The explanation of the terms set forth above is intended to aid in understanding the present disclosure. Therefore, if the terms set forth above are not explicitly stated as matters limiting the contents of the present disclosure, it should be noted that they are not used to limit the technical ideas of the contents of the present disclosure.

FIG. 1 is a block diagram of a computing device according to one embodiment of the present disclosure.

The computing device (100) according to one embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system that allows a plurality of servers and clients to interact with each other to comprehensively process data. Since the above description is only one example related to the type of the computing device (100), the type of the computing device (100) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

Referring to FIG. 1, a computing device (100) according to one embodiment of the present disclosure may include a processor (110), a memory (120), and a network unit (130). However, FIG. 1 is only an example, and the computing device (100) may include other configurations for implementing a computing environment. In addition, only some of the configurations disclosed above may be included in the computing device (100).

The processor (110) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and/or software for performing computing operations. For example, the processor (110) may read a computer program and perform operations for data processing. The processor (110) may process computational processes such as processing of input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) may generate a user interface for communicating with an external terminal or device. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The type of processor (110) described above is only an example, and thus the type of processor (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

The processor (110) can receive information about the properties of a test that a user wants to open, generate a ruleset used to determine detection conditions for monitoring the behavior of a test taker, and recommend the ruleset to the user. At this time, the detection conditions can be understood as judgment classifications and criteria for monitoring the behavior of a test taker. For example, the detection conditions can be criteria for determining the behavior of a test taker as cheating or criteria for determining the behavior of a test taker as abnormal behavior.

The processor (110) can obtain input data including information about the characteristics of the test, such as what type of test it is and how the test is conducted. In addition, the processor (110) can generate a recommended rule set from a rule database in which rules are stored based on input data about the properties of the test intended by the user. For example, the processor (110) can extract rules from a rule database tagged according to information about the properties of the test based on the input data to generate a recommended rule set in order to determine conditions for detecting cheating or abnormal behavior. The processor (110) can provide a user interface so that the user can use this recommended rule set to set conditions for determining cheating or abnormal behavior. The processor (110) can provide a rule set suitable for the user's intention through a recommendation based on input data about the properties of the test, and can provide an environment in which the user can conveniently set detection conditions.

The processor (110) can update a rule database for generating a rule set by utilizing big data collected according to the opening and operation of a test. In the process of creating and operating a test in a computing environment, various pieces of information that may affect the behavior monitoring of test takers, such as information preset by a user for test opening and information on results detected according to detection conditions during the test operation process, may be accumulated. The processor (110) can analyze data including such information to update a rule database for generating a recommended rule set. For example, if there is a detection condition commonly set by a number of users during the test opening process, the processor (110) can update a rule corresponding to the condition in the rule database. At this time, if a rule to be updated does not exist in the rule database, the processor (110) can add or store the corresponding rule in the rule database. If the detailed items of the rule to be updated in the rule database are different, the processor (110) can change the detailed items of the rule matching in the rule database. The processor (110) can increase the reliability and accuracy of rule set recommendations by dynamically updating the rule database that serves as the basis for recommendations based on big data collected during the test opening and operation process.

The processor (110) can dynamically react to rule set customization according to user input and recommend rules or rule sets to the user. When the user customizes the rule set itself by creating, deleting, or modifying rules included in the rule set according to his/her own subjectivity, the processor (110) can recommend detailed items that are deemed suitable for the rule that is the target of customization. At this time, the processor (110) can analyze big data of tests to which rules corresponding to the rules that are the target of customization are applied to determine how to appropriately set detailed items of the rule customized by the user. For example, when a rule called 'raising the arm more than 10 degrees' is added to the rule set during the process of opening a test, the processor (110) can analyze data of tests to which rules highly correlated with the rule called 'raising the arm more than 10 degrees' are applied. Specifically, the processor (110) can analyze the weights of the rules by analyzing how much the detection condition corresponding to the rule of 'raising the arm more than 10 degrees' was triggered, which condition among the detection conditions corresponding to the rule including the action of 'raising the arm' was triggered, etc. Based on the weights of the rules, the processor (110) can determine which value is more appropriate for the condition for determining that the rule of 'raising the arm more than 10 degrees' is a cheating or abnormal behavior than 10 degrees. If it is determined that the value of 10 degrees is too sensitive and the value of 15 degrees is appropriate, the processor (110) can recommend to the user that the rule be changed to 'raising the arm more than 15 degrees'. The processor (110) can provide the user with objective information based on statistical evidence through recommendations based on user customization. In addition, the processor (110) can increase the reliability and accuracy of behavior monitoring for test takers by guiding users to make rational choices and test settings based on objective information through recommendations based on user customization.

The processor (110) can determine detection conditions for monitoring the behavior of a test taker based on a ruleset set based on user input. Then, the processor (110) can generate a test based on the determined detection conditions. For example, the processor (110) can determine conditions for judging cheating or abnormal behavior based on a ruleset set by the user. When a request is received to set a ruleset including a rule of 'raising the arm more than 15 degrees' as cheating based on user input, the processor (110) can determine the rules included in the ruleset as monitoring targets for detecting cheating. The processor (110) can generate a test reflecting this determination. Then, when the generated test is run, the processor (110) can detect whether an action matching the rules determined as monitoring targets occurs.

The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and/or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by the network unit (130). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a RAM (random access memory), a SRAM (static random access memory), a ROM (read-only memory), an EEPROM (electrically erasable programmable read-only memory), a PROM (programmable read-only memory), a magnetic memory, a magnetic disk, or an optical disk. In addition, the memory (120) may also include a database system that controls and manages data in a predetermined system. The type of memory (120) described above is only an example, and thus the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

The memory (120) can manage, by structuring and organizing, data required for the processor (110) to perform operations, combinations of data, and program codes executable by the processor (110). For example, the memory (120) can store input data acquired through the network unit (130) to be described later. The memory (120) can store data regarding user input and requests received through the user interface. The memory (120) can store program codes that cause the processor (110) to perform operations that generate or recommend detection conditions for behavior monitoring of a test taker based on the input data. In addition, the memory (120) can store program codes that cause the processor (110) to train a deep learning model, program codes that cause the processor (110) to perform inference operations on the trained deep learning model, and various data generated as the program codes are executed.

The network unit (130) according to one embodiment of the present disclosure may be understood as a configuration unit that transmits and receives data via any type of known wired and wireless communication system. For example, the network unit (130) may perform data transmission and reception using a wired and wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultra wide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the network unit (130) may be applied in various ways other than the above-described examples.

The network unit (130) can receive data required for the processor (110) to perform calculations through wired or wireless communication with any system, server, or client. In addition, the network unit (130) can transmit data generated through calculations of the processor (110) through wired or wireless communication with any system, server, or client. For example, the network unit (130) can receive user input and input data for test opening through wired or wireless communication with a client, etc. The network unit (130) can transmit various data generated through calculations of the processor (110) based on the input data through wired or wireless communication with a client, etc.

FIG. 2 is a conceptual diagram illustrating a process for generating a recommended ruleset according to one embodiment of the present disclosure.

Referring to FIG. 2, a computing device (100) according to an embodiment of the present disclosure can obtain input data (10) regarding the properties of a test intended by a user. Here, the properties of the test can be understood as an indicator for understanding what type of test the user intends to open and how it is operated. The computing device (100) can obtain input data (10) used to understand the characteristics of the test the user intends to open by a user input through a user interface. For example, the properties of the test can include at least one of a test type, whether or not an open book is available, whether or not searchable, or a test cycle. The test type can indicate a type of test such as a computer based test (CBT), a written test, or a coding test. The presence or absence of an open book can indicate whether information specified by a test administrator or proctor is exposed to a test taker during the test. The presence or absence of searchable can indicate whether or not a test taker can search for information related to the test during the test. The test cycle can indicate whether or not a test is conducted regularly or irregularly. The test properties listed above are only examples, and various indicators may be included to determine the characteristics of the test in addition to the examples described above.

The computing device (100) can generate a recommended rule set (30) from the rule database (20) based on input data (10) regarding the properties of a test. The computing device (10) can generate a recommended rule set (30) by matching detailed information of the input data (10) regarding the properties of a test with rules classified according to the properties of the test in the rule database (20). For example, if the input data (10) includes detailed information such as the type of test, whether it is open book, and whether it is searchable, the computing device (100) can check the rule matching each detailed information of the input data (10) in the rule database (20) and extract the rule. Specifically, the computing device (100) can analyze rules included in a group (A) related to the test type in the rule database (20) based on information about the test type of the input data (10), and include rules matching the information about the test type of the input data (10) in the recommended rule set (30). The computing device (100) can analyze rules included in a group (B) related to the presence or absence of an open book in the rule database (20) based on information about the presence or absence of an open book in the input data (10), and include rules matching the information about the presence or absence of an open book in the input data (10) in the recommended rule set (30). The computing device (100) can analyze rules included in a group (C) related to the presence or absence of a search in the rule database (20) based on information about the searchability of the input data (10), and include rules matching the information about the presence or absence of a searchability of the input data (10) in the recommended rule set (30). At this time, the analysis of the rules included in the group in the rule database (20) may be performed according to the logic defined by the program code, or may be performed through a pre-learned neural network model. In this way, the computing device (100) can extract rules matching the detailed information of the input data (10) from the rule database (20) in which the rules are classified and constructed according to the properties of the test, and generate a rule set (30) including the extracted rules.

Meanwhile, the computing device (100) can update the rule database (20) based on the results of analyzing the big data (40) accumulated in accordance with the opening and operation of the test. The computing device (100) can continuously analyze the big data (40) collected in the process of opening and operating the test to dynamically update the rule database (20). The big data (40) used to update the rule database (20) can include at least one of data on detection conditions adjusted by customizing the rule set in the process of opening the test by the user or data on events detected based on the rules in the process of operating the opened test. In addition, dynamic updating can be understood as creating a rule to be newly included in the rule database (20) or changing the detailed items of a rule already included in the rule database (20). For example, if the computing device (10) analyzes big data (40) and determines that a rule such as 'a body part is not visible in a designated area for more than 0.5 seconds' is frequently set by users' rule set customization, the computing device (100) can check whether the rule identified through the analysis of big data (40) is included in the rule database (20). If the rule identified through the analysis of big data (40) is not included in the rule database (20), the computing device (100) can classify the identified rule into Group A, Group B, or Group C according to the properties of the test and store it in the rule database (20). At this time, the classification may be performed according to logic defined by the program code or may be performed through a pre-learned neural network model. If a rule identified through analysis of big data (40) is included in the rule database (20) in a similar form, such as 'a body part is not visible in a designated area for more than 1 second', the computing device (100) can identify and change information that is inconsistent in the details between the rule already stored in the rule database (20) and the rule identified through analysis of big data (40). Specifically, the computing device (100) can change the rule stored in the rule database (20) from 'a body part is not visible in a designated area for more than 1 second' to 'a body part is not visible in a designated area for more than 0.5 seconds'. In this way, the computing device (100) can improve the reliability and accuracy of customized recommendations by utilizing big data (40) secured through trial opening and operation for dynamic updates of the rule database (20).

FIG. 3 is a block diagram illustrating a test opening process according to one embodiment of the present disclosure.

Referring to FIG. 3, a computing device (100) according to an embodiment of the present disclosure may receive test information in order to open a test desired by a user (S110). At this time, the test information may be information on the properties of a test required to identify the characteristics of a test desired by the user. At this time, the computing device (100) may provide a user interface in order to receive the test information. For example, the computing device (100) may generate a user interface to obtain test information input through a user terminal. The user interface for inputting the test information may be implemented on a display. That is, the computing device (100) may receive information on the properties of a test, such as a test type, whether or not an open book is available, whether or not a search is possible, or a test period, through a user interface implemented on a display of a user terminal.

The computing device (100) can provide a user interface for determining detection conditions used to monitor the behavior of a test taker. Then, the computing device (100) can obtain user inputs necessary to determine the detection conditions through the user interface and set the detection conditions (S120).

For example, the computing device (100) may provide a first user interface (10) that exposes related information so that a recommended rule set based on test information or a preset preset set in advance by the user can be selected. The user may perform an action of selecting one of the recommended rule sets or presets in an area (11) of the first user interface (10) implemented on the display of the user terminal. When one of the recommended rule sets or presets is finally selected according to the user's action, the computing device (100) may determine the finally selected rule set as a detection condition for determining misconduct or abnormal behavior. Meanwhile, in the process of selecting the recommended rule set or preset according to the user's action, the computing device (100) may provide at least one of an area (12) that exposes detailed information of the rule set or an area (13) that exposes graphics for controlling the level of the rule set through the first user interface (10). The user can check which rules are included in the rule set through the area (13) that exposes the details of the rule set of the first user interface (10). The user can adjust the intensity of the detection condition in the area (13) that exposes the graphic for controlling the level of the rule set of the first user interface (10). At this time, the level of the rule set can be distinguished according to the numerical value of the detailed items of the rule that constitutes the rule set. Assuming that there is a rule that 'a body part is not visible in the designated area for more than 1 second', if the user lowers the level of the rule set, the numerical value of 1 second can be increased so that the detection sensitivity based on the rule is lowered. Conversely, if the user raises the level of the rule set, the numerical value of 1 second can be decreased so that the detection sensitivity based on the rule is higher. Through such level control of the rule set, the user can easily and freely adjust the intensity of the detection condition under the designated rule set. The first user interface (10) expressed in FIG. 3 is only an example, and thus the first user interface (10) may be implemented in various forms within a range understandable to those skilled in the art based on the present disclosure.

The computing device (100) may provide a second user interface (20) that provides a user customization environment so that the user can arbitrarily set or modify the ruleset. The user may perform an operation to configure a ruleset according to the user's own thoughts on the second user interface (20) implemented on the display of the user terminal. If the creation or adjustment of rules for configuring the ruleset is performed according to the user's operation, the computing device (100) may determine the customized ruleset as a detection condition for judging fraudulent or abnormal behavior. Meanwhile, although not expressed in FIG. 3, in the process of performing user customization, the computing device (100) may provide an area for recommending detailed items of rules set or adjusted by the user through the second user interface (20). If a rule is set or adjusted by the user, the computing device (100) may analyze big data of a test to which rules corresponding to the set or adjusted rules are applied and recommend detailed items that are determined to be suitable for the set or adjusted rules. At this time, recommending detailed items that are deemed appropriate can be understood as deriving an optimal value by comparing detailed items of rules through statistical analysis or neural network analysis based on big data. The user can select a recommended value exposed through the recommendation area of the second user interface (20), or can set or adjust an arbitrary value without selecting it. That is, the computing device (100) can assist the user's decision-making for setting detection conditions for test opening through a recommendation through the second user interface (20). Since the second user interface (20) expressed in FIG. 3 is only one example, the second user interface (20) can be implemented in various forms within a category understandable to those skilled in the art based on the present disclosure.

When the detection conditions are set through step S120, the computing device (100) can generate test questions based on information about test questions input by the user (S130). Then, the computing device (100) can open a test based on the set detection conditions and the generated test questions (S140). For example, when the user inputs a test format, questions, and answers, the computing device (100) can organize the questions and answers according to the test format and generate test questions. Then, the computing device (100) can generate an access link such as a URL, open a test, and provide it to the user or test taker through a user terminal.

Meanwhile, the computing device (100) can continuously collect the user input data obtained through step S120. Then, the computing device (100) can analyze the collected user input data and manage it as big data. The computing device (100) can utilize such big data for updating a rule database for generating a recommended rule set, generating recommended rules in a user customization process, etc.

Fig. 4 is a flowchart showing a test opening process according to one embodiment of the present disclosure. Below, the description of the remaining steps except for step S260 in Fig. 4 will be replaced with the contents of Figs. 2 and 3.

Referring to FIG. 4, a computing device (100) according to an embodiment of the present disclosure may analyze and recommend detailed items of a rule in a process in which rule set customization is performed by user input (S260). When an input for setting or adjusting a rule is received (S240), the computing device (100) may analyze the weights of the rules based on big data of a test to which rules corresponding to the set or adjusted rule are applied. Then, the computing device (100) may select detailed items of the rule according to the analyzed weights, and determine the detailed items of the selected rule as detailed items suitable for the set or adjusted rule. At this time, the big data affecting the analysis of the weights may include at least one of the rate at which a result detected according to a specific rule is delivered to the test taker, or the number of times a specific rule is activated by the test taker's behavior satisfying the detailed items of a specific rule.

Specifically, when a user customization input is received, the computing device (100) can analyze big data of a test to which rules corresponding to the user customization input are applied. At this time, the rules corresponding to the user customization input may be rules having a high correlation with the rule that is the target of user customization. For example, if the rule of 'raising the arm more than 10 degrees' is the target of customization, the computing device (100) can identify a rule that includes a body related to the arm, such as a hand, and an action related to the movement of the body as a detailed item as a rule having a high correlation, and can analyze big data of a test to which the rule is applied.

The computing device (100) can analyze big data to calculate the weights of rules. For example, a test supervisor can request an explanation from a test taker in order to make an accurate judgment on the detection results. The higher the ratio of such explanation requests, the higher the probability that the detection results will be judged as cheating or abnormal behavior. Accordingly, the computing device (100) can assign a higher weight to a rule as the ratio of the detection results transmitted to the test taker according to the rule is higher. If the ratio of the detected results transmitted to the test taker according to the rule of 'raising the arm more than 10 degrees' is 20%, and the ratio of the detected results transmitted to the test taker according to the rule of 'raising the arm more than 20 degrees' is 30%, the computing device (100) can assign a higher weight to the rule of 20 degrees or more than the rule of 10 degrees or more. In addition, the more times a rule is activated by satisfying a detection condition according to the rule, the higher the probability that the detection result according to the rule corresponds to cheating or abnormal behavior. However, if it is activated too many times, the probability that the rule corresponds to the general behavior of test takers rather than cheating or abnormal behavior is high, so the computing device (100) can assign a higher weight as the number of times the rule is activated is greater than or equal to a predetermined threshold. In addition, the computing device (100) can assign a gradually lower weight as the number of times the rule is activated exceeds a predetermined threshold.

The computing device (100) can determine the detailed items of the rule to be recommended to the user based on the analyzed weights. The computing device (100) can determine the detailed items of the rule with the highest weight as the detailed items of the rule to be recommended to the user, or can determine the detailed items of the rules with weights higher than a reference value as the detailed items of the rule to be recommended to the user by combining the detailed items of the rules. In addition, the computing device (100) can provide the detailed items determined as the recommendation target to the user through the user terminal. At this time, the detailed items of the rule to be recommended to the user can be distinguished by an action based on a specific body part of a person and a numerical value indicating the degree of the action. For example, if the rule of 'raising the arm more than 20 degrees' is analyzed to have the highest weight, the computing device (100) can provide the user with recommendation information suggesting that the rule customized as 'raising the arm more than 10 degrees' be modified from 10 degrees to 20 degrees through the user terminal. Or, if the rule of 'raising the hand more than 10 degrees' is analyzed to have the highest weight, the computing device (100) can provide the user with recommendation information suggesting to change the arm to a hand in the rule customized to 'raising the arm more than 10 degrees' through the user terminal.

FIG. 5 is a flowchart illustrating a test generation method according to one embodiment of the present disclosure.

Referring to FIG. 5, a computing device (100) according to an embodiment of the present disclosure can obtain input data regarding the properties of a test intended by a user (S310). For example, if the computing device (100) is a cloud server, the computing device (100) can input information regarding the properties of a test and receive input data through a user interface implemented through an input/output unit of a user terminal. If the computing device (100) is a client directly equipped with an input/output unit, the computing device (100) can input information regarding the properties of a test through a user interface implemented through the input/output unit.

The computing device (100) can determine detection conditions for monitoring the behavior of test takers based on user input (S320). The computing device (100) can determine detection conditions based on user input through a first user interface where a recommended rule set is exposed. For example, the computing device (100) can generate a recommended rule set from a rule database based on input data regarding the properties of a test. Then, if one of the recommended rule sets or presets exposed through the first user interface is selected according to the user input, the computing device (100) can determine detection conditions based on the selected rule set. In addition, the computing device (100) can determine detection conditions based on user input through a second user interface for customizing the rule set. For example, if at least one rule included in the rule set is set or adjusted according to the user input through the second user interface, the computing device (100) can determine detection conditions based on the rule set including the set or adjusted rule. Specifically, when setting or adjusting at least one rule is performed, the computing device (100) can analyze detailed items judged to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied. Then, the computing device (100) can recommend detailed items judged to be suitable for the set or adjusted rule based on the analysis result.

The computing device (100) can open a test based on the determined detection conditions (S330). The computing device (100) can receive additional information, such as test questions, to open a test, and can open a test by reflecting the received additional information. At this time, the opened test can be provided to the test taker through a platform implemented in the computing environment.

Meanwhile, according to one embodiment of the present disclosure, if the computing device (100) includes an input/output unit or is a client, it may display a first user interface area in which a recommended ruleset generated from a rule database is exposed based on input data regarding the properties of a test intended by a user. In addition, the computing device (100) may display a second user interface area for receiving a user input for customizing the ruleset. The user may perform operations such as selecting, modifying, or changing on the first user interface area or the second user interface area. The computing device (100) may process data based on the input according to the corresponding operation or transmit a data processing request through another server.

The various embodiments of the present disclosure described above can be combined with additional embodiments and can be modified within a range that can be understood by those skilled in the art in light of the detailed description set forth above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concept of the claims of the present disclosure should be interpreted as being included in the scope of the present disclosure.

Claims (16)

A test generation method, performed by a computing device including at least one processor, A step of obtaining input data regarding the properties of a test intended by the user; A step of determining detection conditions for monitoring the behavior of test takers based on user input; and A step of opening a test based on the above determined detection conditions; Including, method. In paragraph 1, The properties of the above test are: Including at least one of the following: exam type, open book, searchability, or exam frequency; method. In paragraph 1, The step of determining detection conditions for monitoring the behavior of test takers based on the above user input is: A step of determining detection conditions based on user input through a first user interface where a recommended ruleset is exposed; Including, method. In the third paragraph, The step of determining detection conditions based on user input through the first user interface where the above recommended ruleset is exposed is: A step of generating a recommended rule set from a rule database based on input data regarding the properties of the above test; and A step of determining a detection condition based on the selected rule set when one of the recommended rule sets or presets exposed through the first user interface is selected according to user input; Including, method. In paragraph 4, The above rule database is, It is updated based on the results of analyzing big data accumulated through the opening and operation of the test. method. In paragraph 5, The above big data is, In the process of opening a test, it includes at least one of data on detection conditions adjusted by the user's rule set customizing or data on events detected based on rules in the process of running the opened test. method. In paragraph 5, The above update is, Generating a rule to be included in the above rule database, or dynamically changing the details of a rule composing a rule set included in the above rule database. method. In the third paragraph, When one of the recommended rulesets or presets exposed through the first user interface is selected according to the user input, the step of determining the detection condition based on the selected ruleset is as follows: A step of adjusting the intensity of a detection condition according to a user input for controlling the level of the above-mentioned selected ruleset; Including, The level of the above ruleset is, Distinguished by the numerical value of the detailed items of the rules that constitute the above ruleset, method. In paragraph 1, The step of determining detection conditions for monitoring the behavior of test takers based on the above user input is: A step of determining detection conditions based on user input via a second user interface for customizing the ruleset; Including, method. In Article 9, The step of determining detection conditions based on user input via a second user interface for customizing the above ruleset is: A step of determining a detection condition based on a ruleset including the set or adjusted rule, when setting or adjusting at least one rule included in the ruleset is performed according to user input through the second user interface; Including, method. In Article 10, When at least one rule included in the ruleset is set or adjusted according to user input through the second user interface, the step of determining a detection condition based on the ruleset including the set or adjusted rule is as follows: When setting or adjusting at least one rule is performed, a step of analyzing detailed items judged to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied; and Based on the results of the above analysis, a step of recommending detailed items deemed suitable in the above set or adjusted rules; Including, method. In Article 11, When setting or adjusting at least one rule above is performed, a step of analyzing detailed items judged to be suitable for the set or adjusted rule based on big data of a test to which rules corresponding to the set or adjusted rule are applied is performed. A step of analyzing the weights of the rules based on big data of a test to which rules corresponding to the above-mentioned set or adjusted rules are applied; and A step of determining the detailed items of the rule selected based on the analyzed weights as detailed items suitable for the set or adjusted rule; Including, method. In Article 12, Big data that influences the analysis of the above weights are: At least one of the rates at which results detected according to a particular rule are communicated to the test taker, or the number of times a particular rule is activated by a test taker's action satisfying a subitem of the particular rule. method. A test generation method, performed by a computing device including at least one processor, A step of displaying a first user interface area in which a recommended ruleset generated from a rule database is exposed based on input data regarding the properties of a test intended by the user; and A step of displaying a second user interface area for receiving user input for customizing the ruleset; Including, method. A computer program stored on a computer-readable storage medium, wherein the computer program, when executed on one or more processors, causes the computer program to perform operations for generating a test, The above actions are, An action to obtain input data about the properties of a test intended by the user; An action to determine detection conditions for monitoring the behavior of test takers based on user input; and An action to open a test based on the detection conditions determined above; Including, Computer program. As a computing device for generating tests, A processor comprising at least one core; a memory containing program codes executable by the processor; and A network unit for obtaining input data regarding the properties of a test intended by the user; Including, The above processor, Determine detection conditions to monitor the behavior of test takers based on user input, Opening a test based on the detection conditions determined above, device.

Images