WO2024187925A1 - Question answering system model training method, and sample generation method - Google Patents

Abstract

The embodiments of the present disclosure provide a question answering system model training method, a sample generation method and apparatus, an electronic device, and a storage medium, wherein the question answering system model training method comprises: obtaining a question text sample; inputting the question text sample into an initial question analysis model, performing iterative training, and obtaining a question analysis model; the initial question analysis model comprising a first encoding layer and a conversion layer; the first encoding layer being used for performing encoding in accordance with the question text sample, so as to obtain a corresponding first sentence vector; the conversion layer being used for generating a preset number of initial intention vectors when the first sentence vector is received, filling each of the initial intention vectors in accordance with the first sentence vector, obtaining a corresponding target intention vector, and converting the target intention vector into a corresponding text segment; and the text segment being used for querying the question answering system for an answer to the question text sample.

Description

Model training method and sample generation method for question answering system

Cross-references

This application claims priority to the Chinese patent application filed with the Chinese Patent Office on March 14, 2023, with application number 202310247102.2, and entitled “Sample generation method, device, electronic device and storage medium for question and answer system”, and the Chinese patent application filed with the Chinese Patent Office on March 14, 2023, with application number 202310247092.2, and entitled “Model training method, device, electronic device and storage medium for question and answer system”, the entire contents of which are incorporated by reference into this application.

Technical Field

The present application relates to the field of knowledge graph technology, and in particular to a model training method, sample generation method, device, electronic device and storage medium for a question-answering system.

Background Art

With the development of Internet technology, the application of question-answering systems has become more and more popular. Automatically answering questions raised by users through question-answering systems can improve answering efficiency and save human resources. However, in actual applications, the questions raised by users may be colloquial, making the sentence structure rich and diverse, which increases the difficulty of intent recognition in question-answering systems. In addition, in actual applications, question-answering systems involving knowledge graphs are often highly customized, and the amount of sample data required for model training of such question-answering systems is very large. If training samples are manually configured, the efficiency is low and it is difficult to meet the model training requirements.

Summary of the invention

The present application provides a model training method, sample generation method, device, electronic device and storage medium for a question-answering system to improve the intent recognition capability and sample generation efficiency of the question-answering system.

On the one hand, the present application provides a model training method for a question-answering system, wherein the question-answering system includes an initial question parsing model; the method includes: obtaining a question text sample; The sample is input into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each of the initial intention vectors according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system.

On the one hand, the present application provides an answering method, including: obtaining a target question to be answered; inputting the target question into a question parsing model for parsing and processing to obtain a corresponding target fragment; the question parsing model is obtained by training with the model training method of the question-answering system as described above; and determining the answer to the target question based on the target fragment.

On the one hand, the present application provides a model training device for a question-answering system, the question-answering system including an initial question parsing model; the device including: a first acquisition unit, used to acquire a question text sample; a first training unit, used to input the question text sample into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each of the initial intention vectors according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system.

On the one hand, the present application provides an answering device, including: a second acquisition unit, used to acquire a target question to be answered; a first parsing unit, used to input the target question into a question parsing model for parsing and processing to obtain a corresponding target fragment; the question parsing model is obtained by training with the model training method of the above-mentioned question-answering system; a first determination unit, used to determine the answer to the target question based on the target fragment.

On the one hand, the present application provides a sample generation method for a question-answering system, the method comprising: obtaining a pre-configured synonym set, a similar question set, a comparison word set, a problem domain set, and a standard entity library; the synonym set comprises a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set comprises a plurality of first attributes and at least one similar question sentence corresponding to each of the first attributes; the comparison word set comprises comparison word information; the problem domain set comprises The method comprises the following steps: the method comprises: comprising: comprising: comprising: a plurality of standard entities; generating a question parsing sample according to the synonym set, the similar question set and at least one of the comparative word set; generating a question classification sample according to the question parsing sample and the question domain set; and generating the entity linking sample according to the question parsing sample and the standard entity library; and constructing a training data set for the question answering system according to the question classification sample, the question parsing sample and the entity linking sample.

On the one hand, the present application provides a model training method for a question-answering system, comprising: generating a training data set by a sample generation method for the question-answering system as described above; the training data set comprises the question classification samples, the question parsing samples and the entity linking samples; inputting the question classification samples into an initial question classification model in the question-answering system for iterative training to obtain a question classification model; inputting the question parsing samples into an initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; inputting the entity linking samples into an initial entity linking model in the question-answering system for iterative training to obtain an entity linking model.

On the one hand, the present application provides an answering method, including: obtaining a target question to be answered; inputting the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by the sample generation method of the above-mentioned question-answering system; when the classification result is used to characterize that the target question belongs to a first preset classification, inputting the target question into a question parsing model for parsing processing to obtain a corresponding target fragment; the question parsing model is obtained by inputting question parsing samples in the training data set into an initial question parsing model for training; inputting the target fragment into an entity linking model for prediction processing to obtain a corresponding target entity; the entity linking model is obtained by inputting entity linking samples in the training data set into an initial entity linking model for training; and determining the answer to the target question based on the target entity.

On the one hand, the present application provides a sample generation device for a question-answering system, the device comprising: a third acquisition unit, used to acquire a pre-configured synonym set, a similar question set, a comparative word set, a problem domain set and a standard entity library; the synonym set comprises a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set comprises a plurality of first attributes and at least one similar question sentence corresponding to each of the first attributes; the comparative word set comprises comparative word information; the problem domain set comprises in-domain question text and out-of-domain question text; the standard entity library comprises a plurality of standard entities; a first generation unit, used to generate a sample according to the synonym set, the similar question set and the comparative word set. A question parsing sample is generated by comparing at least one of the word sets; a second generating unit is used to generate a question classification sample according to the question parsing sample and the problem domain set; and, based on the question parsing sample and the standard entity library, the entity linking sample is generated; a constructing unit is used to construct a training data set for the question answering system according to the question classification sample, the question parsing sample and the entity linking sample.

On the one hand, the present application provides a model training device for a question-answering system, comprising: a third generation unit, used to generate a training data set through a sample generation method for the question-answering system as described above; the training data set includes the question classification samples, the question parsing samples and the entity linking samples; a second training unit, used to input the question classification samples into an initial question classification model in the question-answering system for iterative training to obtain a question classification model; input the question parsing samples into the initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; input the entity linking samples into the initial entity linking model in the question-answering system for iterative training to obtain an entity linking model.

On the one hand, the present application provides an answering device, including: a fourth acquisition unit, used to acquire a target question to be answered; a classification unit, used to input the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by a sample generation method such as the above-mentioned question-answering system; a second parsing unit, used to input the target question into a question parsing model for parsing processing to obtain a corresponding target fragment when the classification result is used to characterize that the target question belongs to a first preset classification; the question parsing model is obtained by inputting question parsing samples in the training data set into an initial question parsing model for training; a prediction unit, used to input the target fragment into an entity linking model for prediction processing to obtain a corresponding target entity; the entity linking model is obtained by inputting entity linking samples in the training data set into an initial entity linking model for training; a second determination unit, used to determine the answer to the target question based on the target entity.

On the one hand, the present application provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to execute a model training method for the question-answering system as described above, or, a response method as described above, or, a sample generation method for the question-answering system as described above.

On the one hand, the present application provides a computer-readable storage medium for storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the model of the question-answering system as described above. Type training method, or, the answering method as mentioned above, or, the sample generation method of the question-answering system as mentioned above.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG1 is a processing flow chart of a model training method for a question-answering system provided in an embodiment of the present application;

FIG2 is a data flow diagram of a question parsing model in a model training method for a question-answering system provided in an embodiment of the present application;

FIG3 is a data flow diagram of an entity linking model in a model training method for a question-answering system provided in an embodiment of the present application;

FIG4 is a processing flow chart of a response method provided in an embodiment of the present application;

FIG5 is a flow chart of session management in a response method provided in an embodiment of the present application;

FIG6 is a schematic diagram of a model training device for a question-answering system provided in an embodiment of the present application;

FIG7 is a schematic diagram of a response device provided in an embodiment of the present application;

FIG8 is a processing flow chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG9 is a schematic diagram of a framework of a question-answering system provided in an embodiment of the present application;

FIG10 is a first processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG11 is a second processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG12 is a third processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG13 is a fourth processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG14 is a fifth processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG15 is a sixth processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG16 is a seventh processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG17 is an eighth processing flow sub-chart of a sample generation method of a question-answering system provided in an embodiment of the present application;

FIG18 is a processing flow chart of another model training method for a question-answering system provided in an embodiment of the present application;

FIG19 is a processing flow chart of another response method provided in an embodiment of the present application;

FIG20 is a working principle diagram of a question-answering system provided in an embodiment of the present application;

FIG21 is a schematic diagram of a sample generation device of a question-answering system provided in an embodiment of the present application;

FIG22 is a schematic diagram of another model training device for a question-answering system provided in an embodiment of the present application;

FIG23 is a schematic diagram of another response device provided in an embodiment of the present application;

FIG. 24 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

In the actual application of the question-answering system, on the one hand, the questions asked by users may be colloquial or one-sided, resulting in a rich and diverse sentence structure of the questions, which increases the difficulty of the question-answering system in identifying the intention; on the other hand, the information that users may provide when asking questions may be one-sided. If you only answer questions based on insufficient information, you may not be able to find a satisfactory answer to the user. In order to solve the above problems, the embodiment of the present application provides a model training method for a question-answering system.

FIG1 is a processing flow chart of a model training method for a question-answering system provided in an embodiment of the present application. The model training method for the question-answering system in FIG1 can be executed by an electronic device, which can be a terminal device, such as a mobile phone, a laptop computer, an intelligent interactive device, etc.; or, the electronic device can also be a server, such as an independent physical server, a server cluster, or a server capable of cloud computing. Cloud server. Referring to FIG1 , the model training method of the question answering system provided in this embodiment specifically includes steps S102 to S104.

The question answering system may include an initial question parsing model, an initial question classification model, and an initial entity linking model.

The initial problem classification model can be an untrained OOD (out of the design scope) model.

In specific implementation, the question domain can be pre-configured. Questions within the question domain are called in-domain questions, and questions outside the question domain are called out-of-domain questions. The question-answering system has the ability to answer in-domain questions, but does not have the ability to answer out-of-domain questions. The OOD model can be used to classify the received questions, and the classification results include at least: in-domain questions and out-of-domain questions.

The initial question parsing model may be an untrained relation extraction model. The relation extraction model may be used to perform slot recognition processing to obtain slot recognition results, which include but are not limited to: entities, attributes, relations, constraints, and the like.

Entities can be names of people, places, organizations, pre-set proper nouns, etc. Concepts, also known as classes, are abstract descriptions of a collection of objects with the same characteristics. Concepts can be used to reflect categories. A concept can correspond to multiple entities.

For example, the concept "plant" can correspond to the following entities: "willow", "cactus", "cherry blossom", and so on.

Attributes can be used to reflect the characteristics of an entity. A concept can correspond to multiple attributes.

For example, the concept "plant" can correspond to the following attributes: "name", "type", "shape", "growing environment", "distribution range" and "propagation method", etc.

For each attribute, you can pre-configure the attribute name, attribute type, and attribute description.

Among them, attribute types include but are not limited to: text, number, picture, rich text, JS object notation (JavaScript Object Notation, json).

Relationships are used to describe the connection between concepts, which can be divided into classification relationships and non-classification relationships. In practical applications, corresponding relationships can be customized according to specific fields and specific applications, such as "cause and effect".

A constraint can be a restriction condition. For example, in the question “What are the services with an annual interest rate less than x%?”, the slot recognition result of “annual interest rate less than x%” is a constraint.

The initial entity linking model can be an untrained Bert (Bidirectional Encoder Representation from Transformers) classification model. The Bert classification model can be used to perform classification prediction processing, linking the entity fragment of the input model to an entity node in a pre-configured knowledge graph.

The knowledge graph includes multiple entity nodes, each of which corresponds to an entity. The knowledge graph can be constructed by extracting knowledge through ontology and corpus to form triples, and turning business data into knowledge.

Step S102, obtaining a question text sample.

The question text sample may be a pre-generated question parsing sample. Each question text sample may be a word, a word combination, an incomplete sentence or a complete sentence, and the like.

Question analysis samples can be generated in the following ways:

Obtain a preconfigured synonym set, similar question set, and comparative word set; the synonym set includes multiple standard words and at least one synonym corresponding to each standard word; the similar question set includes multiple first attributes and at least one similar question sentence corresponding to each first attribute; the comparative word set includes comparative word information; generate a question parsing sample based on at least one of the synonym set, similar question set, and comparative word set.

Among them, the specific implementation method of generating question analysis samples in this embodiment involves the sample generation method of the question and answer system, and can refer to the specific scheme in the following method embodiment.

Step S104, input the question text sample into the initial question parsing model for iterative training to obtain the question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each initial intention vector according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system.

It should be noted that the structure of the initial question parsing model is completely consistent with that of the question parsing model obtained after iterative training, but the model parameters involved in the training are different. The question parsing model obtained after iterative training also has a first encoding layer and a conversion layer.

The first encoding layer can be a Bert encoder or other components that can be used to convert text into sentence vectors. The conversion layer can be composed of a spatial conversion layer and multiple binary classification fully connected layers.

The output of the first coding layer can be the input of the transformation layer.

The initial intention vector can be a vector with a fixed number of elements but each element is unknown. Each initial intention vector can be used to represent a corresponding intention space. After the filling process, some elements are known, and the remaining unknown elements can be filled by specifying numerical values to obtain the target intention vector corresponding to the initial intention vector.

When the first sentence vector is received, a preset number of initial intention vectors are generated, and the preset number of initial intention vectors include the same number of elements.

The number of elements included in each initial intention vector may be a pre-configured fixed value. Specifically, when the first sentence vector is received, a preset number of initial intention vectors are generated; and the number of elements included in the initial intention vector is a preset value.

The number of elements included in each initial intention vector can also be determined based on the number of characters corresponding to the first sentence vector. Specifically, when the first sentence vector is received, a preset number of initial intention vectors are generated according to the first sentence vector; the number of elements included in the initial intention vector is determined based on the number of characters corresponding to the first sentence vector.

In a specific implementation method, the first sentence vector includes a semantic feature sub-vector and multiple character sub-vectors; each initial intention vector is filled according to the first sentence vector to obtain the corresponding target intention vector. The specific implementation method is: intent classification is performed on the semantic feature sub-vector to obtain the corresponding intention classification result; according to the intention classification result, each initial intention vector is filled to obtain the corresponding intermediate intention vector; according to each character sub-vector, each intermediate intention vector is filled to obtain the corresponding target intention vector.

For example, the semantic feature sub-vector can be represented by [cls]. [cls] is not a vector used to represent a certain character in the text, but a semantic feature vector used to represent the entire text, which can be directly used for classification.

The question text sample may include multiple characters. After the question text sample is encoded by the first encoding layer to obtain the corresponding first sentence vector, the first sentence vector may include a character sub-vector corresponding to each character in the question text sample.

The intent classification result can be used to indicate, for each initial intent vector, whether the first sentence vector has the intent corresponding to the initial intent vector. For example, for initial intent vector 1, if the intent classification result is the first classification result, it means that the first sentence vector does not have the intent corresponding to the initial intent vector 1; if the intent classification result is the second classification result, it means that the first sentence vector has the intent corresponding to the initial intent vector 1.

The semantic feature sub-vector is processed for intent classification to obtain the corresponding intent classification result. In specific implementation, it can be processed based on [cls] through a binary classification fully connected layer, that is, a linear layer (linear), to obtain the intent classification result. That is, the input value of linear is [cls], and the output value includes multiple intents. Figure classification results, each intent classification result corresponds to an intent initial vector, and each intent classification result can be one of the first classification result and the second classification result.

According to the intent classification result, each initial intent vector is filled to obtain the corresponding intermediate intent vector, which can be automatically filled with the value used to represent the intent classification result of the initial intent vector in each initial intent vector to obtain the intermediate intent vector. For example, for initial intent vector 1, if the intent classification result is the first classification result, the value "0" used to represent the first classification result is filled in the specified position of the initial intent vector 1; if the intent classification result is the second classification result, the value "1" used to represent the second classification result is filled in the specified position of the initial intent vector 1.

Exemplarily, filling the specified position of the initial intention vector 1 with the value "0" for indicating the first classification result may be performed by replacing the unknown element at the specified position with "0".

It should be noted that if the intent classification result corresponding to the initial intention vector is the first classification result, it means that the first sentence vector does not have the corresponding intention. Therefore, the character recognition results obtained by entity recognition and constraint recognition in subsequent steps are irrelevant to the initial intention vector. Therefore, after filling "0" in the initial intention vector to obtain the intermediate intention vector, the intermediate intention vector can be determined as the corresponding target intention vector.

In a specific implementation method, each intermediate intention vector is filled according to each character sub-vector to obtain the corresponding target intention vector. The specific implementation method is: according to each character sub-vector, entity recognition processing and constraint recognition processing are performed to obtain the corresponding character recognition result; according to the character recognition result, each intermediate intention vector is filled to obtain the corresponding target intention vector.

In specific implementation, a binary fully connected layer linear can be used to perform entity recognition and constraint recognition processing according to each character sub-vector to obtain a corresponding character recognition result, which can be used to represent the entity type and constraint type corresponding to the character sub-vector. Among them, the entity type includes non-entity and multiple preset entity types, and the constraint type includes non-constraint and multiple preset constraint types.

By generating character recognition results, it is possible to determine whether each character sub-vector belongs to an entity or a constraint, thereby achieving entity extraction or constraint extraction.

In a specific implementation method, each intermediate intention vector is filled according to the character recognition result to obtain the corresponding target intention vector. The specific implementation method is: if the character recognition result is used to characterize that the character sub-vector belongs to an entity category or a constraint category, then according to the character recognition result, the corresponding intermediate intention vector is determined and filled.

If the character recognition result is used to represent that the character sub-vector belongs to an entity category, then The entity extraction process is performed on the identification result, and the intent prediction process is performed on the extracted entity to determine the intermediate intent vector corresponding to the entity, and the entity is automatically filled into the corresponding intermediate intent vector.

The filling position of the entity in the intermediate intent vector can be determined based on the character position corresponding to the entity in the question text sample.

If the character recognition result is used to characterize that the character sub-vector belongs to the constraint category, constraint extraction processing can be performed based on the character recognition result, and intention prediction processing can be performed on the extracted constraints to determine the intermediate intention vector corresponding to the constraint, and the constraint can be automatically filled into the corresponding intermediate intention vector.

The filling position of the constraint in the intermediate intention vector can be determined based on the character position corresponding to the constraint in the question text sample.

In a specific implementation, the question text sample includes at least one of an entity element, an attribute element, a relationship element, and a constraint element; a preset number of initial intention vectors include a first intention vector and multiple second intention vectors; the first intention vector corresponds to no intent, and each second intention vector corresponds to an attribute element or a relationship element.

The question text sample includes at least one of an entity element, an attribute element, a relationship element, and a constraint element. The slot recognition result of the entity element may be "entity", the slot recognition result of the attribute element may be "attribute", the slot recognition result of the relationship element may be "relationship", the slot recognition result of the constraint element may be "constraint", and so on.

The following are several examples to illustrate sample question texts.

The question parsing model can convert question text samples into different intent spaces, each of which corresponds to an initial intent vector.

If the question text sample includes an attribute element, the attribute element can be used to determine a target initial intention vector corresponding to the attribute element from a preset number of initial intention vectors.

If the question text sample includes a relation element, the relation element can be used to determine a target initial intention vector corresponding to the relation element from a preset number of initial intention vectors.

Filling each initial intention vector according to the first sentence vector may be performed by labeling entities and constraints under the initial intention vector corresponding to each intention space. Labeling entities and constraints in the target initial intention vector corresponding to the attribute element may enable the generated target intention vector to reflect at least one of the corresponding relationship between the attribute and the entity and the corresponding relationship between the attribute and the constraint. Labeling entities and constraints in the target initial intention vector corresponding to the relationship element may enable the generated target intention vector to reflect at least one of the corresponding relationship between the relationship and the entity and the corresponding relationship between the relationship and the constraint.

In the process of slot identification, in the question text sample, various slot elements such as entity elements, attribute elements, relationship elements, constraint elements, etc. may have certain corresponding relationships with each other.

For example, a sample question text is: What is the price of product A, and when will product B be available? Among them, "product A" and "product B" are entities, "price" and "availability time" are attributes, and "product A" corresponds to "price" and "product B" corresponds to "availability time". If slot recognition is performed for each element separately, the user's intention corresponding to the sample question text may be misunderstood, and the user may be fed back the prices of both products A and B, as well as the availability time of both products A and B.

In this embodiment, when each initial intent vector corresponds to an attribute element or a relationship element, by annotating entities and constraints under the initial intent vectors corresponding to each intent space, slot recognition can be performed simultaneously in different intent spaces, not only identifying each slot element in the question text sample, but also identifying the correspondence between attributes and entities, the correspondence between attributes and constraints, the correspondence between relationships and entities, and the correspondence between relationships and constraints, etc. In this way, there is no need to use a corresponding model for each slot element to perform model training for slot recognition, which reduces the number of models required for the slot recognition processing flow, reduces the time delay in the slot recognition processing flow, and simultaneously performs entity and constraint annotation in multiple intent spaces, which can parse the question text sample at one time to obtain the slot recognition results of each slot element therein, thereby improving the efficiency of slot recognition.

The question text sample may be composed of entity elements and attribute elements. For example, the question text sample is: How is the cost of product A calculated? Among them, the entity element is "product A", and the attribute element is "how is the cost calculated", which corresponds to the attribute "price".

The question text sample may be composed of an entity element and a constraint element. For example, the question text sample is: What restaurant activities can be participated in before this Sunday? Among them, the entity element is "restaurant activities" and the constraint element is "before this Sunday".

The question text sample can be composed of entity elements and relationship elements, for example, what are the handling channels for C activity? Among them, the entity element is "C activity" and the relationship element is "handling channel".

The question text sample can be composed of entity elements, for example, D service. The entity element is "D service". In the case where the question text sample only includes entity elements, the question is usually incomplete and needs to be asked in reverse to guide the user asking the question to supplement the question. In this case, the intent classification result of the first intent vector can be the second classification result, and the intent classification result of each second intent vector can be the first classification result, that is, the first sentence vector does not have any intent.

The structure of the question parsing model and the data processing method within the question parsing model can be exemplarily described below with reference to Figure 2. Figure 2 is a data flow diagram of a question parsing model in a model training method for a question-answering system provided in an embodiment of the present application.

As shown in FIG2 , the question text sample 202 is “How long does it take for liposuction to heal? How much does it cost to slim the face?” [sep] is used to represent the separator between two question text samples. [cls] means classification, which can be understood as a classification task for downstream. For text classification tasks, the BERT model inserts a [cls] symbol before the text and uses the output vector corresponding to the symbol as the semantic representation of the entire text for text classification.

The first encoding layer may be a Bert encoder 204 , and the transformation layer may include a spatial transformation layer 208 , a binary classification fully connected layer 210 , and a binary classification fully connected layer 212 .

The Bert encoder 204 is used to convert the question text sample 202 into a corresponding sentence vector 206 and send it to the spatial conversion layer 208; the spatial conversion layer 208 is used to generate a preset number of initial intention vectors when receiving the sentence vector 206, wherein the initial intention vector 1 corresponds to the intention space Null (null value) 220, the initial intention vector 2 corresponds to the intention space "introduction" 218, the initial intention vector 3 corresponds to the intention space "price" 216, and the initial intention vector 4 corresponds to the intention space "recovery period" 214.

The binary classification fully connected layer 210 can be used to perform intent classification processing according to the semantic feature sub-vector H[cls]2062, and obtain the intent classification results corresponding to each intent space and fill them in: the intent space "recovery period" 214 corresponds to the second classification result, and "1" is filled in the first position; the intent space "price" 216 corresponds to the second classification result, and "1" is filled in the first position; the intent space "introduction" 218 corresponds to the first classification result, and "0" is filled in the first position; the intent space Null220 corresponds to the first classification result, and "0" is filled in the first position.

The binary classification fully connected layer 212 can be used to perform entity recognition processing and constraint recognition processing according to each character sub-vector 2064 to obtain the corresponding character recognition result, and perform entity extraction processing and constraint extraction processing based on the character recognition result to obtain the entities "liposuction" and "face thinning" corresponding to the first sentence vector.

The entity "liposuction" is processed for intention prediction, and the intention space corresponding to the entity "liposuction" is determined to be the intention space "recovery period" 214 and filled, and the target intention vector corresponding to the intention space "recovery period" is obtained. The entity "face thinning" is processed for intention prediction, and the intention space corresponding to the entity "face thinning" is determined to be the intention space "price" 216 and filled, and the target intention vector corresponding to the intention space "price" is obtained.

According to the target intention vector corresponding to the intention space "recovery period" and the intention space "price" corresponding The target intent vector can generate the corresponding text segment. For other intent spaces whose intent classification results are the first classification results, the corresponding target intent vector can be discarded.

In a specific implementation, the question-answering system also includes an initial entity linking model; the model training method of the question-answering system also includes: obtaining entity linking samples; inputting the entity linking samples into the initial entity linking model for iterative training to obtain an entity linking model; the entity linking model includes a second encoding layer and a prediction layer; the second encoding layer is used to encode the entity linking samples to obtain a corresponding second sentence vector; the prediction layer is used to perform prediction processing based on the second sentence vector to determine the corresponding target entity.

Entity link samples can be obtained in the following ways: obtaining a standard entity library; the standard entity library includes multiple standard entities; and generating entity link samples based on the question parsing samples and the standard entity library.

Among them, in this embodiment, the specific implementation method of generating entity link samples based on question parsing samples and standard entity libraries involves a sample generation method of a question-answering system, and the specific scheme in the following method embodiment can be referred to.

The structure of the initial entity link model is exactly the same as that of the entity link model obtained after iterative training, but the model parameters involved in the training are different. The entity link model obtained after iterative training also has a second encoding layer and a prediction layer.

The second encoding layer can be a Bert converter or other components that can be used to convert text into sentence vectors. The prediction layer can be a binary fully connected layer linear or other components that can be used to map semantic feature sub-vectors to corresponding entities.

The prediction layer is used to perform prediction processing according to the second sentence vector to determine the corresponding target entity. Specifically, the prediction layer can be used to perform prediction processing according to the semantic feature subvector in the second sentence vector to determine the corresponding target entity.

In the training phase, the generated entity linking dataset can be used to train the BERT classification model. In the prediction phase, after the entity recognition is completed through the question parsing model, an entity mention segment will be obtained, and the entities with higher similarity will be recalled in the graph. Similar to the construction of data, the question with the mention segment and the recalled entity will be spliced separately. When using the entity linking model for prediction, it will be linked to the unique entity in the graph.

The structure of the entity linking model and the data processing flow within the entity linking model can be exemplarily described below in conjunction with Figure 3. Figure 3 is a data flow diagram of the entity linking model in a model training method for a question-answering system provided in an embodiment of the present application.

As shown in FIG3 , the entity link sample 302 is “[sep] How much does it cost to use $radio frequency face slimming$ [sep] radio frequency lipolysis face slimming”, and the entity link sample 302 is encoded by the Bert encoder 302 to obtain the corresponding second sentence vector, which includes the semantic feature sub-vector H[cls] 306. The semantic feature sub-vector H[cls] 306 is predicted by the binary classification fully connected layer 308 to determine the corresponding entity prediction result 310.

In a specific implementation, the question-answering system also includes an initial question classification model; the model training method of the question-answering system also includes: obtaining question classification samples; inputting the question classification samples into the initial question classification model for iterative training to obtain a question classification model.

Question classification samples can be obtained in the following ways: obtaining a question domain set; the question domain set includes in-domain question texts and out-of-domain question texts; generating question classification samples based on the question parsing samples and the question domain set.

Among them, the specific implementation method of obtaining the question domain set and the specific description of the question domain set involve the sample generation method of the question and answer system, and can refer to the specific scheme in the following method embodiment.

The problem classification sample can be an OOD data set, which is used to construct the training data required for the OOD model. The problem classification sample can include positive and negative samples. The positive sample is used to represent out-of-domain problems and is marked with label "1", and the negative sample is used to represent in-domain problems and is marked with label "0".

For example, a positive sample may be {"text":"Hello","label":1}.

Negative samples can be {"text":"What is the transaction time for unit notice deposit","label":0}, or {"text":"How long can the parking space loan last","label":0}.

Among them, the specific implementation method of generating question classification samples based on question analysis samples and question domain sets involves a sample generation method of a question-answering system, and reference may be made to the specific scheme in the following method embodiment.

In the embodiment shown in FIG1 , first, a question text sample is obtained; then, the question text sample is input into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when the first sentence vector is received, fill each initial intention vector according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system. In this way, the initial question parsing model is iteratively trained by the obtained question text sample. During the training process, the question text sample can be encoded to obtain the corresponding first sentence vector through the first encoding layer, and a preset number of initial intention vectors can be generated through the conversion layer, and then each initial intention vector can be filled based on the first sentence vector to obtain a corresponding target intention vector, and the target intention vector can be converted into a corresponding text fragment. The text fragment is used to query the answer to the question text sample in the question-answering system. The initial intent vector is filled, so that the question text sample can be parsed at one time to obtain a text fragment corresponding to at least one target intent vector. The question intent reflected by each text fragment can be determined by the corresponding initial intent vector, and the question text sample can be converted into different intent spaces through the question parsing model, and entities and constraints can be annotated under the intent vectors corresponding to each intent space, which is conducive to improving slot recognition efficiency and reducing the number of models required for the slot recognition processing flow, thereby reducing latency. In addition, slot recognition is performed simultaneously in each intent space, which can increase concurrency and improve the intent recognition capability and efficiency of the question-answering system.

Based on the same technical concept as the aforementioned method embodiment, the present application embodiment also provides an embodiment of a response method. Figure 4 is a processing flow chart of a response method provided in the present application embodiment. Referring to Figure 4, the processing flow of the response method specifically includes steps S402 to S406.

S402, obtaining a target question to be answered.

S404, inputting the target question into the question parsing model for parsing and processing to obtain the corresponding target segment; the question parsing model is obtained by training using the model training method of the question-answering system.

The model training method for the question-answering system in this step may be a model training method for the question-answering system provided by the aforementioned various method embodiments.

In a specific implementation method, the target problem is input into a problem analysis model for analysis processing to obtain a corresponding target fragment, including: inputting the target problem into a problem classification model for classification processing to obtain a classification result; when the classification result is used to characterize that the target problem belongs to a first preset classification, inputting the target problem into the problem analysis model for analysis processing to obtain a corresponding target fragment.

Step S406: Determine the answer to the target question based on the target segment.

In a specific implementation, the answer to the target question is determined based on the target fragment, including: inputting the target fragment into the entity linking model for prediction processing to obtain the corresponding target entity; performing slot filling processing based on the target entity to obtain the slot filling result of the target question; based on the slot filling result, querying the corresponding answer in the pre-configured knowledge graph to obtain the answer to the target question.

According to the target entity, slot filling processing is performed. For the target fragment obtained in step S404, first, at least one slot template that the target fragment may correspond to is determined, and then, for each slot template, slot filling processing is performed on the slot template through the target fragment to obtain the filled slot template, and the filled slot template is determined as the slot filling result corresponding to the target problem.

Each slot template may include one or more slots to be filled.

For example, slot template 1 is: What is the (attribute slot) of (entity slot)?

Slot template 2 is: (entity slot)(relationship slot)(entity slot)?

Slot template 3 is: (Constraint slot) (Entity slot) What are they?

It should be emphasized that the target entity can be used not only to fill the "entity slot", but also to fill multiple slots such as "attribute slot", "relationship slot" and "constraint slot".

According to the slot filling result, the corresponding answer is queried in the pre-configured knowledge graph. If the slot filling result cannot be used to query in the knowledge graph to obtain the unique corresponding answer, the slot missing information corresponding to the slot filling result is determined; a rhetorical question is generated according to the slot missing information; a user input in response to the rhetorical question is received; a slot filling process is performed according to the user input; and according to the slot filling result after the slot filling process, the corresponding answer is queried in the pre-configured knowledge graph to obtain the answer to the target question.

For example, the slot filling result of the target question includes the filled slot template 1: What is the (attribute slot) of (product A)? The filled slot template 1 cannot be used to query in the knowledge graph to obtain a unique corresponding answer. It can be determined that the slot missing information corresponding to the filled slot template 1 is attribute missing. According to the slot missing information "attribute missing", a corresponding rhetorical question "What information do you want to inquire about product A?" is generated. The user input "price" is received in response to the rhetorical question. The slot filling process is performed according to the user input, that is, the attribute slot in "What is the (attribute slot) of (product A)" is filled to obtain the slot template 1 after secondary filling: What is the (price) of (product A)? According to the slot filling result after the slot filling process, the answer corresponding to "What is the price of product A" is queried in the pre-configured knowledge graph to obtain the answer to the target question.

Next, how to determine the corresponding answer in the knowledge graph according to the slot filling result can be exemplified in conjunction with Figure 5. Figure 5 is a flow chart of session management in a response method provided in an embodiment of the present application.

As shown in Figure 5, after the question input by the user enters the conversation stack, the question-answering system can perform algorithmic analysis on the question to obtain an analysis result, which includes at least one slot identification result.

Perform a parsing result check. If the parsing result is used to indicate OOD or no result, generate a response text according to the pre-configured response words corresponding to "don't know/can't answer/flag bit". The response text is used to inform the user that the question-and-answer system cannot answer the question.

If the parsing result is used to represent a single result, a session stack detection is performed: for the first session, the slot management process is entered; for multiple rounds of dialogue, the session management process is entered.

If the parsing result is used to represent multiple results, the slot management process is entered, and the slot filling process is performed through the slot management process.

In the slot management process, you can check whether the composite constraint slot, entity slot, intent slot, common constraint slot exists, whether the node is a leaf node, and whether the intent slot is a common intent or a multi-value intent. Specifies the question template to be generated, or the answer template.

In the conversation management process, how to perform logical inheritance in multiple rounds of conversations can be determined through composite constraint detection, common constraint detection, entity detection, intent detection, node judgment, entity inheritance, constraint inheritance, and intent inheritance.

Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts may refer to the corresponding description of the method embodiment provided above.

In the above-mentioned embodiment, a model training method for a question-answering system is provided. Correspondingly, based on the same technical concept, an embodiment of the present application also provides a model training device for a question-answering system, which is described below in conjunction with the accompanying drawings.

FIG6 is a schematic diagram of a model training device for a question-answering system provided in an embodiment of the present application.

The present embodiment provides a model training device 600 for a question-answering system, wherein the question-answering system includes an initial question parsing model; the device includes: a first acquisition unit 602, used to acquire a question text sample; a first training unit 604, used to input the question text sample into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each of the initial intention vectors according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system.

Optionally, the first sentence vector includes a semantic feature sub-vector and multiple character sub-vectors; the specific implementation method of filling each of the initial intention vectors according to the first sentence vector to obtain the corresponding target intention vector is: performing intent classification processing on the semantic feature sub-vector to obtain the corresponding intention classification result; filling each of the initial intention vectors according to the intention classification result to obtain the corresponding intermediate intention vector; filling each of the intermediate intention vectors according to each of the character sub-vectors to obtain the corresponding target intention vector.

Optionally, the intention classification result includes a first classification result and a second classification result; the specific implementation method of filling each of the intermediate intention vectors according to each of the character sub-vectors to obtain the corresponding target intention vector is: performing entity recognition processing and constraint recognition processing according to each of the character sub-vectors to obtain the corresponding character recognition result; filling each of the intermediate intention vectors according to the character recognition result to obtain the corresponding target intention vector.

Optionally, the specific implementation method of filling each of the intermediate intention vectors according to the character recognition result to obtain the corresponding target intention vector is: if the character recognition result is used to characterize that the character sub-vector belongs to an entity category or a constraint category, then according to the character recognition result, the corresponding intermediate intention vector is determined and filled.

Optionally, the question text sample includes at least one of entity elements, attribute elements, relationship elements, and constraint elements; the preset number of initial intention vectors includes a first intention vector and multiple second intention vectors; the first intention vector corresponds to no intent, and each of the second intention vectors corresponds to one attribute element or one relationship element.

Optionally, the question-answering system also includes an initial entity linking model; the first acquisition unit 602 is also used to obtain entity linking samples; the first training unit 604 is also used to input the entity linking samples into the initial entity linking model for iterative training to obtain an entity linking model; the entity linking model includes a second encoding layer and a prediction layer; the second encoding layer is used to encode the entity linking samples to obtain a corresponding second sentence vector; the prediction layer is used to perform prediction processing based on the second sentence vector to determine the corresponding target entity.

Optionally, the question-answering system further includes an initial question classification model; the first acquisition unit 602 is further used to acquire question classification samples; the first training unit 604 is further used to input the question classification samples into the initial question classification model for iterative training to obtain a question classification model.

The model training device of the question-answering system provided in the embodiment of the present application includes a first acquisition unit and a first training unit, wherein: the first acquisition unit is used to acquire a question text sample; the first training unit is used to input the question text sample into the initial question parsing model for iterative training to obtain the question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each initial intention vector according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system. In this way, the initial question parsing model is iteratively trained through the obtained question text samples. During the training process, the question text sample can be encoded to obtain the corresponding first sentence vector through the first encoding layer, and a preset number of initial intention vectors can be generated through the conversion layer, and then each initial intention vector is filled based on the first sentence vector, so that the question text sample is parsed at one time to obtain a text segment corresponding to at least one target intention vector, and the question intent reflected by each text segment can be determined by the corresponding initial intention vector, and the question text sample can be converted into a target intention vector through the question parsing model. Switching to different intent spaces and marking entities and constraints under the intent vectors corresponding to each intent space can help improve slot recognition efficiency and reduce the number of models required for the slot recognition processing flow, thereby reducing latency. Simultaneous slot recognition in each intent space can increase concurrency and improve the intent recognition capability and efficiency of the question-answering system.

In the above-mentioned embodiment, a response method is provided. Correspondingly, based on the same technical concept, an embodiment of the present application also provides a response device, which will be described below in conjunction with the accompanying drawings.

FIG. 7 is a schematic diagram of a response device provided in an embodiment of the present application.

This embodiment provides an answering device 700, including: a second acquisition unit 702, used to acquire a target question to be answered; a first parsing unit 704, used to input the target question into a question parsing model for parsing and processing to obtain a corresponding target segment; the question parsing model is obtained by training using a model training method of a question-answering system; a first determination unit 706, used to determine an answer to the target question based on the target segment.

Optionally, the first parsing unit 704 is specifically used to: input the target problem into the problem classification model for classification processing to obtain a classification result; when the classification result is used to characterize that the target problem belongs to a first preset classification, input the target problem into the problem parsing model for parsing processing to obtain a corresponding target fragment.

Optionally, the first determination unit 706 is specifically used to: input the target fragment into the entity linking model for prediction processing to obtain a corresponding target entity; perform slot filling processing based on the target entity to obtain a slot filling result for the target question; based on the slot filling result, query the corresponding answer in a pre-configured knowledge graph to obtain an answer to the target question.

The answering device provided in the embodiment of the present application includes: a second acquisition unit, which is used to acquire a target question to be answered; a first parsing unit, which is used to input the target question into a question parsing model for parsing and processing to obtain a corresponding target segment; the question parsing model is obtained by training through a model training method of a question-answering system; and a first determination unit, which is used to determine the answer to the target question based on the target segment. In this way, during the training process, the question parsing model can encode the question text sample through the first encoding layer to obtain the corresponding first sentence vector, and through the conversion layer, a preset number of initial intention vectors can be generated, and then each initial intention vector can be filled based on the first sentence vector, so that the question text sample can be parsed at one time to obtain a text segment corresponding to at least one target intention vector, and the question intent reflected by each text segment can be determined by the corresponding initial intention vector, and the question text sample can be converted to different intention spaces through the question parsing model, and entities and constraints can be annotated under the intention vectors corresponding to each intention space, which is conducive to improving the efficiency of slot recognition and reducing The number of models required for the slot recognition processing flow is reduced, thereby reducing latency, and slot recognition is performed simultaneously in each intent space, which can increase concurrency and improve the intent recognition capability and efficiency of the question-answering system. Furthermore, the target fragments are generated through the question parsing model, and the answer to the target question is determined through the target fragments, which can improve the intent recognition accuracy and efficiency of the question-answering system.

Question-answering systems involving knowledge graphs are usually highly customized and require a large number of training samples to participate in model training in order to achieve good model effects and thus achieve efficient and accurate automatic responses. However, if training samples are manually configured, the efficiency is very low and it is difficult to meet the model training requirements. In order to solve the above problems, an embodiment of the present application provides a sample generation method for a question-answering system.

Figure 8 is a processing flow chart of a sample generation method of a question-and-answer system provided in an embodiment of the present application. The sample generation method of the question-and-answer system of Figure 8 can be executed by an electronic device, which can be a terminal device, such as a mobile phone, a laptop computer, an intelligent interactive device, etc.; or, the electronic device can also be a server, such as an independent physical server, a server cluster, or a cloud server capable of cloud computing. Referring to Figure 8, the sample generation method of the question-and-answer system provided in this embodiment specifically includes steps S802 to S808.

The question answering system may include an initial question classification model, an initial question parsing model, and an initial entity linking model.

Among them, the specific description of the question-answering system, the initial question classification model, the initial question parsing model and the initial entity linking model in the question-answering system in this embodiment involves the model training method of the question-answering system, and reference can be made to the specific scheme in the above method embodiment.

FIG9 is a schematic diagram of a framework of a question-answering system provided in an embodiment of the present application.

As shown in FIG9 , before the question answering system is put into use, it is necessary to first perform model training through the model training part. Specifically, it is necessary to iteratively train the initial question parsing model to obtain the question parsing model 902; it is necessary to iteratively train the initial entity linking model to obtain the entity linking model 904; it is necessary to iteratively train the initial OOD model to obtain the OOD model 906. To this end, data preparation is required before model training. Specifically, the training data set to be generated includes: a question parsing data set 908 for iterative training of the initial question parsing model, an entity linking data set 910 for iterative training of the initial entity linking model, and an OOD data set 912 for iterative training of the initial OOD model.

Step S802, obtaining a pre-configured synonym set, similar question set, comparison word set, problem domain set, and standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each standard word; the similar question set includes a plurality of first attributes and a corresponding entity library for each first attribute; At least one similar question; the comparison word set includes comparison word information; the question domain set includes in-domain question text and out-of-domain question text; the standard entity library includes multiple standard entities.

During specific implementation, the synonym set may be obtained from a pre-configured synonym template.

In the synonym template, the following fields can be displayed: standard word, type, synonym, update time, operation, etc.

Specifically: (a1) Standard words: automatically generated by the system, and the values are derived from the entities and attributes defined in the corresponding graph.

(a2) Type: Automatically generated by the system, the value comes from the type corresponding to the entity and attribute defined in the corresponding graph.

(a3) Synonyms: Manually enter via "Edit" or batch enter via "Upload Synonyms".

(a4) Update time: the time when the synonym was last modified.

The synonym set includes multiple standard words and at least one synonym corresponding to each standard word. For example, the synonym set includes: standard word x1, synonyms x2, synonyms x3 and synonyms x4 corresponding to the standard word x1; standard word y1, synonym y2 corresponding to the standard word y1; standard word z1, synonyms z2 and synonym z3 corresponding to the standard word z1, and so on.

During specific implementation, the similar question set can be obtained from a pre-configured similar question template.

In the similarity question template, the entity type column corresponds to the concept, the attribute column corresponds to the attribute, and similar questions are constructed for the attributes. When maintaining the similarity question template, pay attention to configuring the mask [e], that is, leaving space for filling in synonyms.

For example, "How do I apply for this parking space loan?" should be maintained as "How do I apply for this [e]?" in the similar question template.

Similar question templates can be used in question parsing datasets to form complete sentences together with synonyms and comparative words, simulating user questions in real scenarios.

In the similarity question template, the following fields can be displayed: concept, attribute name, attribute type, similarity question, update time, operation, etc.

Specifically: (b1) Concept: corresponds to the concept defined in the graph, which can be searched but cannot be added or edited.

(b2) Attribute type: corresponds to the attribute defined in the graph and cannot be edited.

Attribute types include but are not limited to: text, number, image, rich text, map, etc.

(b3) Similar questions: Similar questions of a specific attribute can be maintained individually or imported in batches. The number of similar questions can be one or more.

(b4) Update time: The time when the similar question was last modified.

(b5) Operation: Edit, Delete.

The similar question set includes multiple first attributes and at least one similar question corresponding to each first attribute. For example, the similar question set includes: first attribute A, and similar question A1 and similar question A2 corresponding to the first attribute A; first attribute B, and similar question B1 corresponding to the first attribute B, and so on.

During specific implementation, the comparison word set may be obtained from a pre-configured comparison word template.

The comparison word template can be used to generate a question parsing dataset. The maintainable candidate words are mainly attribute words with the concept attribute type of "number". Its purpose is to ensure that training data of relevant comparison types are generated in the parsing dataset.

The comparative word information may include a comparative word and a unit.

Comparative words include but are not limited to: lowest, smallest, least, shortest, cheapest, low, small, few, short, cheap, highest, largest, most, longest, most expensive, high, big, many, long, expensive, etc.

Units include but are not limited to: yuan, thousand, thousand yuan, ten thousand, ten thousand yuan, hundred million yuan, day, day, month, year, %, percent, etc.

During specific implementation, the problem domain set may be obtained from a pre-configured problem domain template.

The question domain template is used to generate OOD model data sets and maintain some questions and small talk that the question-answering system cannot answer. The backend will sample some question parsing data sets to form binary positive and negative samples to ensure the reasonable leakage of question-answering data. Due to certain limitations in the generated data, the OOD template provides editing and importing of both in-domain and out-of-domain questions to ensure classification accuracy and data maintainability.

The question domain set includes in-domain question texts and out-of-domain question texts.

The question domain set can be generated based on the pre-configured question domain. The question answering system has the ability to answer questions within the domain, but does not have the ability to answer questions outside the domain.

An in-domain problem may be a problem within the problem domain, and the text used to describe the in-domain problem may be the in-domain problem text. An out-of-domain problem may be a problem outside the problem domain, and the text used to describe the out-of-domain problem may be the out-of-domain problem text.

In-domain question text, such as "When is the deadline for activity A?"; out-of-domain question text, such as "Should I go out today?"

Step S804: based on at least one of the synonym set, the similar question set and the comparison word set, Generate problem parsing samples.

The problem analysis samples include but are not limited to: single slot samples, single attribute samples, single entity single attribute samples, single entity dual attribute samples, dual entity single attribute samples, dual entity dual attribute samples, composite attribute constraint samples, comparison type samples, etc. Exemplarily, the generated problem analysis samples can be stored in the following format:

The above storage format is merely exemplary and does not constitute any special limitation to this embodiment.

The graph attribute may be an attribute of the generated data in the graph, and is used to indicate the storage intention of the generated data in the graph.

Different types of question parsing samples are generated in different ways, and several of them are described in detail below.

In a specific implementation, generating a question analysis sample according to at least one of a synonym set, a similar question set, and a comparative word set includes: filtering the synonym set according to a first preset filtering condition to obtain a candidate word list; the candidate word list includes candidate standard words and candidate synonyms corresponding to the candidate standard words; performing a first sampling process on the candidate word list to obtain a single-slot sample, and determining the single-slot sample as a question parsing sample; or performing a second sampling process on multiple first attributes in a similar question set to obtain a target first attribute; performing a third sampling process on at least one similar question sentence corresponding to the target first attribute to obtain an initial similar question sentence; determining the corresponding single-attribute sample based on the initial similar question sentence, and determining the single-attribute sample as a question parsing sample.

Problem analysis samples may include single-slot samples or single-attribute samples.

In specific implementation, when the question analysis sample includes a single-slot sample, the question analysis sample is generated according to at least one of a synonym set, a similar question set and a comparative word set. The synonym set may be filtered according to a first preset filtering condition to obtain a candidate word list; the candidate word list includes candidate standard words and candidate synonyms corresponding to the candidate standard words; and the candidate word list is subjected to a first sampling process to obtain a single-slot sample.

A single-slot sample may be a text sample that corresponds to only one preset slot type among multiple preset slot types such as entity, attribute, relationship, constraint, etc.

Each preset slot type corresponds to one of the aforementioned slot identification results.

For example, the question "What is the price of product A?" is processed by slot recognition through the relationship extraction model, and the obtained slot recognition results include: the slot recognition result of "product A" is "entity", the slot recognition result of "price" is "attribute", and so on.

The slot recognition result "entity" corresponds to the preset slot type "entity". The single slot samples belonging to the preset slot type "entity" can be "A product", "B activity", "C product and D product", and so on.

The slot recognition result "attribute" corresponds to the preset slot type "attribute". Single slot samples belonging to the preset slot type "attribute" may be "price", "activity deadline", "interest rate", "discount", and so on.

The generation of single-slot samples is to simulate questions asked when the user's intention is unclear, and can be used in guided rhetorical questions and continuous rhetorical questions scenarios.

Exemplarily, a single-slot sample may be a single-entity sample, a double-entity sample, or a single-constraint sample.

It should be noted that although the "dual-entity sample" includes two entities, the two entities both correspond to a preset slot type, namely the preset slot type "entity", so the "dual-entity sample" is also a single-slot sample.

FIG. 10 is a first processing diagram of a sample generation method of a question-answering system provided in an embodiment of the present application. Flowchart sub-graph. FIG. 10 exemplarily shows a method for generating a single-slot sample, where the single-slot sample includes but is not limited to: single-entity data 1008 , double-entity data 1010 , and single-constraint data 1012 .

As shown in FIG10 , the first preset filtering condition may include the filtering condition “entity” and the filtering condition “json key (JS object notation keyword)”. Based on the filtering condition “entity”, the synonym set 1002 is filtered to obtain a candidate word list 1004, and the candidate word list 1004 is subjected to a first sampling process to generate single entity data 1008 and/or double entity data 1010. The sampling method of the first sampling process may be random sampling or other sampling methods.

Single entity data 1008, for example,

Among them, "Bill Business" located in the "text" row may be the generated data, that is, the text corresponding to the single entity data 1008. "NULL" may be a graph attribute, used to indicate that the storage intention of the single entity data 1008 is empty. "0" may be the start position of the header entity, "4" may be the end position of the header entity, "Business" may be the header entity category, and "Bill Business" located in the "value" row may be the standard name of the header entity.

Based on the filtering condition "json key", the synonym set 1002 is filtered to obtain a candidate word list 1006. The candidate word list 1006 is subjected to a first sampling process to generate single constraint data 1012.

Single constraint data 1012, for example,

Among them, "X Branch" in the "text" row may be the generated data, that is, the text corresponding to the single constraint data 1012. "Process flow" may be a graph attribute, used to indicate that the storage intention of the single constraint data 1012 is the process flow. "0" may be the starting position of the header entity, "4" may be the ending position of the header entity, "constraint" may be the header entity category, and "X Branch" in the "value" row may be the standard name of the header entity.

In a specific implementation, when the question parsing sample includes a single-attribute sample, the question parsing sample is generated according to at least one of a synonym set, a similar question set, and a comparative word set. The second sampling process may be performed on multiple first attributes in the similar question set to obtain a target first attribute; a third sampling process may be performed on at least one similar question sentence corresponding to the target first attribute to obtain an initial similar question sentence; and a corresponding single-attribute sample is determined according to the initial similar question sentence, and the single-attribute sample is determined as the question parsing sample.

Specifically, according to the initial similar question, the corresponding single-attribute sample is determined, and the single-attribute sample is determined as the question parsing sample. It can be: if the initial similar question carries a mask, the mask is deleted in the initial similar question to obtain the single-attribute sample; if the initial similar question does not carry a mask, the initial similar question is determined as a single-attribute sample.

A single attribute sample can be a text sample that only corresponds to the preset slot type "attribute". A single attribute sample can be a special single slot sample. Since the purpose of a single attribute sample is different from the "single entity sample", "double entity sample" and "single constraint sample" listed above, it is explained separately.

The generation of single-attribute samples is to simulate questions asked by users when the subject is unknown, and is used in guided rhetorical questions and continuous rhetorical questions scenarios.

Fig. 11 is a second processing flow chart of a sample generation method of a question-answering system provided in an embodiment of the present application. Fig. 11 exemplarily shows a generation method of a single attribute sample, and the single attribute sample can be single attribute data 1108 .

As shown in FIG11 , a second sampling process is performed on multiple first attributes in the similar question set 1102 to obtain a target first attribute; a third sampling process is performed on at least one similar question sentence corresponding to the target first attribute to obtain an initial similar question sentence. If the initial similar question sentence carries a mask, that is, the initial similar question sentence is a similar question sentence 1104 with [e], then [e] is deleted to obtain single attribute data 1108; if the initial similar question sentence does not carry a mask, that is, the initial similar question sentence is a similar question sentence 1106 without [e], then the similar question sentence 1106 is determined as single attribute data 1108.

The processing methods of the second sampling process and the third sampling process may be random sampling or other sampling methods.

Single attribute data 1108, for example,

Among them, "What is the handling process?" in the "text" row may be generated data, that is, the text corresponding to the single attribute data 1108. "Handling process" may be a graph attribute, used to indicate that the storage intention of the single attribute data 1108 is the handling process. The single attribute data 1108 has neither a head entity nor a tail entity.

In a specific implementation, a question parsing sample is generated according to at least one of a synonym set, a similar question set and a comparative word set, including: filtering the synonym set according to a second preset filtering condition to obtain a candidate entity word list; the candidate entity word list includes multiple candidate entity words; screening the similar question set to obtain an intermediate similar question set; the intermediate similar question set includes multiple first attributes and at least one candidate similar question sentence carrying a mask corresponding to each first attribute; according to the candidate word list and the intermediate similar question set, target candidate entity words and target candidate similar question sentences corresponding to the same attribute category are determined; the mask in the target candidate similar question sentence is replaced by the target candidate entity word to obtain a composite sample, and the composite sample is determined as a question parsing sample; or, filtering the synonym set according to a third preset filtering condition to obtain attribute synonyms and entity synonyms; generating random numbers; and selecting a composite sample according to comparative word information, attribute synonyms, entity synonyms, The random numbers and the preset description words are concatenated to obtain comparison type samples, and the comparison type samples are determined as problem analysis samples.

Problem analysis samples can include composite samples as well as comparative samples.

In a specific implementation, when the question parsing sample includes a composite sample, the question parsing sample is generated according to at least one of a synonym set, a similar question set and a comparative word set. The method may be to filter the synonym set according to a second preset filtering condition to obtain a candidate entity word list; the candidate entity word list includes a plurality of candidate entity words; the similar question set is screened to obtain an intermediate similar question set; the intermediate similar question set includes a plurality of first attributes and at least one candidate similar question sentence carrying a mask corresponding to each first attribute; according to the candidate word list and the intermediate similar question set, the target candidate entity words and the target candidate similar question sentence corresponding to the same attribute category are determined; and the mask in the target candidate similar question sentence is replaced by the target candidate entity word to obtain a composite sample.

A composite sample may be a text sample corresponding to multiple preset slot types among multiple preset slot types such as entity, attribute, relationship, constraint, etc.

Composite samples include but are not limited to: single-entity single-attribute samples, single-entity dual-attribute samples, dual-entity single-attribute samples, and dual-entity dual-attribute samples, etc.

Among them, the generation of single-entity single-attribute samples is to simulate the questions when users normally ask about the attributes of a thing, and is used in single-entity single-attribute scenarios. The generation of single-entity dual-attribute samples is to simulate the questions when users ask about two or more attributes of a thing at the same time, and is used in single-entity multi-attribute scenarios. The generation of dual-entity single-attribute samples is to simulate the questions when users ask about the same attribute of multiple things at the same time, and is used in multi-entity single-attribute scenarios. The generation of dual-entity dual-attribute samples is to simulate the questions when users ask about multiple attributes of multiple things at the same time, and is used in multi-entity multi-attribute scenarios.

FIG12 is a third processing flow sub-diagram of a sample generation method of a question-answering system provided in an embodiment of the present application. FIG12 exemplarily shows a generation method of a composite sample, and the composite sample can be composite data 1212. Composite data 1212 includes but is not limited to: single-entity single-attribute data, single-entity dual-attribute data, dual-entity single-attribute data, and dual-entity dual-attribute data.

As shown in FIG12 , the second preset filtering condition may include the filtering condition “entity”. Based on the filtering condition “entity”, the synonym set 1202 is filtered to obtain a candidate entity word list; the candidate entity word list includes multiple candidate entity words; the similar question set 1204 is screened to obtain an intermediate similar question set; the intermediate similar question set includes multiple first attributes and at least one candidate similar question sentence 1206 with a mask corresponding to each first attribute; according to the candidate word list and the intermediate similar question set, Determine the candidate entity words and candidate similar questions 1208 corresponding to each attribute category; randomly sample the candidate entity words and candidate similar questions 1208 corresponding to each attribute category to obtain target candidate entity words and target candidate similar questions 1210 corresponding to the same attribute category; replace the mask [e] in the target candidate similar questions with the target candidate entity words to obtain composite data 1212.

Determine the target candidate entity words and target candidate similar questions 1210 of the same attribute category; replace the mask [e] in the target candidate similar questions through the target candidate entity words to obtain composite data 1212, which can be, randomly select a target candidate entity word and a target candidate similar question under the same attribute category, and replace the mask [e] in the target candidate similar questions through the target candidate entity words to obtain single entity single attribute data.

Single-entity single-attribute data, for example,

Among them, "Where to support the processing of business A" can be the generated data, that is, the text corresponding to the single-entity single-attribute data. "Processing channel" can be a graph attribute, which is used to indicate that the storage intention of the single-entity single-attribute data is the processing channel. "9" can be the starting position of the header entity, "12" can be the ending position of the header entity, "Business" can be the header entity category, and "Business A" can be the standard name of the header entity.

Determine the target candidate entity words and target candidate similar questions of the same attribute category 1210; replace the mask [e] in the target candidate similar questions with the target candidate entity words to obtain composite data 1212; or randomly select a target candidate entity word and two target candidate similar questions under the same attribute category, replace the mask [e] in the first target candidate similar question with the target candidate entity words to obtain attribute question 1, and replace the mask [e] in the second target candidate similar question with the target candidate entity words. Delete [e] to get attribute question 2, concatenate attribute question 1 and attribute question 2 to get single-entity dual-attribute data.

Single entity dual attribute data, for example,

Among them, "What information do I need to submit to handle business X? What are the application requirements?" can be the generated data, that is, the text corresponding to the single-entity dual-attribute data. "Handling information" and "Handling conditions" can be two different graph attributes, used to indicate that the storage intention of the single-entity dual-attribute data includes handling information and handling conditions. "4" can be the starting position of the header entity, "11" can be the ending position of the header entity, "Business" can be the header entity category, "Export Letter of Credit" can be the first header entity standard name, and "X" can be the second header entity standard name.

It should be noted that, in the single-entity dual-attribute data, the first header entity and the second header entity are the same entity, so the starting position of the first header entity is the same as the starting position of the second header entity, and the ending position of the first header entity is the same as the ending position of the second header entity.

Determine the target candidate entity words and target candidate similar questions 1210 of the same attribute category; replace the mask [e] in the target candidate similar questions through the target candidate entity words to obtain composite data 1212. Alternatively, randomly select two target candidate entity words and a target candidate similar question under the same attribute category, randomly select one of multiple preset splicing methods as the target splicing method, splice the two target candidate entity words through the target splicing method to obtain the target splicing method, and replace the mask [e] in the target candidate similar questions through the target splicing word to obtain dual-entity parametric data.

A preset splicing method, for example, uses one of “and”, “and”, “,”, “,” to splice two words together.

Dual-entity single-attribute data, for example,

Among them, "How much is the credit line for business A and business B" can be the generated data, that is, the text corresponding to the double-entity single-attribute data. "Amount" can be a graph attribute used to represent the double entity The storage intention of single attribute data is the amount. "0" can be the start position of the first header entity, "6" can be the end position of the first header entity, "7" can be the start position of the second header entity, "12" can be the end position of the second header entity, "Business" can be the header entity category, "A Business" can be the standard name of the first header entity, and "B Business" can be the standard name of the second header entity.

It should be noted that, in the dual-entity single-attribute data, the first head entity and the second head entity are two different entities, but the graph attributes of the two entities are the same.

Fig. 13 is a fourth processing flow chart of a sample generation method of a question-answering system provided in an embodiment of the present application. Fig. 13 exemplarily shows a generation method of a dual-entity dual-attribute sample, where the dual-entity dual-attribute sample can be dual-entity dual-attribute data 1306 .

In specific implementation, multiple data can be randomly selected from the multiple generated single-entity single-attribute data, for example, single-entity single-attribute data 1302 and single-entity single-attribute data 1304, and single-entity single-attribute data 1302 and single-entity single-attribute data 1304 can be spliced together through randomly selected connectors to form dual-entity dual-attribute data 1306.

Dual-entity dual-attribute data 1306, for example,

Among them, "What are the specific ways to use the credit limit of product Y, and what kind of cards can generally be used for business Z" can be the generated data, that is, the text corresponding to the dual-entity and dual-attribute data. "Use of credit limit" and "Type" can be two different graph attributes, used to indicate that the storage intention of the dual-entity and dual-attribute data includes the use of credit limit and type. "0" can be the starting position of the first header entity, "3" can be the end position of the first header entity, "16" can be the starting position of the second header entity, "24" can be the end position of the second header entity, "Business" can be the header entity category, "Y Product" can be the standard name of the first header entity, and "Z Business" can be the standard name of the second header entity.

In addition, Fig. 14 is a fifth processing flow chart of a sample generation method of a question-answering system provided in an embodiment of the present application. Fig. 14 exemplarily shows a generation method of a composite attribute constraint sample. The composite attribute constraint sample can be composite attribute constraint data 1416.

The generation of compound attribute constraint samples is to simulate the questions asked by users when they ask about a certain attribute of a thing under certain limited conditions. They are used in compound attribute constraint scenarios and continuous rhetorical question scenarios.

As shown in FIG. 14 , the synonym set 1402 is filtered based on the filter condition “json key” to obtain json attribute synonyms 1406 , and the synonym set 1402 is filtered based on the filter condition “entity” to obtain entity synonyms 1408 .

The similar question set 1404 is screened to obtain similar questions with masks, that is, similar questions 1410 with [e].

According to the json attribute synonyms 1406, entity synonyms 1408 and similar questions 1410 with [e], the entity synonyms, json attribute synonyms and similar questions 1412 corresponding to each attribute category can be determined.

By randomly sampling the entity synonyms, json attribute synonyms, and similar questions 1412 corresponding to each attribute category, target entity synonyms, target json attribute synonyms, and target similar questions 1414 can be obtained. The [e] in the target similar questions is replaced by the target entity synonyms to obtain the intermediate similar questions, and then the splicing method is randomly determined. The target json attribute synonyms and the intermediate similar questions are spliced according to the determined splicing method to obtain the composite attribute constraint data 1416.

Composite attribute constraint data 1416, for example,

Among them, "Can you tell me about the process of handling business B mobile banking?" can be the generated data, that is, the text corresponding to the composite attribute constraint data 1416. "Process" can be a graph attribute, used to indicate that the storage intention of the composite attribute constraint data 1416 is the process. "3" can be the start position of the head entity, "6" can be the end position of the head entity, "11" can be the start position of the tail entity, "15" can be the end position of the tail entity, "business" can be the head entity category, "constraint" can be the tail entity category, "business B" can be the head entity standard name, and "mobile banking" can be the tail entity standard name.

In a specific implementation, when the question analysis sample includes a comparison type sample, the question analysis sample is generated according to at least one of a synonym set, a similar question set and a comparison word set. The synonym set may be filtered according to a third preset filtering condition to obtain attribute synonyms and entity synonyms; a random number is generated; and a splicing process is performed based on comparison word information, attribute synonyms, entity synonyms, random numbers and preset description words to obtain a comparison type sample.

It should be noted that, in the first preset filtering condition, the second preset filtering condition and the third preset filtering condition, “first”, “second” and “third” are only used to distinguish different filtering conditions and have no actual meaning.

The generation of comparison type samples is to simulate users asking questions that require numerical comparison, such as the maximum, minimum, greater than, less than, and between, of a certain attribute under a certain category of things, and is used in comparison sentence scenarios.

Fig. 15 is a sixth processing flow chart of a sample generation method of a question-answering system provided in an embodiment of the present application. Fig. 15 exemplarily shows a generation method of a comparison type sample, and the comparison type sample can be comparison type data 1512.

As shown in FIG. 15 , the synonym set 1504 is filtered according to the third preset filtering condition to obtain attribute synonyms 1506 and entity synonyms 1508 .

Generate a random number 1510.

Comparison word information 1502 , attribute synonyms 1506 , entity synonyms 1508 , random numbers 1510 , and preset description words in the comparison word set are concatenated to obtain comparison type data 1512 .

Comparison type data 1512, for example,

Among them, "How many years can the M product last at most" can be the generated data, that is, the text corresponding to the comparison type data 1512. "Term" can be a graph attribute, used to indicate that the storage intention of the comparison type data 1512 is the term. "0" can be the starting position of the header entity, "4" can be the ending position of the header entity, "Business" can be the category of the header entity, and "Asset Business" can be the standard name of the header entity.

Preset descriptive words, for example, what are there, what are there, which ones, what are all, what are there, let me tell you about it, etc.

In specific implementation, the comparison word information can be synchronized from the comparison word template, such as attributes, comparison words, units, etc.; the attribute synonyms of digital type in the synonym template, such as amount, quota, term, etc., are taken, and the entities of the same type of non-leaf nodes are obtained from the entity synonyms; when generating maximum, minimum, greater than, and less than type data, the attribute synonyms, comparison words, generated random numbers, and units are first spliced, and then the spliced string is randomly selected at a position with the entity synonyms and preset description words for splicing.

Assuming that the above contents are ABC, the possible splicing is: ABC, ACB, BAC, BCA, CBA, CAB, etc., which constitute the attribute question.

In specific implementation, the data of numerical single constraint can also be composed of comparison words, random numbers and units. For example, greater than 4.35% and less than 5 years.

In specific implementation, we can also select entities and synonyms of the same type as leaf nodes in the knowledge graph, randomly select a splicing method to splice two entity synonyms, and then use the conjunction to splice the attribute words and comparison words to form an attribute question. For example, which one has a higher interest rate, A or B; which one has a longer term, A or C?

Step S806, generating a question classification sample based on the question parsing sample and the question domain set; and generating an entity linking sample based on the question parsing sample and the standard entity library.

It should be noted that the generation of question classification samples and entity linking samples may use a portion of the question parsing samples generated in step S804.

Among them, the specific description of the question classification samples in this embodiment can refer to the corresponding content in the above method embodiment.

In a specific implementation, a problem classification sample is generated based on a problem analysis sample and a problem domain set, including: performing a fourth sampling process on the problem analysis sample to obtain a first domain sample; generating a corresponding second domain sample based on the domain problem text; generating a corresponding out-of-domain sample based on the out-of-domain problem text; and generating a problem classification sample based on the first domain sample, the second domain sample and the out-of-domain sample.

The sampling method of the fourth sampling process may be random sampling or other sampling methods.

It should be noted that in the first sampling process, the second sampling process, the third sampling process and the fourth sampling process, "first", "second", "third" and "fourth" are for the convenience of distinguishing the sampling operations performed in different steps and have no actual meaning.

Figure 16 is a seventh processing flow sub-diagram of a sample generation method of a question-answering system provided in an embodiment of the present application. Figure 16 exemplarily shows a method for generating a question classification sample, and its specific processing flow includes steps S1602 to S1606:

Step S1602: extract a first in-domain sample from the problem analysis sample, and obtain a second in-domain sample and an out-of-domain sample from the problem domain set.

Step S1604: adding positive labels to the first domain samples and the second domain samples, and adding negative labels to the out-of-domain samples.

Step S1606: merge all samples and save them into a json file.

In a specific implementation, an entity linking sample is generated based on a problem parsing sample and a standard entity library, including: determining a target parsing sample carrying a non-standard entity based on the problem parsing sample; calculating the similarity between the non-standard entity and each standard entity in the standard entity library and sorting them; determining a preset number of target standard entities corresponding to the non-standard entity based on the sorting result; and Quasi-entities and a preset number of target standard entities are used to construct positive and negative samples corresponding to the non-standard entities, and the positive and negative samples are determined as entity link samples.

Determining a preset number of target standard entities corresponding to the non-standard entity according to the sorting result may be determining the preset number of standard entities with the greatest similarity as the target standard entities.

For example, for the non-standard entity “Unit Notice Deposit”, the top 5 most similar entities are recalled in the standard entity library based on similarity, which are “Unit Notice Deposit”, “Unit Regular Deposit”, “Unit Current Deposit”, “Unit Agreement Deposit”, and “Unit Regular Deposit Passbook”.

According to the non-standard entity and a preset number of target standard entities, positive and negative samples corresponding to the non-standard entity are constructed. The positive sample can be constructed based on the non-standard entity and the target standard entity with the greatest similarity, and the negative sample can be constructed based on the non-standard entity and the target standard entity other than the target standard entity with the greatest similarity.

For example, the constructed positive sample is {"text":"What is the transaction time of "Unit Notice Deposit" [sep] Unit Notice Deposit","label":1}.

The constructed negative samples include: {"text":"What is the transaction time of "Unit Notice to Deposit Money"[sep]Unit Regular Deposit Money","label":0}; {"text":"What is the transaction time of "Unit Notice to Deposit Money"[sep]Unit Current Deposit Money","label":0}; {"text":"What is the transaction time of "Unit Notice to Deposit Money"[sep]Unit Agreement Deposit","label":0}; {"text":"What is the transaction time of "Unit Notice to Deposit Money"[sep]Unit Regular Deposit Book","label":0}.

Fig. 17 is an eighth processing flow sub-diagram of a sample generation method of a question-answering system provided in an embodiment of the present application. Fig. 17 exemplarily shows a generation method of entity link samples, and its specific processing flow includes steps S1702 to S1708.

Step S1702: randomly select a certain number of target parsing samples carrying non-standard entities from the question parsing samples.

Step S1704: Take out the non-standard entity and recall the top 5 most similar standard entities in the standard entity library.

Step S1706, constructing positive and negative samples based on the non-standard entities and the recalled standard entities.

Step S1708: merge all samples into a json file.

Step S808, constructing a training data set for the question answering system based on the question classification samples, question parsing samples, and entity linking samples.

This training dataset is used to train the initial question classification model, initial question parsing model, and initial entity linking model in the question answering system.

In the embodiment shown in FIG8 , first, a pre-configured synonym set, similar question set, and A set of synonyms, a set of comparison words, a set of problem domains, and a standard entity library; a set of synonyms includes multiple standard words and at least one synonym corresponding to each standard word; a set of similar questions includes multiple first attributes and at least one similar question sentence corresponding to each first attribute; a set of comparison words includes comparison word information; a problem domain set includes in-domain question texts and out-of-domain question texts; a standard entity library includes multiple standard entities; then, a question parsing sample is generated according to at least one of the synonym set, the set of similar questions, and the set of comparison words; then, a question classification sample is generated according to the question parsing sample and the problem domain set; and, an entity link sample is generated according to the question parsing sample and the standard entity library; finally, a training data set for the question answering system is constructed according to the question classification sample, the question parsing sample, and the entity link sample. In this way, on the one hand, question parsing samples, question classification samples and entity linking samples are generated according to pre-configured synonym sets, similar question sets, comparative word sets, problem domain sets and standard entity libraries, so that a large number of training samples can be generated using a small amount of pre-configured data, thereby improving sample generation efficiency and reducing manual workload; on the other hand, question parsing samples can not only be used to train the initial question parsing model, but also can be used to generate question classification samples and entity linking samples, thereby improving data utilization.

Based on the same technical concept as the aforementioned method embodiment, the embodiment of the present application also provides an embodiment of a model training method for a question-answering system. FIG18 is a processing flow chart of another model training method for a question-answering system provided by the embodiment of the present application. Referring to FIG18 , the processing flow of the model training method for a question-answering system specifically includes steps S1802 to S1804.

Step S1802, generating a training data set by using a sample generation method of a question-answering system; the training data set includes question classification samples, question parsing samples, and entity linking samples.

The sample generation method of the question-answering system may be the sample generation method of the question-answering system provided by the aforementioned sample generation method embodiment of the question-answering system.

Step S1804, input the question classification samples into the initial question classification model in the question-answering system for iterative training to obtain a question classification model; input the question parsing samples into the initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; input the entity linking samples into the initial entity linking model in the question-answering system for iterative training to obtain an entity linking model.

Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts may refer to the corresponding description of the method embodiment provided above.

Based on the same technical concept as the aforementioned method embodiment, the present application embodiment also provides an embodiment of a response method. FIG. 19 is a processing flow chart of another response method provided by the present application embodiment. FIG. 20 is a working principle diagram of a question-and-answer system provided by the present application embodiment. Referring to FIG. 19, The processing flow of the response method specifically includes step S1902 to step S1910.

The answering method shown in the embodiment of Figure 19 can be applied to a question-answering system, which may include a question classification model, a question parsing model, and an entity linking model connected in sequence. The output of the question classification model may be the input of the question parsing model, and the output of the question parsing model may be the input of the entity linking model.

Step S1902, obtaining the target question to be answered.

Step S1904, input the target question into the question classification model for classification processing to obtain the classification result; the question classification model is obtained by inputting the question classification samples in the training data set into the initial question classification model for training; the training data set is generated by the sample generation method of the question answering system.

The sample generation method of the question-answering system may be the sample generation method of the question-answering system provided by the aforementioned sample generation method embodiment of the question-answering system.

Step S1906, when the classification result is used to characterize that the target question belongs to the first preset classification, the target question is input into the question parsing model for parsing processing to obtain the corresponding target segment; the question parsing model is obtained by inputting the question parsing samples in the training data set into the initial question parsing model for training.

Step S1908, inputting the target segment into the entity linking model for prediction processing to obtain the corresponding target entity; the entity linking model is obtained by inputting the entity linking samples in the training data set into the initial entity linking model for training;

Step S1910, determining the answer to the target question based on the target entity.

In a specific implementation, the answer to the target question is determined according to the target entity, including: performing slot filling processing according to the target entity to obtain a slot filling result for the target question; and querying a corresponding answer in a pre-configured knowledge graph according to the slot filling result to obtain the answer to the target question.

According to the target entity, slot filling processing is performed. For the target entity obtained in step S1908, first, at least one slot template that may correspond to the target entity is determined, and then, for each slot template, slot filling processing is performed on the slot template through the target entity to obtain the filled slot template, and the filled slot template is determined as the slot filling result corresponding to the target problem.

Among them, the slot template in this embodiment is the same as the slot template in the aforementioned method embodiment, and the specific scheme in the aforementioned method embodiment can be referred to.

In a specific implementation, based on the slot filling result, The method comprises: if the slot filling result cannot be used to query the knowledge graph to obtain the unique corresponding answer, determining the slot missing information corresponding to the slot filling result; generating a rhetorical question according to the slot missing information; receiving user input in response to the rhetorical question; performing slot filling processing according to the user input; and querying the corresponding answer in a pre-configured knowledge graph according to the slot filling result after the slot filling processing to obtain the answer to the target question.

For example, the slot filling result of the target question includes the filled slot template 1: What is the (attribute slot) of (product A)? The filled slot template 1 cannot be used to query in the knowledge graph to obtain a unique corresponding answer. It can be determined that the slot missing information corresponding to the filled slot template 1 is attribute missing. According to the slot missing information "attribute missing", a corresponding rhetorical question "What information do you want to inquire about product A?" is generated. The user input "price" is received in response to the rhetorical question. The slot filling process is performed according to the user input, that is, the attribute slot in "What is the (attribute slot) of (product A)" is filled to obtain the slot template 1 after secondary filling: What is the (price) of (product A)? According to the slot filling result after the slot filling process, the answer corresponding to "What is the price of product A" is queried in the pre-configured knowledge graph to obtain the answer to the target question.

Next, in conjunction with the working principle diagram of the question-answering system shown in FIG. 20 , the processing flow of the answering method shown in FIG. 19 can be exemplified:

In actual use, when a user's question enters the question-and-answer system, the OOD model is first used to determine whether the user's question is an out-of-domain question or an in-domain question. If it is an in-domain question, the process continues; if it is an out-of-domain question, the system will uniformly reply with the pre-configured out-of-domain question response script.

Next, the question parsing model is used to identify slots in the domain question, including but not limited to: entities, attributes, relations, constraints, etc. After the entity recognition is completed through the question parsing model, one or more entity fragments will be obtained.

In the pre-configured knowledge graph, the entity node corresponding to the currently recognized entity fragment can be recalled, and the entity fragment can be linked to the unique entity in the knowledge graph using the entity linking model.

After the question parsing model is used for analysis, DM (Dialog Management) can ask counter-questions based on the missing slots. Users can personalize the relevant counter-question scripts in the counter-question template.

If the entity is missing, such as: How to apply? DM can ask: What product do you want to consult? If the attribute is missing, such as: Parking space loan. DM can ask: What specific questions do you want to consult about parking space loan?

The slot filling process can be performed through multiple rounds of strategies until the knowledge graph can be queried and returned. Until the only answer.

In the question-answering system shown in FIG20 , the OOD model can be obtained by iteratively training the initial OOD model based on the OOD training set; the question parsing model can be obtained by iteratively training the initial question parsing model based on the question parsing training set; and the entity linking model can be obtained by iteratively training the initial entity linking model based on the entity linking training set.

Among them, the question parsing training set can be generated based on synonym templates, similar question templates, comparative word templates and rhetorical question templates; the entity linking training set can be generated based on the standard entity library and the question parsing training set; the OOD training set can be generated based on the OOD template and the question parsing training set. The synonym template, similar question template, comparative word template, OOD template and rhetorical question template can be generated based on the customer knowledge base.

Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts may refer to the corresponding description of the method embodiment provided above.

In the above-mentioned embodiment, a sample generation method of a question-answering system is provided. Correspondingly, based on the same technical concept, an embodiment of the present application also provides a sample generation device of a question-answering system, which is described below in conjunction with the accompanying drawings.

FIG21 is a schematic diagram of a sample generation device of a question-answering system provided in an embodiment of the present application.

The present embodiment provides a sample generation device 2100 for a question-answering system, comprising: a third acquisition unit 2102, used to acquire a pre-configured synonym set, a similar question set, a comparison word set, a problem domain set, and a standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set includes a plurality of first attributes and at least one similar question sentence corresponding to each of the first attributes; the comparison word set includes comparison word information; the problem domain set includes an in-domain question text and an out-of-domain question text; the standard entity library includes a plurality of standard entities; a first generation unit 2104, used to generate a question parsing sample according to at least one of the synonym set, the similar question set, and the comparison word set; a second generation unit 2106, used to generate a question classification sample according to the question parsing sample and the problem domain set; and, based on the question parsing sample and the standard entity library, to generate the entity link sample; a construction unit 2108, used to construct a training data set for the question-answering system according to the question classification sample, the question parsing sample, and the entity link sample.

Optionally, the first generation unit 2104 is specifically configured to: filter the synonym set according to the first preset filtering condition to obtain a candidate word list; the candidate word list includes candidate standard words and candidate synonyms corresponding to the candidate standard words; perform a first sampling on the candidate word list Processing is performed to obtain a single-slot sample, and the single-slot sample is determined as the problem parsing sample; or, a second sampling process is performed on multiple first attributes in the similar question set to obtain a target first attribute; a third sampling process is performed on at least one similar question sentence corresponding to the target first attribute to obtain an initial similar question sentence; based on the initial similar question sentence, a corresponding single-attribute sample is determined, and the single-attribute sample is determined as the problem parsing sample.

Optionally, the first generating unit 2104 is specifically used to: filter the synonym set according to the second preset filtering condition to obtain a candidate entity word list; the candidate entity word list includes multiple candidate entity words; screen the similar question set to obtain an intermediate similar question set; the intermediate similar question set includes multiple first attributes and at least one candidate similar question sentence carrying a mask corresponding to each of the first attributes; determine the target candidate entity words and target candidate similar questions corresponding to the same attribute category according to the candidate word list and the intermediate similar question set; replace the mask in the target candidate similar question sentence with the target candidate entity word to obtain a composite sample, and determine the composite sample as the question parsing sample; or, filter the synonym set according to the third preset filtering condition to obtain attribute synonyms and entity synonyms; generate random numbers; splice the comparison word information, the attribute synonyms, the entity synonyms, the random numbers and the preset description words to obtain a comparison type sample, and determine the comparison type sample as the question parsing sample.

Optionally, the second generation unit 2106 is specifically used to: perform a fourth sampling process on the question parsing sample to obtain a first in-domain sample; generate a corresponding second in-domain sample based on the in-domain question text; generate a corresponding out-of-domain sample based on the out-of-domain question text; generate the question classification sample based on the first in-domain sample, the second in-domain sample and the out-of-domain sample.

Optionally, the second generation unit 2106 is specifically used to: determine a target parsing sample carrying a non-standard entity based on the problem parsing sample; calculate the similarity between the non-standard entity and each of the standard entities in the standard entity library and sort them; determine a preset number of target standard entities corresponding to the non-standard entity based on the sorting result; construct positive and negative samples corresponding to the non-standard entity based on the non-standard entity and the preset number of target standard entities, and determine the positive and negative samples as the entity link samples.

The sample generation device of the question-answering system provided in the embodiment of the present application includes a third acquisition unit for acquiring a pre-configured synonym set, a similar question set, a comparison word set, a problem domain set, and a standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each standard word; the similar question set includes a plurality of first attributes and at least one similar word corresponding to each first attribute. A question sentence; a comparison word set including comparison word information; a problem domain set including a problem text in the domain and a problem text outside the domain; a standard entity library including a plurality of standard entities; a first generation unit, used to generate a problem parsing sample according to at least one of a synonym set, a similar question set and a comparison word set; a second generation unit, used to generate a problem classification sample according to the problem parsing sample and the problem domain set; and, based on the problem parsing sample and the standard entity library, to generate an entity linking sample; a construction unit, used to construct a training data set for a question answering system according to the problem classification sample, the problem parsing sample and the entity linking sample. On the one hand, by generating a problem parsing sample, a problem classification sample and an entity linking sample according to the pre-configured synonym set, similar question set, comparison word set, problem domain set and standard entity library, a large number of training samples can be generated using a small amount of pre-configured data, the sample generation efficiency is improved, and the manual workload is reduced; on the other hand, the problem parsing sample can not only be used for the training of the initial problem parsing model, but also can be used to generate a problem classification sample and an entity linking sample, thereby improving the data utilization rate.

In the above-mentioned embodiment, a model training method for a question-answering system is provided. Correspondingly, based on the same technical concept, an embodiment of the present application also provides a model training device for a question-answering system, which is described below in conjunction with the accompanying drawings.

Figure 22 is a schematic diagram of a model training device for another question-answering system provided in an embodiment of the present application.

This embodiment provides a model training device 2200 for a question-answering system, comprising: a third generation unit 2202, used to generate a training data set by a sample generation method of the question-answering system; the training data set includes the question classification sample, the question parsing sample and the entity linking sample; a second training unit 2204, used to input the question classification sample into an initial question classification model in the question-answering system for iterative training to obtain a question classification model; input the question parsing sample into the initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; input the entity linking sample into the initial entity linking model in the question-answering system for iterative training to obtain an entity linking model.

The model training device of the question-answering system provided in the embodiment of the present application includes a third generation unit, which is used to generate a training data set through a sample generation method of the question-answering system; the training data set includes question classification samples, question parsing samples and entity linking samples; the second training unit is used to input the question classification samples into the initial question classification model in the question-answering system for iterative training to obtain a question classification model; input the question parsing samples into the initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; input the entity linking samples into the initial entity linking model in the question-answering system for iterative training to obtain an entity linking model. In this way, on the one hand, question parsing samples, question domain sets and standard entity libraries are generated according to the pre-configured synonym sets, similar question sets, comparative word sets and problem domain sets. Question classification samples and entity linking samples can generate a large number of training samples using a small amount of pre-configured data, improve sample generation efficiency, and reduce manual workload; on the other hand, question parsing samples can not only be used to train the initial question parsing model, but also to generate question classification samples and entity linking samples, which improves data utilization. On this basis, a large number of training samples can be efficiently generated for model training, thereby improving the accuracy of the question-answering system.

In the above-mentioned embodiment, a response method is provided. Correspondingly, based on the same technical concept, an embodiment of the present application also provides a response device, which will be described below in conjunction with the accompanying drawings.

FIG. 23 is a schematic diagram of another response device provided in an embodiment of the present application.

This embodiment provides an answering device 2300, including: a fourth acquisition unit 2302, used to acquire a target question to be answered; a classification unit 2304, used to input the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by a sample generation method of a question-answering system; a second parsing unit 2306, used to input the target question into a question parsing model for parsing processing to obtain a corresponding target segment when the classification result is used to characterize that the target question belongs to a first preset classification; the question parsing model is obtained by inputting question parsing samples in the training data set into an initial question parsing model for training; a prediction unit 2308, used to input the target segment into an entity linking model for prediction processing to obtain a corresponding target entity; the entity linking model is obtained by inputting entity linking samples in the training data set into an initial entity linking model for training; a second determination unit 2310, used to determine the answer to the target question according to the target entity.

Optionally, the second determination unit 2310 is specifically used to: perform slot filling processing according to the target entity to obtain a slot filling result for the target question; and query a corresponding answer in a pre-configured knowledge graph according to the slot filling result to obtain an answer to the target question.

Optionally, the second determination unit 2310 is also specifically used to: if the slot filling result cannot be used to query the knowledge graph to obtain a unique corresponding answer, then determine the slot missing information corresponding to the slot filling result; generate a rhetorical question based on the slot missing information; receive user input in response to the rhetorical question; perform slot filling processing based on the user input; and query the corresponding answer in a pre-configured knowledge graph based on the slot filling result after the slot filling processing to obtain the answer to the target question.

The answering device provided in the embodiment of the present application includes: a fourth acquisition unit, used to acquire a target question to be answered; a classification unit, used to input the target question into a question classification model for classification processing, The classification result is obtained; the question classification model is obtained by inputting the question classification samples in the training data set into the initial question classification model for training; the training data set is generated by the sample generation method of the question-answering system; the second parsing unit is used to input the target question into the question parsing model for parsing processing when the classification result is used to characterize that the target question belongs to the first preset classification, so as to obtain the corresponding target segment; the question parsing model is obtained by inputting the question parsing samples in the training data set into the initial question parsing model for training; the prediction unit is used to input the target segment into the entity linking model for prediction processing to obtain the corresponding target entity; the entity linking model is obtained by inputting the entity linking samples in the training data set into the initial entity linking model for training; the second determination unit is used to determine the answer to the target question according to the target entity. In this way, on the one hand, question parsing samples, question classification samples and entity linking samples are generated according to the pre-configured synonym sets, similar question sets, comparative word sets, problem domain sets and standard entity libraries, so that a large number of training samples can be generated using a small amount of pre-configured data, thereby improving the efficiency of sample generation and reducing manual workload; on the other hand, question parsing samples can not only be used to train the initial question parsing model, but also can be used to generate question classification samples and entity linking samples, thereby improving data utilization. On this basis, a large number of training samples can be efficiently generated for model training, thereby improving the response accuracy of the question and answer system.

Corresponding to the model training method of the question-answering system described above, or, corresponding to the answering method described above, or, corresponding to the sample generation method of the question-answering system described above, based on the same technical concept, an embodiment of the present application also provides an electronic device, which is used to execute the model training method of the question-answering system provided above, or, the answering method provided above. Figure 24 is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

As shown in FIG. 24 , electronic devices may have relatively large differences due to different configurations or performances, and may include one or more processors 2401 and memory 2402, and the memory 2402 may store one or more storage applications or data. Among them, the memory 2402 may be a short-term storage or a persistent storage. The application stored in the memory 2402 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 2401 may be configured to communicate with the memory 2402 to execute a series of computer executable instructions in the memory 2402 on the electronic device. The electronic device may also include one or more power supplies 2403, one or more wired or wireless network interfaces 2404, one or more input/output interfaces 2405, one or more keyboards 2406, and the like.

In a specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are stored in the memory. The program may include one or more modules, and each module may include a series of computer executable instructions in an electronic device, and is configured to be executed by one or more processors. The one or more programs include the following computer executable instructions: obtaining a question text sample; inputting the question text sample into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each of the initial intention vectors according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question and answer system.

In another specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, and include the following computer executable instructions: obtaining a preconfigured synonym set, a similar question set, a comparison word set, a problem domain set, and a standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set The set includes multiple first attributes and at least one similar question corresponding to each of the first attributes; the comparison word set includes comparison word information; the problem domain set includes in-domain question text and out-of-domain question text; the standard entity library includes multiple standard entities; based on the synonym set, the similar question set and at least one of the comparison word set, a question parsing sample is generated; based on the question parsing sample and the problem domain set, a question classification sample is generated; and, based on the question parsing sample and the standard entity library, the entity link sample is generated; based on the question classification sample, the question parsing sample and the entity link sample, a training data set for the question answering system is constructed.

In another specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, and include the following computer executable instructions: obtaining a target question to be answered; inputting the target question into a problem parsing model for parsing processing to obtain a corresponding target question; The question parsing model is obtained by training through the model training method of the question-answering system; and the answer to the target question is determined according to the target segment.

In another specific embodiment, the electronic device includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the electronic device, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following: generating a training data set by a sample generation method of a question-answering system; the training data set includes the question classification sample, the question parsing sample and the entity linking sample; inputting the question classification sample into an initial question classification model in the question-answering system for iterative training to obtain a question classification model; inputting the question parsing sample into an initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; inputting the entity linking sample into an initial entity linking model in the question-answering system for iterative training to obtain an entity linking model.

In another specific embodiment, the electronic device includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the electronic device, and the one or more programs are configured to be executed by one or more processors to include the following computer executable instructions: obtaining a target question to be answered; inputting the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by a sample generation method of a question-answering system; when the classification result is used to characterize that the target question belongs to a first preset classification, inputting the target question into a question parsing model for parsing processing to obtain a corresponding target segment; the question parsing model is obtained by inputting question parsing samples in the training data set into an initial question parsing model for training; inputting the target segment into an entity linking model for prediction processing to obtain a corresponding target entity; the entity linking model is obtained by inputting entity linking samples in the training data set into an initial entity linking model for training; and determining the answer to the target question based on the target entity.

An embodiment of a computer-readable storage medium provided in this specification is as follows:

Corresponding to the model training method of a question-answering system described above, based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium.

The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process is implemented: obtaining a question text sample; inputting the question text sample into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; the first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; the conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each of the initial intention vectors according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question-answering system.

It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the model training method for the question-answering system in this specification are based on the same concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method mentioned above, and the repeated parts will not be repeated.

Corresponding to the response method described above, based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium.

The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process is implemented: obtaining a target question to be answered; inputting the target question into a question parsing model for parsing and processing to obtain a corresponding target segment; the question parsing model is obtained by training using a model training method of a question-answering system; and determining an answer to the target question based on the target segment.

It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the response method in this specification are based on the same concept, so the specific implementation of this embodiment can refer to the implementation of the corresponding method mentioned above, and the repeated parts will not be repeated.

Corresponding to the sample generation method of a question-answering system described above, based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium.

The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process is implemented: obtaining a pre-configured synonym set, a similar question set, a comparison word set, a problem domain set, and a standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set includes a plurality of first attributes and at least one similar question sentence corresponding to each of the first attributes; the comparison word set includes comparison word information; the problem domain set includes in-domain question text and out-of-domain question text; the standard entity library includes a plurality of standard entities; Generate a question parsing sample based on a set of similar questions and at least one of the comparison word sets; generate a question classification sample based on the question parsing sample and the problem domain set; and generate the entity linking sample based on the question parsing sample and the standard entity library; construct a training data set for the question-answering system based on the question classification sample, the question parsing sample and the entity linking sample.

It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the sample generation method for the question-and-answer system in this specification are based on the same concept, so the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

Corresponding to the model training method of a question-answering system described above, based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium.

The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process is implemented: a training data set is generated by a sample generation method of a question-answering system; the training data set includes the question classification samples, the question parsing samples and the entity linking samples; the question classification samples are input into an initial question classification model in the question-answering system for iterative training to obtain a question classification model; the question parsing samples are input into an initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; the entity linking samples are input into an initial entity linking model in the question-answering system for iterative training to obtain an entity linking model.

It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the model training method for the question-answering system in this specification are based on the same concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method mentioned above, and the repeated parts will not be repeated.

Corresponding to the response method described above, based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium.

The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, and the computer-executable instructions implement the following process when executed by the processor: obtaining a target question to be answered; inputting the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by a sample generation method of a question-answering system; when the classification result is used to characterize that the target question belongs to a first preset classification, inputting the target question into a question parsing model for parsing processing to obtain a corresponding target segment; the question parsing model is obtained by inputting question parsing samples in the training data set into an initial question parsing model The target segment is input into the entity linking model for prediction processing to obtain the corresponding target entity; the entity linking model is obtained by inputting the entity linking samples in the training data set into the initial entity linking model for training; according to the target entity, the answer to the target question is determined.

It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the response method in this specification are based on the same concept, so the specific implementation of this embodiment can refer to the implementation of the corresponding method mentioned above, and the repeated parts will not be repeated.

The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

This specification is described with reference to the flowchart and/or block diagram of the method, device (system), and computer program product according to the embodiment of this specification. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to produce a machine, so that the instructions executed by the processor of the computer or other programmable device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions can also be loaded into a computer or other programmable device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a method for implementing a process in the flowchart. or multiple flows and/or block diagrams with steps of functions specified in one or more blocks.

In a typical configuration, a computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Embodiments of the present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is The description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

The above description is only an embodiment of this document and is not intended to limit this document. For those skilled in the art, this document may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document should be included in the scope of the claims of this document.

Claims (24)

A model training method for a question-answering system, the question-answering system comprising an initial question parsing model; the method comprising: Get sample question text; Inputting the question text sample into the initial question parsing model for iterative training to obtain a question parsing model; the initial question parsing model includes a first encoding layer and a conversion layer; The first encoding layer is used to perform encoding processing according to the question text sample to obtain a corresponding first sentence vector; The conversion layer is used to generate a preset number of initial intention vectors when receiving the first sentence vector, fill each of the initial intention vectors according to the first sentence vector to obtain a corresponding target intention vector, and convert the target intention vector into a corresponding text fragment; the text fragment is used to query the answer to the question text sample in the question and answer system. The method according to claim 1, wherein the first sentence vector includes a semantic feature sub-vector and a plurality of character sub-vectors; and the specific implementation methods of performing filling processing on each of the initial intention vectors according to the first sentence vector to obtain the corresponding target intention vector are: Performing intent classification processing on the semantic feature sub-vector to obtain a corresponding intent classification result; According to the intention classification result, each of the initial intention vectors is filled to obtain a corresponding intermediate intention vector; According to each of the character sub-vectors, each of the intermediate intention vectors is filled to obtain a corresponding target intention vector. According to the method of claim 2, wherein the specific implementation methods of performing filling processing on each of the intermediate intention vectors according to each of the character sub-vectors to obtain the corresponding target intention vector are: According to each of the character sub-vectors, entity recognition processing and constraint recognition processing are performed to obtain a corresponding character recognition result; According to the character recognition result, each of the intermediate intention vectors is filled to obtain a corresponding target intention vector. According to the method of claim 3, wherein the specific implementation manner of performing filling processing on each of the intermediate intention vectors according to the character recognition result to obtain the corresponding target intention vector is: If the character recognition result is used to characterize that the character sub-vector belongs to an entity category or a constraint category, then the corresponding intermediate intention vector is determined and filled according to the character recognition result. According to the method described in any one of claims 1 to 4, the question text sample includes at least one of an entity element, an attribute element, a relationship element, and a constraint element; the preset number of the initial intention vectors includes a first intention vector and multiple second intention vectors; the first intention vector corresponds to no intent, and each of the second intention vectors corresponds to one of the attribute elements or one of the relationship elements. The method according to claim 1, wherein the question answering system further comprises an initial entity linking model; the method further comprises: Get entity link samples; Inputting the entity link sample into the initial entity link model for iterative training to obtain an entity link model; the entity link model includes a second encoding layer and a prediction layer; The second encoding layer is used to encode the entity link sample to obtain a corresponding second sentence vector; The prediction layer is used to perform prediction processing according to the second sentence pattern vector to determine the corresponding target entity. The method according to claim 1, wherein the question answering system further comprises an initial question classification model; the method further comprises: Get problem classification samples; The question classification samples are input into the initial question classification model for iterative training to obtain a question classification model. A response method, comprising: Get the target questions to be answered; Inputting the target problem into the problem parsing model for parsing and processing to obtain a corresponding target segment; The question parsing model is obtained by training through the model training method of the question answering system according to any one of claims 1 to 7; An answer to the target question is determined based on the target segment. The method according to claim 8, wherein the step of inputting the target question into a question parsing model for parsing to obtain a corresponding target segment comprises: Inputting the target problem into the problem classification model for classification processing to obtain a classification result; In the case where the classification result is used to characterize that the target question belongs to the first preset classification, the target question is input into the question parsing model for parsing to obtain a corresponding target segment. The method according to claim 8, wherein determining the answer to the target question based on the target segment comprises: Inputting the target segment into an entity linking model for prediction processing to obtain a corresponding target entity; Perform slot filling processing according to the target entity to obtain a slot filling result of the target problem; According to the slot filling result, the corresponding answer is queried in the pre-configured knowledge graph to obtain the answer to the target question. A response device, comprising: A second acquisition unit, used to acquire a target question to be answered; A first parsing unit, configured to input the target question into a question parsing model for parsing to obtain a corresponding target segment; the question parsing model is obtained by training using the model training method for the question-answering system according to any one of claims 1 to 7; The first determining unit is used to determine an answer to the target question according to the target segment. A sample generation method for a question-answering system, the method comprising: Acquire a pre-configured synonym set, similar question set, comparative word set, problem domain set and standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set includes a plurality of first attributes and at least one similar question sentence corresponding to each of the first attributes; the comparative word set includes comparative word information; the problem domain set includes in-domain question text and out-of-domain question text; the standard entity library includes a plurality of standard entities; Generate a question parsing sample according to at least one of the synonym set, the similar question set and the comparative word set; Generate a question classification sample according to the question parsing sample and the question domain set; and generate the entity linking sample according to the question parsing sample and the standard entity library; A training data set for the question answering system is constructed based on the question classification samples, the question parsing samples, and the entity linking samples. The method according to claim 12, wherein generating a question parsing sample based on at least one of the synonym set, the similar question set, and the comparative word set comprises: According to the first preset filtering condition, the synonym set is filtered to obtain a candidate word list; the candidate word list includes candidate standard words and candidate synonyms corresponding to the candidate standard words; Performing a first sampling process on the candidate word list to obtain a single-slot sample, and determining the single-slot sample as a question parsing sample; or, Performing a second sampling process on a plurality of the first attributes in the similar question set to obtain a target first attribute; Performing a third sampling process on at least one similar question sentence corresponding to the first attribute of the target to obtain an initial similar question sentence; According to the initial similar question, a corresponding single attribute sample is determined, and the single attribute sample is determined as the question resolution sample. The method according to claim 12, wherein generating a question parsing sample based on at least one of the synonym set, the similar question set, and the comparative word set comprises: According to the second preset filtering condition, the synonym set is filtered to obtain a candidate entity word list; the candidate entity word list includes a plurality of candidate entity words; The similar question set is screened to obtain an intermediate similar question set; the intermediate similar question set includes a plurality of first attributes and at least one candidate similar question sentence with a mask corresponding to each of the first attributes; Determine target candidate entity words and target candidate similar question sentences corresponding to the same attribute category according to the candidate word list and the intermediate similar question set; Using the target candidate entity word, the mask in the target candidate similar question is replaced to obtain a composite sample, and the composite sample is determined as the question parsing sample; or, Filter the synonym set according to a third preset filtering condition to obtain attribute synonyms and entity synonyms; Generate random numbers; A splicing process is performed according to the comparison word information, the attribute synonyms, the entity synonyms, the random number and the preset description word to obtain a comparison type sample, and the comparison type sample is determined as the question parsing sample. The method according to claim 12, wherein generating a problem classification sample based on the problem parsing sample and the problem domain set comprises: Performing a fourth sampling process on the problem analysis sample to obtain a first domain sample; Generating a corresponding second in-domain sample according to the in-domain question text; Generate corresponding out-of-domain samples according to the out-of-domain question text; The question classification sample is generated according to the first in-domain sample, the second in-domain sample and the out-of-domain sample. The method according to claim 12, wherein generating the entity link sample according to the question parsing sample and the standard entity library comprises: Determine a target parsing sample carrying a non-standard entity according to the problem parsing sample; Calculating the similarity between the non-standard entity and each of the standard entities in the standard entity library and sorting them; According to the sorting result, determining a preset number of target standard entities corresponding to the non-standard entity; According to the non-standard entity and the preset number of target standard entities, positive and negative samples corresponding to the non-standard entity are constructed, and the positive and negative samples are determined as the entity link samples. A model training method for a question answering system, comprising: Generate a training data set by using the sample generation method of the question answering system according to any one of claims 12 to 16; the training data set includes the question classification sample, the question parsing sample and the entity linking sample; The question classification samples are input into the initial question classification model in the question-answering system for iterative training to obtain a question classification model; the question parsing samples are input into the initial question parsing model in the question-answering system for iterative training to obtain a question parsing model; the entity linking samples are input into the initial entity linking model in the question-answering system for iterative training to obtain an entity linking model. A response method, comprising: Get the target questions to be answered; Inputting the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by the sample generation method of the question answering system according to any one of claims 12 to 16; In the case where the classification result is used to characterize that the target question belongs to the first preset classification, the target question is input into a question parsing model for parsing processing to obtain a corresponding target segment; the question parsing model is obtained by inputting question parsing samples in the training data set into an initial question parsing model for training; Inputting the target segment into an entity linking model for prediction processing to obtain a corresponding target entity; the entity linking model is obtained by inputting entity linking samples in the training data set into an initial entity linking model for training; An answer to the target question is determined according to the target entity. The method according to claim 18, wherein determining the answer to the target question according to the target entity comprises: Perform slot filling processing according to the target entity to obtain a slot filling result of the target problem; According to the slot filling result, the corresponding answer is queried in the pre-configured knowledge graph to obtain the answer to the target question. The method according to claim 19, wherein the querying a corresponding answer in a preconfigured knowledge graph according to the slot filling result comprises: If the slot filling result cannot be used to query in the knowledge graph to obtain a unique corresponding answer, then determining the slot missing information corresponding to the slot filling result; Generate a rhetorical question sentence according to the slot missing information; receiving a user input in response to the rhetorical question; Perform slot filling processing according to the user input; According to the slot filling results after the slot filling process, the corresponding answer is queried in the pre-configured knowledge graph to obtain the answer to the target question. A sample generation device for a question-answering system, the device comprising: The third acquisition unit is used to acquire a pre-configured synonym set, a similar question set, a comparative word set, a problem domain set, and a standard entity library; the synonym set includes a plurality of standard words and at least one synonym corresponding to each of the standard words; the similar question set includes a plurality of first attributes and at least one similar question sentence corresponding to each of the first attributes; the comparative word set includes comparative word information; the problem domain set includes in-domain question text and out-of-domain question text; the standard entity library includes a plurality of standard entities; A first generating unit, configured to generate a question parsing sample according to at least one of the synonym set, the similar question set and the comparative word set; A second generating unit is used to generate a question classification sample according to the question parsing sample and the question domain set; and to generate the entity linking sample according to the question parsing sample and the standard entity library; A construction unit is used to construct a training data set for the question answering system according to the question classification samples, the question parsing samples and the entity linking samples. A response device, comprising: A fourth acquisition unit, used to acquire a target question to be answered; A classification unit, used for inputting the target question into a question classification model for classification processing to obtain a classification result; the question classification model is obtained by inputting question classification samples in a training data set into an initial question classification model for training; the training data set is generated by the sample generation method of the question answering system according to any one of claims 12 to 16; A second parsing unit is used to input the target question into a question parsing model for parsing and obtaining a corresponding target segment when the classification result is used to characterize that the target question belongs to a first preset classification; the question parsing model is obtained by inputting the question parsing samples in the training data set into an initial question parsing model for training; A prediction unit, configured to input the target segment into an entity linking model for prediction processing to obtain a corresponding target entity; the entity linking model is obtained by inputting entity linking samples in the training data set into an initial entity linking model for training; The second determining unit is used to determine the answer to the target question according to the target entity. An electronic device, comprising: A processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to execute the model training method for the question-answering system as described in any one of claims 1 to 7, or the answering method as described in any one of claims 8 to 10, or the sample generation method for the question-answering system as described in any one of claims 12 to 16, or the model training method for the question-answering system as described in claim 17, or the answering method as described in any one of claims 18 to 20. A computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the model training method for a question-answering system as described in any one of claims 1-7, or the answering method as described in any one of claims 8-10, or the sample generation method for a question-answering system as described in any one of claims 12-16, or the model training method for a question-answering system as described in claim 17, or the answering method as described in any one of claims 18-20.