WO2025092056A1 - Question-and-answer data generation method and apparatus, and computer device and storage medium - Google Patents

Abstract

The present disclosure relates to the field of data processing, and in particular to a question-and-answer (QA) data generation method and apparatus, and a computer device and a storage medium. The method comprises: acquiring a plurality of segmented document blocks, wherein each segmented document block includes a first preset number of pieces of text data; on the basis of the text data in the segmented document blocks and a target questioning model, obtaining target question data corresponding to the segmented document blocks; acquiring full data, wherein the full data is content information which is associated with the target question data and is included in a complete document composed of the plurality of segmented document blocks; on the basis of the segmented document blocks, the target question data, the full data and a target answering model, obtaining target answer data corresponding to the target question data; and generating QA data on the basis of the target question data and the target answer data. The present disclosure solves the problems in the related art of low efficiency and poor data quality in QA data mining for a document.

Description

Question and answer data generation method, device, computer equipment and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese invention patent application number 202311434678.6, entitled “Method, device, computer equipment and storage medium for generating question and answer data” and filed on October 31, 2023, and the entire application is incorporated herein by reference.

Technical Field

The present disclosure relates to the field of data processing, and in particular to a method, device, computer equipment and storage medium for generating question and answer data.

Background Art

Most companies have a large amount of existing text data, which is accumulated in the long-term business development process. If companies want to try to make large models intelligent, they need to use these existing data. However, these data have not been cleaned, sorted, or labeled, which makes it inefficient for technicians to mine the required QA (Question and Answer) data (also known as question and answer data) from existing documents, and the mined data is of low quality.

Currently, in order to ensure that high-quality QA data can be mined from existing documents, manual annotation is usually used. However, this method costs a lot of labor and time, and timeliness cannot be guaranteed. At the same time, manual annotation depends on the professional qualities of the staff themselves, resulting in uneven quality of the mined data, and even a large amount of poor quality data.

Therefore, when related technologies are used for QA data mining of documents, there are problems of low efficiency and poor data quality.

Summary of the invention

In view of this, the present disclosure provides a method, apparatus, computer device and storage medium for generating question and answer data to solve the problems of low efficiency and poor data quality when performing QA data mining on documents in related technologies.

In a first aspect, the present disclosure provides a method for generating question-answer data, the method comprising:

Acquire a plurality of cut document blocks, wherein each cut document block contains a first preset number of text data;

According to the text data in the cut document block and the target question model, the target question data corresponding to the cut document block is obtained;

Acquire full data, wherein the full data is content information associated with the target question data contained in a complete document composed of a plurality of cut document blocks;

According to the cut document blocks, the target question data, the full amount of data and the target answer model, the target answer data corresponding to the target question data is obtained;

Generate question and answer data based on the target question data and the target answer data.

In an optional implementation, before obtaining the target question data corresponding to the cut document block according to the text data in the cut document block and the target question model, the method further includes:

Determine the questioning strategy based on the text data in the cut document block;

Process the text data according to the questioning strategy and extract multiple questioning questions;

Obtaining first scores for multiple questions according to a question scoring model;

Determine a plurality of candidate questions from the plurality of questions according to the first score;

The initial question model is optimized according to the candidate questions to obtain the target question model.

In an optional implementation, determining a questioning strategy based on text data in the cut document block includes:

Determine the text type of the text data;

Determine the corresponding questioning strategy based on the text type.

In an optional implementation, before obtaining target answer data corresponding to the target question data according to the segmented document blocks, the target question data, the full data, and the target answer model, the method further includes:

Determine the answer strategy based on the text data in the cut document block and the target question data;

Processing the text data according to the answer strategy to obtain multiple answer data;

Obtaining a second score for the plurality of answer data according to the answer scoring model;

determining a plurality of candidate answer data from the plurality of answer data according to the second score;

The initial answer model is optimized according to the candidate answer data to obtain the target answer model.

In an optional implementation, determining an answer strategy based on text data and target question data in the segmented document block includes:

Determine the text type of the text data;

Determine the answer strategy based on the text type and target question data.

In an optional implementation, target answer data corresponding to the target question data is obtained according to the segmented document blocks, the target question data, the full amount of data, and the target answer model, including:

According to the cut document blocks, the target question data, the full amount of data and the target answer model, multiple answer data are obtained;

Split the answer data into the smallest unit to obtain multiple target fields;

comparing the description contents of the target field in each second preset number of answer data;

The description content corresponding to the answer data with the second highest score is retained;

Integrate the description content to obtain the target answer data.

In an optional implementation, after obtaining the question-answer data about the cut document block, the method further includes:

The question and answer data is scored using the question and answer data scoring model to obtain a third score for the question and answer data;

The question and answer data is cleaned according to the third score to obtain target question and answer data that meets the preset requirements;

The data sample is expanded according to the target question and answer data to obtain a third preset number of question and answer data.

In an optional implementation, the cut document block includes text data after image recognition.

In a second aspect, the present disclosure provides a device for generating question-answer data, the device comprising:

A first acquisition module is used to acquire a plurality of cut document blocks, wherein each cut document block contains a first preset number of text data;

The first obtaining module is used to obtain the target question data corresponding to the cut document block according to the text data in the cut document block and the target question model;

A second acquisition module is used to acquire full data, wherein the full data is content information associated with the target question data contained in a complete document composed of a plurality of cut document blocks;

The second obtaining module is used to obtain target answer data corresponding to the target question data according to the cut document blocks, the target question data, the full amount of data and the target answer model;

The third module is used to generate question-answer data based on the target question data and the target answer data.

In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the generation of question and answer data of the first aspect or any corresponding embodiment thereof. Into method.

In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for generating question and answer data of the first aspect or any corresponding embodiment thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a flow chart of a method for generating question and answer data according to some embodiments of the present disclosure;

FIG2 is a schematic diagram of a complete flow chart of a method for generating question and answer data according to some embodiments of the present disclosure;

FIG3 is a structural block diagram of a device for generating question and answer data according to some embodiments of the present disclosure;

FIG. 4 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

Most companies have a large amount of existing text data, which has been accumulated in the long-term business development process in the past. If companies want to try to make large models intelligent, they need to use this existing data. However, this data has not been cleaned, sorted, or labeled, and can only be pre-trained. The cost of pre-training is very high, and the amount of data that most companies can accumulate is not large enough, so the effect of pre-training is limited.

Therefore, the best strategy for enterprises to try is to fine-tune the model. At this time, it is necessary to clean and QA the past business data. However, due to data security issues, it is difficult for enterprises to give data to third-party labeling companies for data processing. Therefore, in order to ensure that high-quality QA data can be mined from existing documents, manual labeling is usually used. However, this method costs a lot of labor and time costs, and timeliness cannot be guaranteed. At the same time, manual labeling needs to rely on the professional qualities of the staff themselves, resulting in uneven quality of the mined data, and even a lot of poor quality data.

In order to solve the above problems, the embodiment of the present disclosure proposes an embodiment of a method for generating question and answer data. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

In this embodiment, a method for generating question and answer data is provided. FIG. 1 is a flow chart of a method for generating question and answer data according to some embodiments of the present disclosure. As shown in FIG. 1 , the method can be applied to a server side. The method flow includes the following steps:

Step S101: obtaining a plurality of cut document blocks, wherein each cut document block contains a first preset number of text data.

Optionally, in the disclosed embodiment, the server side obtains a document of question and answer data (i.e., QA) to be extracted (or mined), and cuts the document, such as cutting it into 10 parts in equal proportion, to obtain a plurality of cut document blocks. If the cut is equal proportion, each cut document block contains the same first preset amount of text data, for example, if the document contains 100,000 words of text data in total, after being cut into 10 parts, each cut document block contains the same 10,000 words of text data.

In addition, before the document is segmented, there may be many problems with the text in the acquired document. In this case, these texts need to be preprocessed to obtain the correct text in a unified format. As shown in Table 1, Table 1 records some possible problems with the text and the corresponding solutions:

Table 1

In addition to the problems in the above table, there may be other text problems, such as: misunderstandings caused by grammatical errors and typos in documents when automatically extracting system text. The corresponding solutions can be: error detection and correction, semantic ambiguity elimination, etc. It should be noted that the above content is only an example, and the embodiments of the present disclosure include but are not limited to the above text problems and data preprocessing methods.

Step S102, obtaining target question data corresponding to the cut document block according to the text data in the cut document block and the target question model.

Optionally, the text data in each cut document block is input into a trained target question model to obtain target question data corresponding to the cut document block.

Step S103, obtaining full data, wherein the full data is content information associated with the target question data contained in a complete document composed of a plurality of cut document blocks.

Alternatively, some answers may require context to be fully understood, and simply extracting questions and answers may result in the loss of important context information. This can be achieved by expanding the scope of the context during processing to include context information that is closely related to the question and answer. This allows the model to capture the necessary context information to better understand the question and answer. Context information can also be encoded by using sentence or paragraph representation methods. This approach can help the model capture long-range context information and incorporate it into the processing flow when making predictions.

However, if we only use context to obtain question-answering data, there may be a lack of comprehensiveness. For example, the answer to a question in a document may depend on the content of another document. In this case, full data is needed. The full data is stored in a vector database (a complete document database composed of multiple cut document blocks). After the question is vectorized, it is matched with the data in the vector database. The similarity of the vector is used to filter out the content information most relevant to the target question data, such as 10 pieces of data, and then The target question data and the 10 data fragments related to it are given to the question selection answer strategy related model. After applying the answer strategy, the corresponding target answer data is obtained.

At this time, the full amount of data can be determined in the following ways: 1. Through the Embedding model + vector database, information can be searched and extracted across documents. 2. Document links: If there are clear links between documents, documents can be captured and processed in sequence according to the links to form a contextual understanding. 3. Compound document modeling: Establish a multi-input model and input multiple related documents at the same time for understanding and reasoning. 4. Answer extraction and synthesis: When the answer span is large, answer extraction will be performed first, all possible answer fragments will be extracted, and then answer synthesis will be performed to splice these fragments in an appropriate way. In the extraction stage, multiple answer fragments can be set and each answer fragment can be given a score. Then, in the synthesis stage, the answer with the highest score can be selected, or the answers with higher scores can be reasonably spliced. 5. Paragraph-level processing: When the answer may span multiple paragraphs, the paragraph is used as the processing unit, and the information of multiple paragraphs is considered at the same time to generate the answer. Documents can be cut using sliding windows or other methods to capture a wider range of context.

It can be understood that the full amount of data already contains context information associated with the target question data.

Step S104, obtaining target answer data corresponding to the target question data according to the segmented document blocks, the target question data, the full data and the target answer model.

Optionally, when determining the target answer data corresponding to the target question data of each cut document block, the embodiment of the present disclosure needs to consider the text data of each cut document block, the target question data output by the target question model, and all the full data associated with the target question data, and then input these data that need to be considered into the trained target answer model to obtain the target answer data corresponding to the target question data.

Step S105, generating question and answer data according to the target question data and the target answer data.

Optionally, each cut document block corresponds to a target question data and a target answer data. Then the question-answer data consisting of a target question data and a target answer data is the final question-answer data mined from each cut document block.

In addition, the final question and answer data mined from all the cut document blocks can be used to form a QA library for storing these question and answer data.

In the embodiment of the present disclosure, a plurality of cut document blocks are obtained, wherein each cut document block contains a first preset number of text data; target question data corresponding to the cut document block is obtained according to the text data in the cut document block and the target question model; full data is obtained, wherein the full data is content information associated with the target question data contained in a complete document composed of the plurality of cut document blocks; According to the cut document blocks, target question data, full data and target answer model, target answer data corresponding to the target question data is obtained; according to the target question data and the target answer data, question and answer data is generated. In this way, the disclosed embodiment can efficiently mine high-quality question and answer data from existing documents, achieve the purpose of automation, greatly reduce labor costs and time costs, and solve the problems of low efficiency and poor data quality in manual mining when mining question and answer data in documents in related technologies.

In some optional implementations, before obtaining the target question data corresponding to the cut document block according to the text data in the cut document block and the target question model, the method further includes:

Determine the questioning strategy based on the text data in the cut document block;

Process the text data according to the questioning strategy and extract multiple questioning questions;

Obtaining first scores for multiple questions according to a question scoring model;

Determine a plurality of candidate questions from the plurality of questions according to the first score;

The initial question model is optimized according to the candidate questions to obtain the target question model.

Optionally, in the disclosed embodiment, before generating the trained target question model, the initial question model needs to be continuously optimized to obtain the final target question model. Before optimizing the initial question model, the QA structured data needs to be screened. Among them, the text suitable for extracting QA structured data has the following characteristics:

a. Rich content: The text should contain rich information points and descriptions

b. Clear logic: The text should be well-structured and accurately expressed to ensure that the questions and answers extracted from it are accurate.

c. Factualism and objectivity: The text is factual in nature and helps generate precise and informative questions and answers.

d. Domain and task relevance: The ideal text type should be highly relevant to specific domains and tasks to ensure that the extracted QA data has practical application value.

At this time, according to the above-mentioned text characteristics suitable for extracting QA structured data, the question strategy of each cut document block is determined, so as to extract multiple question questions (hereinafter also referred to as "questions") from each cut document block.

In order to mine high-quality questions and construct relevant answers in the subsequent process, the present embodiment sets up a question scoring model in advance to score the multiple questions obtained. Among them, the scoring criteria of the question scoring model are designed as follows: 1. Difficulty of the question: select challenging questions. 2. Diversity of questions: ensure that the questions come from different fields and knowledge points. 3. Questions 1. Precision: measures whether the question accurately describes a specific concept or fact. 2. Quality of question presentation: ensure that the question has good grammar and clarity. 3. Clarity of question: whether the question is clear and easy to understand. 4. Relevance of question: whether the question is closely related to the given topic or data set. 5. Information content of question: whether the question involves meaningful information. 6. Complexity of question: whether the question is difficult enough to be answered by a simple search.

Multiple questions are input into the question scoring model to obtain a score value for each question. Questions are sorted according to the score value to select high-quality candidate questions. It is understandable that there may be multiple high-quality candidate questions, in which case multiple cycles are required, and a scoring threshold is set at the same time, and only questions above the threshold are retained. In addition, the scoring threshold can be adjusted according to the needs of the embodiment of the present disclosure to obtain questions of different quantities and qualities.

Based on the feedback from the question scoring model, the retained questions are optimized, including modifying the question wording and adding key information.

The selected and optimized candidate questions are used as training samples and input into the initial question model for training. In this way, the initial question model is continuously optimized to improve the question quality. After multiple rounds of iterations, the system will generate a high-quality question set and finally obtain the target question model.

In addition, in the process of training the initial question model and obtaining the target question model, the embodiment of the present disclosure also proposes some optimization directions:

a. Multi-factor scoring: In the design of the question scoring model, in addition to the scoring criteria mentioned above, multiple factors can also be combined to score the questions. For example, the NLP model can be used to perform semantic analysis on the questions, and the relevance, information content, and complexity of the questions can be combined to give a comprehensive score.

b. Adaptive threshold: In the question screening stage, an adaptive threshold strategy will be used to dynamically adjust the scoring threshold. For example, the scoring threshold is dynamically adjusted based on factors such as the number of questions, question difficulty, and question category. This helps to more accurately screen out high-quality questions and improve the quality of the question library.

c. Balanced sample weights: In the return training phase, try to use the method of balanced sample weights to avoid model overfitting. Different weights are assigned to questions based on factors such as the importance and difficulty of the questions, balancing the impact of different types of questions in the training process.

d. Introducing expert knowledge: In the problem optimization stage, we will try to introduce the knowledge of domain experts and optimize the problem through expert feedback. Experts will correct and modify the problem to improve the quality of the problem. At the same time, based on the feedback from experts, the problem scoring model can be further optimized.

e. Cross-validation: During the question collection and preprocessing phase, cross-validation is used to increase the question pool. For example, when dealing with different datasets on the same topic, a subset of questions can be extracted from each dataset and then scored and filtered. This approach can provide a wider set of questions, which helps improve the quality of the question bank.

In the disclosed embodiment, high-quality questions are screened out through the question scoring model, so that the initial question model can continuously adapt to the mining strategy and style of a specific enterprise or industry, and improve the accuracy of the trained target question model. At the same time, relevant strategies and algorithms are used for dynamic optimization, so as to learn and improve in the continuous generation of high-quality and authentic synthetic data.

In some optional implementations, determining a questioning strategy based on text data in the segmented document block includes:

Determine the text type of the text data;

Determine the corresponding questioning strategy based on the text type.

Optionally, feature analysis of the text type of the text data is performed. The following text types are very suitable for extracting QA structured data:

a. Educational materials: Textbooks, handouts, tutorials, etc. contain rich knowledge and explanatory information.

b. Technical documentation: Product manuals, API documentation, development guides, etc. provide detailed technical descriptions and operating methods.

c. Research papers: Research texts such as academic papers and industry reports usually contain rich analysis and conclusions, especially the abstract, which often describes the research questions and conclusions.

d. Regulations and policy documents: Official regulations, policy documents and legal documents are usually well-structured and factual.

e. News reports and articles: They contain a large amount of descriptions of facts, events, and opinions, which helps to extract relevant QA data such as current affairs.

f. Q&A communities and forums: The Q&A and discussions on the platform usually already have clear questions and answers, which can be directly extracted to build a QA library.

g. Manuals and Guides: These documents usually contain specific questions and corresponding answers, such as user guides, FAQs, product manuals, etc.

According to the above text types, the corresponding questioning strategy is determined, as shown in Table 2 (wherein Table 2 includes both the question extraction strategy and the answer extraction strategy corresponding to the question):

Table 2

Table 2 only determines the corresponding questioning strategies and answer extraction strategies for educational materials, such as textbooks, handouts, tutorials and other text types; technical documents, such as product manuals, API documents, development guides, and research papers, such as academic papers, industry reports, etc., and regulations and policy documents, news reports and articles also obtain corresponding questioning strategies and answer extraction strategies based on the current text type. Here, the embodiments of the present disclosure will not be repeated.

When determining the corresponding questioning strategy based on the text type, there are some auxiliary strategies:

a. Use different question models: Depending on the type of text, different question models may be needed to extract questions in a targeted manner. For example, when processing technical documents, use a pre-trained model for the technical field; when processing regulatory documents, use a pre-trained model for the legal field.

b. Dynamically adjust the questioning strategy: Dynamically adjust the questioning strategy based on the quality, domain, and background knowledge of the text. For example, adjust the threshold for keyword extraction, use different entity types and relationship types, etc.

c. Increase interpretability: Maintain interpretability during the question generation process so that users can track and understand the question generation strategy. In order to improve the interpretability of the question generation process, the question generation strategy is visualized, the reasons for generating questions are provided, and the degree of attention paid by the model to different parts of the text is given.

d. Adaptive strategy adjustment: When processing a large amount of text, the model may need to automatically adapt to the characteristics and difficulty of different texts. To this end, develop adaptive questioning strategies, such as dynamically adjusting the questioning model according to the complexity and domain of the text to achieve higher quality question mining.

e. User interaction: In order to improve the effectiveness and flexibility of the questioning strategy, user interaction is introduced to enable users to participate in the adjustment of the questioning strategy, provide feedback and receive correction suggestions. For example, after the question is generated, users are asked to evaluate it and the questioning strategy is adjusted according to user feedback.

f. Combining unsupervised and supervised methods: In the process of selecting question strategies, combining unsupervised and supervised methods will improve the robustness of the model. Unsupervised methods will help extract the latent structure in the text, while supervised methods will generate more accurate questions based on existing annotated data.

g. Weigh the difficulty and diversity of questions: When choosing a questioning strategy, you need to balance the difficulty and diversity of questions. Simple and direct questions may be easier for users to understand, but may not cover the entire content of the text; while complex and diverse questions will cover more dimensions, but may pose challenges to user understanding. When building a QA database, it is recommended to consider the difficulty and diversity of questions comprehensively to avoid over-emphasizing one type of question.

In the embodiments of the present disclosure, questioning strategies, answering strategies, etc. are flexibly selected and adjusted according to different texts and industry requirements to make them more suitable for actual application scenarios.

In some optional implementations, before obtaining target answer data corresponding to the target question data according to the segmented document blocks, the target question data, the full data, and the target answer model, the method further includes:

Determine the answer strategy based on the text data in the cut document block and the target question data;

Processing the text data according to the answer strategy to obtain multiple answer data;

Obtaining a second score for the plurality of answer data according to the answer scoring model;

determining a plurality of candidate answer data from the plurality of answer data according to the second score;

The initial answer model is optimized according to the candidate answer data to obtain the target answer model.

Optionally, based on the text data and target question data in the cut document block, the answer strategy can be determined (see Table 2), and then multiple answer data can be obtained. Collecting these answer data requires a series of preprocessing of the answers, including cleaning irrelevant words, correcting spelling errors, etc., to optimize the expression of the answers.

Set up an answer scoring model in advance, score the generated answers, retain high-quality answers, and feed the scoring information back to the trained initial answer model to optimize the performance of the initial answer model. At this time, you can choose a common natural language processing answer scoring model. The goal of the answer scoring model is to sort the quality of multiple answer data according to specific criteria. The scoring criteria include: Answer accuracy: evaluate whether the content of the answer is accurate and how well it matches the question. Answer completeness: evaluate whether the answer has obtained complete information and whether it can fully answer the question. Answer expression quality: check the language expression of the answer, including grammatical accuracy, comprehensibility, etc.

Then use the answer scoring model to score each answer, which can reflect the overall quality of the answer.

The second scores of multiple answers are sorted, a threshold is set, and multiple candidate answer data with scores exceeding the threshold are retained. The multiple candidate answer data after screening and optimization are used as training samples and input into the initial answer model for further training, cyclic iteration, and optimization of the initial answer model performance.

Iterate the above steps repeatedly as needed. During the iteration process, the system will continue to generate high-quality answer sets to prepare for the construction of the QA library and subsequent processes, and then obtain the final trained target answer model. The target answer data obtained by the target answer model should be accurate, complete and clearly expressed.

There are some auxiliary strategies during the training of the initial answer model:

a. Multi-task learning: During the training process, a multi-task learning strategy is used to optimize the accuracy, completeness, and presentation quality of the answer at the same time. This is achieved by sharing hidden layer parameters and using soft parameter sharing between tasks.

b. Use reinforcement learning strategies: Introduce reinforcement learning strategies, such as using Actor-Critic algorithms or Deep Q-Learning to determine the best answer. Further improve the quality of answer generation.

c. Integrate with knowledge graph: In the answer scoring model, reference external knowledge graph to enhance the accuracy and trust of the answer. By leveraging the knowledge graph, further verify and support the correctness of the answer and provide additional references when possible.

d. Real-time fine-tuning: For real-time application scenarios, a real-time fine-tuning strategy is adopted, that is, the answer scoring model is updated in real time according to the newly generated answers. This will enable the model to adapt to the changing data distribution and improve the quality of the answers.

e. Explainability: Add explainability mechanisms to the answer scoring model, such as attention mechanisms or model sensitivity analysis, to ensure the quality of the answers and provide a basis for analysis and adjustment.

In the disclosed embodiments, iterative optimization and the use of advanced machine learning methods can lay a solid foundation for building a high-quality question-answer library. At the same time, the questioning strategy, answering strategy, etc. can be flexibly selected and adjusted to make them more suitable for actual application scenarios.

In some optional implementations, determining an answer strategy based on text data and target question data in the segmented document block includes:

Determine the text type of the text data;

Determine the answer strategy based on the text type and target question data.

Optionally, as shown in Table 2, an answer strategy can be determined based on the text type and the target question data extraction strategy. Then, according to the answer strategy, multiple answers related to the target question data can be obtained, and then these answers are scored to obtain a high-quality answer set consisting of multiple candidate answer data, and then the initial answer model is continuously iterated and optimized to obtain a trained target answer model. It should be explained that the number of answers output by the target answer model should only be one target answer data.

In some optional implementations, obtaining target answer data corresponding to the target question data according to the segmented document blocks, the target question data, the full amount of data, and the target answer model includes:

According to the cut document blocks, the target question data, the full amount of data and the target answer model, multiple answer data are obtained;

Split the answer data into the smallest unit to obtain multiple target fields;

comparing the description contents of the target field in each second preset number of answer data;

The description content corresponding to the answer data with the second highest score is retained;

Integrate the description content to obtain the target answer data.

Optionally, for each question, n answers may be generated. These answers may be generated by different initial answer models, or by the same initial answer model under slightly different input conditions. Break the screened answers into smaller information units (i.e. target fields). These units may be sentences or phrases describing specific facts. Compare each two information units, and if they describe the same facts or details, only keep the answer data with the higher score. If they describe different facts or details, both are retained.

The selected information units are combined to form a new answer. At the same time, auxiliary strategies are used to determine the order of the information units, which include but are not limited to their order in the original answer, or according to a certain logical order. Finally, the reconstructed answer is post-processed, including checking grammar, adjusting word order, and correcting spelling errors, etc., to obtain the final target answer data.

In the disclosed embodiment, by synthesizing multiple answer data, the quality and comprehensiveness of the answers are efficiently guaranteed, and the quality of QA library construction is greatly improved compared to a single answer.

In some optional implementations, after obtaining the question-answer data about the cut document blocks, the method further includes:

The question and answer data is scored using the question and answer data scoring model to obtain a third score for the question and answer data;

The question and answer data is cleaned according to the third score to obtain target question and answer data that meets the preset requirements;

The data sample is expanded according to the target question and answer data to obtain a third preset number of question and answer data.

Optionally, the embodiment of the present disclosure sets up a question-answer data scoring model in advance to check the quality of the obtained QA. Among them, the core indicators of QA data are:

a. Completeness of questions and answers: Check whether the extracted questions and answers are complete, without being truncated or missing key parts. Ensure that the answers provided are meaningful to users.

b. Consistency: Are similar or repeated questions and answers extracted from different parts or different documents consistent? Ensure that the answers provided are consistent across documents or document parts.

c. Relevance: Are the extracted questions and answers relevant to the context or topic? Make sure the QA data is relevant for a specific query or task.

d. Readability: Are the questions and answers easy to read and understand? Make sure the final model can understand the extracted content well.

Based on the above core indicators, the question and answer data of each cut document block is scored to obtain the third score of the question and answer data. The scoring criteria are:

Rule-based checking: Use regular expressions or other rules to check the format of questions and answers. Automatically check the completeness of answers, such as ensuring that answers are not truncated.

Sentence Embedding Comparison: Use a pre-trained sentence embedding model to convert questions and answers into vectors. Compare the embeddings of questions and answers to evaluate their relevance or similarity.

Feedback loop: Automatically generate questions using language model, validate answers against documents. Compare automatically generated questions with actual extracted questions to assess their quality.

Diversity and duplication check: Use Jaccard similarity, cosine similarity or other text similarity methods to check the duplication between the extracted QA data.

Statistical analysis: Automatically count the frequency of certain keywords or words to determine if there are too many repeated or missing topics. Use TF-IDF (term frequency–inverse document frequency, Common weighting techniques used in information retrieval and data mining) or other techniques to identify unusual or rare words that appear in questions or answers.

Contextual consistency: Use a pre-trained language model to evaluate the consistency of the answer in context, ensuring that the answer is relevant to the text around it.

Error analysis: Use automated tools, such as grammar checkers or text classifiers, to identify possible text errors or inconsistencies. Also design an automated system to continuously fine-tune and optimize the data extraction model based on feedback and errors.

Reference dataset comparison: Comparison with a seed dataset of verified quality, using an automated scoring system to assess the quality of the QA data.

Afterwards, the disclosed embodiment will clean the question and answer data according to the third score, including removing low-quality question and answer data, removing sensitive privacy data, etc., to obtain target question and answer data that meets the preset requirements to ensure the high quality of the question and answer data. In addition, the data sample is expanded according to the target question and answer data obtained after cleaning to ensure the sufficiency of the data sample of high-quality QA data.

At the same time, after generating multiple QA data and obtaining the QA library, the embodiment of the present disclosure also proposes some auxiliary strategies to support the application of the QA library in more scenarios.

a. Incremental update strategy: As the business develops, the enterprise may generate new document data. At this time, an incremental update mechanism can be designed to regularly update and optimize the existing QA library to maintain its timeliness and relevance.

b. User feedback integration: If the QA library will be used in scenarios such as customer support, user feedback on answers to questions can be collected to continuously optimize and update the QA library. For example, users can provide feedback on the relevance, accuracy, and ease of understanding of answers, and the system will continuously optimize based on these feedbacks.

c. Data hierarchical annotation: The questions and answers in the QA library are annotated with hierarchical annotations, such as classification, difficulty level, etc., to facilitate more accurate matching and display of relevant information in user queries and improve user experience.

d. Introducing topic models: Use topic models to cluster questions so as to maintain a good diversity of questions in the QA database. This helps avoid excessive concentration on a single topic and ensures that the QA database covers multiple fields and knowledge points.

e. Visual data evaluation: Provides visualization tools to present the process and results of QA scoring screening. It can display key indicators such as the distribution of questions and answers, quality score change trends, etc., to help companies gain insight into the overall situation of data quality.

f. Integrate external knowledge base: Combine the QA library with external knowledge base to expand the coverage of questions and answers. coverage and improve adaptability to user queries.

In the embodiments of the present disclosure, strategies such as multi-task learning and reinforcement learning are applied to improve data quality. At the same time, the robustness of the model is improved through methods such as model integration and data enhancement, and the impact of uncertainty on model performance is reduced.

In some optional implementations, the cut document block includes text data after image recognition.

Optionally, in the disclosed embodiment, after the server side obtains the document of the question and answer data (i.e., QA) to be extracted (or mined), it can first determine whether the document is an image document or a non-image document. If it is an image document, it is necessary to identify the image document, identify the corresponding text content, and then cut the document, such as cutting it into 10 parts in equal proportions, etc., to obtain multiple cut document blocks, so that each cut document block includes the text data after the image recognition.

As shown in FIG. 2 , FIG. 2 is a schematic diagram of a complete process of a method for generating question-answer data according to some embodiments of the present disclosure. The specific process is as follows:

Obtain unstructured data, such as industry books, articles, papers, etc.;

If the unstructured data is a picture, the picture is processed, and visual encoder feature extraction is used. Then, OCR technology is used to extract text from the picture, and the context information of the extracted text is obtained at the same time. The extracted text and context information are used for text preprocessing and document segmentation, such as segmenting into n document blocks of less than 10k. If the unstructured data is not a picture, the text in the document is preprocessed and the document is segmented into n document blocks of less than 10k. The questioning strategy is selected according to the text type of the text in the document block (such as regular documents, special professional documents, news reports, literary works, etc. in Figure 2). A variety of questioning strategies and questioning models (i.e., initial questioning models) are applied to extract n questions, and the n questions are input into the question scoring model for scoring. This cycle is repeated N times to screen out high-quality questions to optimize the questioning model, and finally an optimal question is output to input into the QA library and determine the answer strategy module.

When determining the answer strategy, the answer strategy is determined based on the text type in each document block (such as regular documents, special professional documents, news reports, literary works, etc. in Figure 2) and the input best question. Then, multiple answer strategies, each document block, and the full amount of data associated with the best question obtained by vector processing of unstructured data are input into the answer model to obtain multiple answer data. Then, the multiple answer data are input into the answer scoring model. After N rounds of cycles, high-quality answers are screened out to learn and optimize the answer model; n answers are integrated to form an optimal answer (that is, the best answer), which is input into the QA library. It can be seen that the QA library stores QA data corresponding to one best answer for one best question.

Then, the QA scoring model is used to score each QA data in the QA library for quality inspection. At the same time, the QA data can be cleaned to remove low-quality data and sensitive privacy data. The cleaned QA data can be expanded. Finally, all models can be fine-tuned based on the expanded and cleaned QA data.

In the above embodiments, data processing is completed locally, and enterprise data will not be exposed to third parties, which greatly increases data security. Through multiple rounds of training, including the initial question model, the initial answer model, the question scoring model, and the answer scoring model, these models are continuously adapted to the mining strategy and style of a specific enterprise or industry, thereby achieving knowledge transfer and building an enterprise-specific mining model. At the same time, a trade-off is made based on hardware resources and performance requirements, and the appropriate model architecture and parameter settings are selected, which can reduce the demand for computing resources without affecting the quality of synthetic data.

In this embodiment, a device for generating question and answer data is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and/or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

This embodiment provides a device for generating question-answer data, as shown in FIG3 , including:

A first acquisition module 301 is used to acquire a plurality of cut document blocks, wherein each cut document block contains a first preset number of text data;

The first obtaining module 302 is used to obtain target question data corresponding to the cut document block according to the text data in the cut document block and the target question model;

The second acquisition module 303 is used to acquire full data, wherein the full data is content information associated with the target question data contained in a complete document composed of a plurality of cut document blocks;

The second obtaining module 304 is used to obtain target answer data corresponding to the target question data according to the cut document blocks, the target question data, the full amount of data and the target answer model;

The third obtaining module 305 is used to generate question-answer data according to the target question data and the target answer data.

In some optional embodiments, the device further comprises:

A first determination module is used to determine a questioning strategy based on the text data in the cut document block before obtaining target question data corresponding to the cut document block based on the text data in the cut document block and the target question model;

An extraction module, used to process text data according to a questioning strategy and extract multiple questioning questions;

A third acquisition module is used to obtain first scores for multiple questions according to the question scoring model;

A second determination module is used to determine a plurality of candidate questions from a plurality of questions according to the first score;

The fourth obtaining module is used to optimize the initial question model according to the candidate questions to obtain the target question model.

In some optional implementations, the first determining module includes:

A first determining unit, used to determine the text type of the text data;

The second determining unit is used to determine a corresponding questioning strategy according to the text type.

In some optional embodiments, the device further comprises:

A third determination module is used to determine an answer strategy based on the text data in the cut document block and the target question data before obtaining the target answer data corresponding to the target question data based on the cut document block, the target question data, the full amount of data and the target answer model;

The fifth obtaining module is used to process the text data according to the answer strategy to obtain a plurality of answer data;

A fourth acquisition module, used for acquiring a second score for the plurality of answer data according to the answer scoring model;

a fourth determination module, configured to determine a plurality of candidate answer data from the plurality of answer data according to the second score;

The sixth obtaining module is used to optimize the initial answer model according to the candidate answer data to obtain the target answer model.

In some optional implementations, the third determining module includes:

A third determining unit, used to determine the text type of the text data;

The fourth determination unit is used to determine the answer strategy according to the text type and the target question data.

In some optional implementations, the second obtaining module 304 includes:

An obtaining unit, used for obtaining a plurality of answer data according to the cut document blocks, the target question data, the full amount of data and the target answer model;

Splitting unit is used to split the answer data into the smallest unit to obtain multiple target fields;

A comparing unit, used for comparing the description content of the target field in each second preset number of answer data;

A retaining unit, used for retaining the description content corresponding to the answer data with the second highest score;

The integration unit is used to integrate the description content to obtain target answer data.

In some optional embodiments, the device further comprises:

A scoring module, for scoring the question and answer data using a question and answer data scoring model after obtaining the question and answer data about the cut document block, to obtain a third score for the question and answer data;

A cleaning module, used to clean the question and answer data according to the third score to obtain target question and answer data that meets preset requirements;

The expansion module is used to expand the data sample according to the target question and answer data to obtain a third preset number of question and answer data.

In some optional implementations, the cut document block includes text data after image recognition.

The question and answer data generating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and/or other devices that can provide the above functions.

The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

The embodiment of the present disclosure also provides a computer device having the device for generating question and answer data shown in FIG. 3 above.

Please refer to Figure 4, which is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present disclosure. As shown in Figure 4, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input/output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and/or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). In Figure 4, a processor 10 is taken as an example.

The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

The memory 20 stores instructions executable by at least one processor 10 to enable at least one The processor 10 executes the method shown in the above embodiment.

The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims (11)

A method for generating question-answering data, comprising: Acquire a plurality of cut document blocks, wherein each of the cut document blocks contains a first preset number of text data; Obtaining target question data corresponding to the cut document block according to the text data in the cut document block and the target question model; Acquire full data, wherein the full data is content information associated with the target question data and contained in a complete document composed of a plurality of the cut document blocks; Obtaining target answer data corresponding to the target question data according to the cut document blocks, the target question data, the full amount of data and the target answer model; The question and answer data is generated according to the target question data and the target answer data. According to the method of claim 1, before obtaining the target question data corresponding to the cut document block according to the text data in the cut document block and the target question model, the method further comprises: Determining a questioning strategy based on the text data in the cut document block; Processing the text data according to the questioning strategy to extract multiple questions; Obtaining first scores for the plurality of questions according to a question scoring model; Determining a plurality of candidate questions from the plurality of questions according to the first scores; The initial question model is optimized according to the candidate question to obtain the target question model. The method according to claim 2, wherein determining the questioning strategy based on the text data in the cut document block comprises: Determining a text type of the text data; The corresponding questioning strategy is determined according to the text type. The method according to claim 1, wherein, before obtaining the target answer data corresponding to the target question data according to the cut document blocks, the target question data, the full amount of data and the target answer model, the method further comprises: Determining an answer strategy based on the text data in the cut document block and the target question data; Processing the text data according to the answer strategy to obtain a plurality of answer data; Obtaining a second score for the plurality of answer data according to the answer score model; determining a plurality of candidate answer data from the plurality of answer data according to the second score; The initial answer model is optimized according to the candidate answer data to obtain the target answer model. The method according to claim 4, wherein the step of determining the answer strategy based on the text data in the cut document block and the target question data comprises: Determining a text type of the text data; The answer strategy is determined according to the text type and the target question data. The method according to claim 4, wherein the step of obtaining target answer data corresponding to the target question data according to the cut document blocks, the target question data, the full data, and the target answer model comprises: Obtaining a plurality of answer data according to the cut document blocks, the target question data, the full amount of data and the target answer model; Splitting the answer data into minimum units to obtain multiple target fields; comparing the description contents of the target fields in each second preset number of the answer data; retaining the description content corresponding to the second highest-scoring answer data; The description contents are integrated to obtain the target answer data. The method according to any one of claims 1 to 6, wherein after obtaining the question-answer data about the cut document block, the method further comprises: Scoring the question and answer data using a question and answer data scoring model to obtain a third score for the question and answer data; Performing data cleaning on the question and answer data according to the third score to obtain target question and answer data that meets preset requirements; Data samples are expanded according to the target question and answer data to obtain a third preset number of question and answer data. The method according to any one of claims 1 to 6, wherein the cut document block includes text data after image recognition. A device for generating question and answer data, comprising: The first acquisition module is used to acquire a plurality of cut document blocks, wherein each of the cut document blocks contains Contains a first preset number of text data; A first obtaining module is used to obtain target question data corresponding to the cut document block according to the text data in the cut document block and the target question model; A second acquisition module is used to acquire full data, wherein the full data is content information associated with the target question data contained in a complete document composed of a plurality of the cut document blocks; A second obtaining module is used to obtain target answer data corresponding to the target question data according to the cut document blocks, the target question data, the full amount of data and the target answer model; The third obtaining module is used to generate the question and answer data according to the target question data and the target answer data. A computer device comprising: The memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for generating question and answer data of any one of claims 1 to 7 by executing the computer instructions. A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for generating question and answer data according to any one of claims 1 to 7.