KR20130021944A - Method and apparatus for descriptive question answering - Google Patents

Abstract

The present invention relates to a descriptive question and answer technique, which extracts key keywords by analyzing a user's natural language question, adds clue words to improve search performance, and searches related documents using a search engine. Search for snippets retrieved through search engines, find and weight documents that contain answers to descriptive questions, rerank the documents with higher weights to rank higher, and find the correct answers within the reranked search results. It is characterized by providing the user with the surrounding information and the correct answer. According to the present invention, it is possible to obtain information desired by a user quickly and simply by automatically presenting a correct answer to a user's narrative question. Thus, the user can handle most of the possible questions that the user is asking by using a familiar natural language or by using existing short-answer questions.

Description

METHOD AND APPARATUS FOR DESCRIPTIVE QUESTION ANSWERING}

The present invention relates to a question answering technique based on a question answering system, and more particularly, to a method and an apparatus for describing a question answering method suitable for receiving a descriptive question in a natural language form from a user and automatically extracting and providing a correct answer in a sentence form. .

In general, the information retrieval system refers to a system that can easily and quickly retrieve data containing the desired information for a large amount of documents, media, and the like. Documents to be searched for information may be web documents or large documents used by a company.

In this case, the information search receives a combination of keywords such as 'information' and 'search' from the user as a query, and provides a service that finds and provides the most relevant document in the system to the user. The user searches for a document that may have information to find through an information retrieval system, and obtains desired information by reading the retrieved document.

The user's query usually consists of one to five words that can represent the user's information needs. However, it is difficult to completely express the user's desire for information using a small number of words, and thus, it is difficult for the user to obtain satisfactory results. In addition, since the user directly finds the desired information in the searched document, and if there is no desired information in the searched document, the user has to go through a re-searching process, it may take a long time to acquire the information.

The question and answer system is a system that can secure the disadvantages of such information retrieval system. In the Q & A system, instead of a keyword-level query, a natural language query is input. The query in the form of natural language is a form of query used in actual language life such as 'Who is the 14th president of the United States?', Which has the advantage of being familiar to users and expressing their information needs more specifically.

The question and answer system analyzes the query to find out what information the user wants, searches for documents such as web documents or encyclopedias, and provides the user with a Franklin Pierce answer.

That is, while the information retrieval system only searches for documents related to the query, the question and answer system provides an answer to the user's question directly, so that the user can shorten the process of reading the retrieved document and finding the correct answer.

As a conventional technology that provides a service as an intermediate step between such a question and answer and information retrieval, (Patent Literature 1) analyzes a question, extracts a main keyword, analyzes a type, and then retrieves a document using the extracted keyword, Analyzes the summary by question type and presents it to the user.

There is also a conventional technique of analyzing a question for a question and answer, extracting a keyword, searching for a keyword, and finding a correct answer in the searched document. In the case of (Patent Document 2), a method of extracting correct candidates from the searched documents and presenting them to the user was used.

Patent Document 1: U.S. Patent No. 7587420 (August 2009. Announcement) Patent Document 2: US Patent Publication No. 2005/0114327 (published May 26, 2005)

In the question-answering system of (Patent Documents 1 and 2) according to the prior art operating as described above, since the process can be processed only when the question input by the user is a short answer form (Factoid), for example, There is a problem that cannot be processed for a descriptive query such as?

Accordingly, an embodiment of the present invention may provide a method and apparatus for descriptive question and answer that may receive a descriptive question in natural language form from a user and automatically extract a correct answer in sentence form.

In addition, the embodiment of the present invention, after analyzing the user's natural language questions to extract the key keywords, add the clue words for improving the search performance to the key keywords, search for related documents using a search engine, and search engine Find and weight documents that contain answers to descriptive questions for the snippets retrieved through them, rerank the documents with higher weights to rank higher, and then search for the correct answers within the reranked search results. A narrative question answering method and apparatus which can distinguish surrounding information from correct answers and provide them to a user can be provided.

According to an exemplary embodiment of the present invention, a narrative question answering method receives a narrative question, extracts a key keyword of the question through analysis, classifies the question type according to the intention of the question, and then adds a clue word for each type. Searching related documents in the linked web site document or the previously collected document based on the extracted key keywords and clue words, and reranking the documents including the correct answer among the searched documents so that they are ranked in a higher rank. It may include.

The adding process may determine at least one question type among definitions, causes, methods, objectives, and origins, and may remove words other than the message and the verb through morphological analysis.

The re-ranking process may be performed by retrieving a sentence of a document including the key keyword, considering a word included in the sentence, or considering a distance between the word included in the sentence and the key keyword. It may include a process of scoring the probability of including the descriptive correct answer for each snippet of the document based on the machine learning algorithm, and the process of reranking the high scores in the order of priority based on the score for each snippet.

In the re-ranking process, a pattern may be constructed for each question type, and weights assigned to documents matching the constructed pattern may be re-ranked in the order of the weights.

The narrative question answering method may further include extracting a correct answer sentence from the reranked document, outputting the extracted correct answer sentence, or classifying the correct answer sentence with surrounding information other than the correct answer sentence. Can be.

The narrative question answering apparatus according to an embodiment of the present invention includes an input unit that receives a narrative question, extracts key keywords of the question through analysis of the narrative question, classifies question types according to the intention of the question, and then types A question analysis engine unit for adding a clue word to a web search engine, a web search engine unit for searching a related document in a linked web site document or a previously collected document based on the extracted key keywords and clue words, and a correct answer among the searched documents It may include a re-ranking engine unit for re-ranking the included document to be in a higher rank.

The question analysis engine unit may determine at least one question type among definition, cause, method, purpose, and origin, and may remove words other than the message and the verb through morpheme analysis.

The reranking engine unit searches for a sentence of a document including the key keyword, considers a word included in the sentence, or considers each document in consideration of the distance between the word included in the sentence and the key keyword. The probability of including descriptive correct answers for each snippet of is scored based on the machine learning algorithm, and the high scores can be reranked based on the score for each snippet.

In addition, the reranking engine unit may construct a pattern for each question type, assign a weight assigned to a document matching the constructed pattern, and rerank the assigned weights in order of high order.

The narrative question answering method further includes a correct answer extracting unit extracting a correct answer sentence from the reranked document, outputting the extracted correct answer sentence, or classifying the correct answer sentence with surrounding information other than the correct answer sentence. can do.

According to the method and apparatus for a descriptive query response according to the embodiment of the present invention as described above, there are one or more effects as follows.

According to the method and apparatus for a narrative question answering method according to an embodiment of the present invention, a correct answer is automatically presented to a user's narrative question so that information desired by a user can be obtained quickly and simply. Therefore, it is effective for the user to process most of the possible questions that the user asks through the familiar natural language or through the existing short answer question and answer.

1 is a block diagram showing the structure of a descriptive query response device according to an embodiment of the present invention;
2 illustrates an interface for ranking and providing snippets as a result of an information search engine;
3 is a flowchart illustrating an operation procedure of a narrative query response device according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions in the embodiments of the present invention, which may vary depending on the intention of the user, the intention or the custom of the operator. Therefore, the definition should be based on the contents throughout this specification.

Each block of the accompanying block diagrams and combinations of steps of the flowchart may be performed by computer program instructions. These computer program instructions may be loaded into a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus so that the instructions, which may be executed by a processor of a computer or other programmable data processing apparatus, And means for performing the functions described in each step are created. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of each step of the block diagram. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions noted in the blocks or steps may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently or the blocks or steps may sometimes be performed in the reverse order, depending on the functionality involved.

In an embodiment of the present invention, a descriptive question in a natural language form is input from a user and automatically extracts a correct answer in a sentence form, thereby extracting a key keyword by analyzing a user's natural language question and improving search performance on the core keyword. After adding the clue words for the search, search the relevant documents using the search engine, search for the snippets searched through the search engine containing the answers to the descriptive questions, and weighted by weighting After the document is reranked to a high rank, the surrounding information of the correct answer and the correct answer are provided to the user in the reranked search result.

In other words, by analyzing the query of the narrative user and adding the key keyword and the clue keyword, the search performance can be improved, and the user satisfaction can be improved by finding the correct answer in the searched snippet and dividing the surrounding information with the correct answer.

In order to improve search performance, clue words can be added for each type and searched, and the searched documents can be re-ranked using machine learning and pattern matching without using them immediately. And when the correct answer is extracted from the reranked document, the user's satisfaction can be improved by first presenting the correct answer from the high rank to the user.

The present invention may provide the user with the correct answer to the descriptive question, such as 'what is the cause of the earthquake?' In the embodiment of the present invention, by using the existing information retrieval engine for web documents and searching for related documents and extracting the correct answer for the retrieved documents, all general narrative questions that are not domain dependent can be processed. In this case, the narrative question has a feature that the correct answer is provided in a sentence form, unlike the short answer question.

Table 1 below is an example of a descriptive question and a short answer question.

Question Question Type answer What is the cause of the earthquake? Narrative
(cause) The direct cause of the earthquake is the movement of the plates in the rock zone, a hard layer about 100 km thick from the earth's surface. What is Alice syndrome? Narrative
(Justice) Alice's syndrome is a condition in which objects appear large, small, or distorted, usually accompanied by migraine headaches. Please tell me how to get ePassport Narrative
(Way) The issuance method is to receive and review the passport application form and send the application form online or in a pouch to the Ministry of Foreign Affairs. IU's real name? Short answer Lee, Ji - Eun What is the representative destination of Jeju Island? Short answer Seopjikoji How old is Lee? Short answer 71 years old

As shown in Table 1, a question of a descriptive question type may be received as a user input, and a correct answer may be provided to the user in the same form as the information in the 'correct' column of the table.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the structure of a descriptive query response device according to an embodiment of the present invention.

Referring to FIG. 1, the narrative question answering apparatus 100 is for receiving a user's question as an input and providing a correct answer to a corresponding query as a final result. The input unit 110, the question analysis engine unit 120, and a web search are provided. The engine unit 130, the reranking engine unit 140, and the correct answer extractor 150 may be included.

The narrative question answering apparatus 100 is connected to the Internet network or interworked with a plurality of websites in order to perform a narrative question answer based on a large document used in a web document, a company, or the like. Descriptive query responses can be performed based on web document information on the website.

In detail, the input unit 110 receives a narrative type, that is, a natural language question from a user, and may receive a natural language question through various client terminals such as a user's mobile phone, a smartphone, a notebook computer, a personal computer, and the like. When implemented using a keypad or a touch screen method, a natural language question may be input by a user directly pressing a keypad or touching a touch screen.

The input unit 110 may transmit the input natural language question to the question analysis engine unit 120 as a user input signal.

The question analysis engine unit 120 may perform a function of extracting a key keyword for searching from the natural language question information received from the input unit 110 and determining the type of the question. The question type is divided according to the intention of the question, and in the case of the descriptive question, may include definition, cause, method, purpose, origin, and the like, and at least one question type may be selected according to the determination result.

After determining the question type, stemming analysis can remove words other than words and verbs, and then add clue words appropriate to the question type to improve the search performance. That is, at least one clue word preset for each determined question type may be added.

However, if the question already contains at least one clue word, the clue word may not be additionally included. And when removing words other than spoken and used words, words that appear too often can be removed.

The following (Table 2) shows the type of question, keyword extraction results.

Question Extraction Keyword Clue word What is the cause of the earthquake? Earthquake Reason What is the fastest growing cause of Facebook? Facebook, booming, causes Reason Please tell me how to get an e-passport EPassport, issuance, method Way What is Ellis Syndrome? Ellis syndrome Definition, meaning, meaning What is the placebo effect? Placebo effect Definition, meaning, meaning

The web search engine unit 130 receives the result information of the question analysis engine unit 120, uses the extracted keyword and the clue word as a query, and searches related documents using an information search engine. have. The information search engine can be any search engine (eg, Naver search, Google search, etc.) that targets the web and large-capacity collected documents. The information search engine uses a snippet, which is a summary of the document, as shown in FIG. Can be provided as a search result.

The reranking engine unit 140 may reorder and sort the search results that are the result of the web search engine unit 130 so that the snippet containing the correct answer of the question is placed in a higher rank. The reranked result may be used by the correct answer extractor 150, and may provide the user with the correct answer extracted from the higher priority.

For this reranking, there are methods using machine learning and methods using pattern. In the method using the machine learning, the probability of including the descriptive correct answer for each snippet is scored based on the machine learning algorithm such as SVM (Support Vector Machine) using three qualities and exported as a result. The reranking engine unit 140 may rerank based on the corresponding scores so that the high score snippet comes out with a high rank.

The first of the three qualities to be used is to extract words from sentences such as key keywords extracted from the question analysis engine unit 120. This may find a sentence including a key keyword in a previously searched document, and extract the words appearing in the sentences to determine the weight of the extracted words. The weights of the extracted words may be determined based on the inclusion of a word similar to or derived from a key keyword or a clue word, or a preset word for each determined question type.

 Secondly, the distance information between the words extracted from the first qualities and the key keywords extracted from the question analysis engine unit 120 is provided. This means that the narrower the distance between the key keywords and the words selected from the first qualities, the higher the score.

Lastly, as the relevance ranking provided by the web search engine unit 130 for the snippet, the related ranking format (case sensitive, trusted site ranking, number of key keywords included in each information search engine) Star, location, etc.).

Equation 1 is for calculating a weight in the SVM machine learning method applied in the embodiment of the present invention.

Where x represents the retrieved snippet as a vector using its features. The larger the value of U (x), the higher the probability that the snippet will contain the correct answer. Machine learning learns the appropriate weight value w for each vector over t lessons. Accordingly, the reranking engine unit 140 may rerank the document having a higher value to rank higher using the scoring value.

On the other hand, in the method using a pattern, a pattern is constructed for each descriptive type (for example, a definition type, a method type, a weaning type, etc.), and weights are assigned for each pattern to give a high weight to the snippet in which the high weight pattern appears, How to rank.

Table 3 below shows examples of patterns and weights for each type, which are weighted if there is a matching snippet, and assigned the highest weight for snippets satisfying the various patterns, or assigned to each pattern. A method of summing weights may be used.

pattern type weight [Keyword] is * Definition 0.7 [Keyword] how to * Method type 0.3 [Keyword] causes * Weaning 0.5

The correct answer extractor 150 classifies the sentence describing the correct answer and information describing other information other than the correct answer in each snippet, and shows only the sentence describing the correct answer to the user, or the sentence describing the correct answer may have a different color. Or you can use bold and other sentences that are dimmed so that the user can find the answer more intuitively and quickly.

However, when the user selects or presets the search result through the reranking engine unit 140 to the predetermined state or the descriptive query response device 100, the reranking engine unit 140 Instead of delivering the reranked result to the correct answer extractor 150, the reranked result itself may be provided to the user as a search result.

3 is a flowchart illustrating an operation procedure of a narrative query response device according to an embodiment of the present invention.

Referring to FIG. 3, in operation 300, the narrative question answering apparatus 100 receives a natural language question from a user through the input unit 110, and in step 302, the natural language question is analyzed by the question analysis engine unit 120 to generate a main keyword. Extract and determine the question type.

In step 304, the question analysis engine unit 120 determines whether the clue word is added and adds the clue word to the extraction keyword when the clue word is added in step 306.

In operation 308, the web search engine unit 130 searches for the relevant document based on the extracted keyword. In this case, when the clue word is added, the related document is searched based on the extracted keyword and the clue word.

The retrieved result is transmitted to the reranking engine unit 140, and in step 310, the reranking engine unit 140 performs reranking so that the document including the correct answer in the searched document is in a higher rank. For example, re-ranking a snippet that is a searched document and a summary of the searched document on each hyperlinkable web page.

Thereafter, in step 312, it is determined whether to extract the correct answer. The result is immediately available to the user.

However, when the preset state in the user's selection or descriptive question and answer device is set to "output through extraction of the correct answer from the reranked result" in step 312, the reranking engine information 140 re-ranks the result information. Is transmitted to the correct answer extractor 150, and the correct answer extractor 150 extracts the correct answer sentence from the reranked document in step 316 and classifies it with the surrounding information to provide the user.

As described above, the method and apparatus for descriptive query response according to an embodiment of the present invention receives a descriptive question in a natural language form from a user and automatically extracts a sentence-type correct answer. By adding keywords and clue keywords, the search performance can be improved, and the user satisfaction can be improved by finding the correct answer in the searched snippet and distinguishing the surrounding information from the correct answer.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but is capable of various modifications within the scope of the invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the scope of the appended claims, and equivalents thereof.

100: descriptive question and answer device 110: input unit
120: question analysis engine unit 130: web search engine unit
140: reranking engine unit 150: correct answer extraction unit

Claims (1)

Take the narrative questions, extract the key keywords of the question through analysis, classify the question types according to the intention of the question, and add clue words for each type,
Searching related documents in a linked website document or a previously collected document based on the extracted key keywords and clue words;
The process of reranking the documents that contain the correct answer among the retrieved documents to rank higher.
Descriptive query response method comprising a.

Images