KR20030039575A - Method and system for summarizing document - Google Patents

Abstract

The present invention relates to a method and system for automatically summarizing a document, which can be applied to summarize a specialized document such as a patent document, as well as a document summarization method which can improve summarization efficiency and readability by increasing summarization efficiency. We want to provide a system, which has a purpose.

The present invention is to automatically summarize the document to identify the structural features of the document and structured by a predetermined rule, and then to extract patterns frequently occurring in the structured paragraph of the document, and to divide the rhetorical information and document level paragraphs of the document A method and system for automatically summarizing a document using natural language processing (NLP) technology such as word and air information are provided.

Description

Document Summary Method and System {Method And System For Summarizing Document}

The present invention relates to a method and system for automatically summarizing a document. More particularly, the present invention provides a method and system for automatically summarizing a document. The present invention relates to a method and system for automatically summarizing a document by extracting a pattern generated and using natural language processing (NLP) techniques such as document structure information, document level paragraph division, and word air information.

Conventional techniques for automatically summarizing documents include Korean Patent Publication Nos. 96-28890 and 97-707499.

Korean Patent Publication No. 96-28890 discloses an information summary method, an information summary device, a weighting method, and a character broadcast receiving device. In this case, when a plurality of character string data divided into predetermined units are given, a plurality of them are provided. The information is summarized by extracting effective and important keywords as information summaries and calculating scores for each keyword.

In addition, Korean Patent Publication No. 97-707499 discloses a document summary method and apparatus, wherein input messages, command sets, and maximum messages are made possible by selecting an important information part and removing unnecessary message parts based on a reference number. The length is provided to generate a message that conveys all the important information in the document message but within the limited range of the maximum message length.

Meanwhile, Korean Patent Publication No. 2000-54268 discloses a document classification retrieval method and a document classification retrieval system using automatic document summarization. Here, after constructing a keyword database and a topic sentence database using an automatic summary, It takes a document as input and searches for a document with content similar to that of the key document.

In other words, the document classification retrieval method using the document automatic summarization and the automatic document summary system used in the document classification retrieval system have a parsing step of reading the content of the document and classifying it into an analysis unit for summary and the lowest element of the document. It consists of two stages of constructing the topic information by collecting frequency information based on the word.

However, such conventional methods use the document summary as an additional means for targeting a general document or a message of a mail system or for use in a document classification retrieval method and a document classification retrieval system. It is not applicable to summarizing a specialized document, and when extracting an important keyword or summarizing using only word frequency, there is a problem of degrading summary performance and degrading readability because the summarization efficiency cannot be improved.

The present inventors have conducted research to solve the above-mentioned problems of the prior art, and based on the results, the present invention has been proposed, and the present invention can be applied not only to summarizing specialized documents such as patent documents, The purpose of the present invention is to provide a method and system for summarizing documents that can improve summary efficiency and readability.

1 is a flow diagram illustrating an example of a method of summarizing a document in accordance with the present invention.

2 is a block diagram showing an example of a document summary system consistent with the present invention.

Explanation of symbols on the main parts of the drawings

11. . . Document input section 12. . . Document Determination Unit 13. . . Short circuit section 14. . . Pattern Extraction Unit 15. . . Sentence Extractor 16. . . Document summary section

EMBODIMENT OF THE INVENTION Hereinafter, this invention is demonstrated.

The present invention relates to a method for summarizing a document,

Performing pattern search on a document having a structural feature to preform a pattern structure;

Dividing the stereotyped pattern into a core paragraph and an additional paragraph according to importance;

Inputting a document to be summarized;

Determining whether the input document is atypical or atypical;

If the input document is an unstructured pattern, searching the contents of the document and extracting a pattern identical or similar to a pre-formed pattern;

Summarizing a document by removing a less important sentence using paragraph division, rhetorical structure information, and word air information from the extracted pattern as described above;

When the input document is a standardized document, classifying the document into a core paragraph and an additional paragraph according to the paragraph division criteria separated above;

Determining the presence or absence of a pattern that is the same as or similar to the preformed pattern in the divided core paragraphs;

Extracting a pattern when there is a pattern in the core paragraph;

From the extracted patterns and additional paragraphs, paragraph sentences, rhetorical structure information, and word air information are used to remove less important sentences, and to extract important sentences and important sentences from the extracted patterns and important sentences in additional paragraphs. Summarizing the document using; And

If there is no pattern identical or similar to the preformed pattern in the core paragraph, the sentence is extracted by removing less important sentences using paragraph division, rhetorical structure information, and word air information from the core paragraph and the additional paragraph. The present invention relates to a document summary method comprising the steps of summarizing a document using an important sentence in a core paragraph and an important sentence in an additional paragraph.

In addition, the present invention provides a system for summarizing a document,

A document input unit for receiving a document to be summarized;

Document determination unit for determining whether the document input to the document input unit is atypical or standardized:

If it is determined by the document determination unit that the input document is a standardized paragraph separation section for dividing the core paragraph and additional paragraphs according to a predetermined paragraph classification criteria after document structure;

A pattern extraction unit for extracting a pattern identical or similar to a preformed pattern in the core paragraph or the atypical document separated by the paragraph separator;

A sentence extracting unit extracting an important sentence by removing a less important sentence using paragraph division, rhetorical structure information, and word air information in a pattern extracted from the pattern extracting unit, a core paragraph and an additional paragraph without a pattern; And

If the sentence extracted by the sentence extractor is made of a pattern extracted from an unstructured document, the document is summarized based on the sentence extracted from the sentence extractor, and the sentence extracted from the sentence extractor is formatted. If the pattern is extracted from the core paragraph of the document, the document is summarized based on the pattern extracted from the sentence extraction unit and the important sentence extracted from the additional paragraph. If there is no pattern in the core paragraph of the standardized document, the core is summarized. The present invention relates to a document summary system including a document summary section that summarizes documents based on important sentences extracted from paragraphs and subparagraphs.

Hereinafter, the present invention will be described in more detail.

In order to summarize the document according to the present invention, as shown in FIG.

Pattern search is performed on a document having structural characteristics to form a pattern structure (step 110).

Documents include general documents, technical documents, and specialized documents such as patent documents.

In the case where the document is a patent document which is one of the specialized documents, the step 110 will be described in detail as follows.

That is, in order to summarize a patent document according to the present invention, for example, a pattern search should be performed on a specification attached to an application (hereinafter, also referred to as an "application") to form a pattern structure.

The application is a formal document and is configured to describe the field of use of the invention, the purpose of the invention, the effects of the invention, the constitution of the invention, and the prior art and problems.

One pattern for the field of use of the invention is, "the present invention (design) relates to a system (method, equipment) to be described."

Moreover, as one pattern with respect to the objective of the said invention, "the present invention (design) has the objective in the case of (the purpose)").

Moreover, as one pattern about the effect of the said invention, "the effect of-is according to this invention (design)." Is mentioned.

That is, in order to summarize the patent document according to the present invention, first, pattern search is performed on the patterned application as described above to form a pattern structure.

Next, the formalized pattern as described above is divided into a core paragraph and an additional paragraph, and a relatively important item is assigned to the core paragraph, and a relatively non-item is assigned to the additional paragraph (step 120).

For example, in the case of a patent document, it is preferable that the core paragraph is assigned such a field of use, purpose, and effect, and the additional paragraph is assigned such that the prior art and problems, composition, operation, claims, and the like are allocated.

Next, enter the document to be summarized (step 130).

When the document to be summarized is input, it is first determined whether the input document is unstructured or standardized (step 140).

In the case where the document is a patent document, for example, the application (e.g., invention, design, proposal, etc.) has an informal structure created by the field creator, and the patent specification requirements required to obtain the patent. Documents with a structured structure written in conformance can be classified, for example, as an application (the specification attached to the application).

The application includes a formalized structure of the pre-patent application stage or the patent application stage, as well as the formalization of the patent application publication, publication, registration and post-registration stage.

The application is a document that is not described in a patented manner and generally has a missing core, a missing paragraph, and low readability.

If the input document is an atypical document (application), the pattern is searched for an atypical document (application) to extract a pattern set as described above (step 150).

For example, in the case of a patent document, a pattern for the field of use of the invention, a pattern for the purpose of the invention, and a pattern for the effect of the invention are extracted from the description of the application.

Next, the document is summarized by removing less important sentences using paragraph division, rhetorical structure information, and word air information based on the extracted pattern as described above (step 160 and step 200).

On the other hand, if the input document is a formalized (application), the document is divided into core and additional paragraphs according to the paragraph classification criteria classified above (step 170).

That is, the contents of the document (application) formatted according to the allocation pattern criteria for each of the core paragraphs and the additional paragraphs set as described above are divided into the core paragraphs and the additional paragraphs.

For example, if the document is a patent document, the description of the field of application, purpose, and effect of the invention in the above-mentioned application shall be the core paragraph, and the description of the prior art and problems, composition, operation, claims, etc. It is preferable to divide by additional paragraph.

Next, it is determined whether or not the same or similar pattern as the pre-formed pattern in the core paragraph divided as described above. (Step 180)

If there is a pattern in the core paragraph, the pattern is extracted (step 190).

For example, in the case of a patent document, patterns related to the field of use, purpose, and effect are extracted from the above-mentioned core paragraphs.

From the extracted patterns and additional paragraphs, paragraph sentences, rhetorical structure information, and word air information are used to remove less important sentences, and to extract important sentences and important sentences from the extracted patterns and important sentences in additional paragraphs. The documents are summarized (step 160 and step 200).

If the core paragraph does not have the same or similar pattern as the pre-formed pattern, the sentence is extracted by removing the less important sentence using paragraph division, rhetorical structure information, and word air information from the core paragraph and the additional paragraph. The document is summarized using the important sentences in the extracted core paragraph and the important sentences in the additional paragraph (steps 160 and 200).

Hereinafter, in the extracted pattern, the core paragraph without the pattern, and the additional paragraph, the less important sentences are removed by using paragraph division, rhetorical structure information, and word air information to extract important sentences, and the documents are summarized using these. An example of how to do this is described.

In order to summarize the document by removing the less important sentences from the extracted pattern, the core paragraph without the pattern, and the additional paragraph, the rhetorical information between the units (sentences or paragraphs) that make up the document is used or paragraph level division is performed. And word co-occurrence using the information retrieval methodology at the paragraph level.

First, in order to use the rhetorical information, in-paragraph conjunctions (and, but, or, by the way), infix conjunctions (and, therefore, in order to ...), transitional conjunctions (but, on the other hand, , On the other hand, on the other hand, ..) gives differentiation according to the relative importance to give the weight of importance to the summary part the highest, and removes the less important sentences in parallel and example order.

At this time, the parallel and exemplary sentences may be selectively accommodated according to the desired summary rate.

Summary-in conclusion, conclusion, in conclusion, summary, briefly, briefly, necessarily,

Parallel-and-and-and-or-or-or-or -...

Example-e.g., For example, say, eg, Excuse me,

In addition, paragraph division at the sentence level is divided into two stages. Segment division in the first stage uses the topic marker of the transition relationship, and sentence division in the second stage uses the similarity between the sentences to separate the document by topic. The important sentence is extracted by calculating the similarity between adjacent sentences in the physical segment of.

At this time, the topic marker is "but, on the other hand," as the words can be important information (cue information) in sentence division.

Finally, the key sentence is extracted by re-adjusting the weight of key words by the word-air information approach.

Hereinafter, a document summary system according to the present invention will be described in detail with reference to FIG.

As shown in FIG. 2, the document summary system 10 of the present invention includes a document input unit 11, a document determination unit 12, a paragraph separator 13, a pattern extraction unit 14, and a sentence extraction unit 15. , And document summary portion 16.

The document input unit 11 is configured to receive a document, the document determination unit 12 is a document input to the document input unit 11 is an unstructured (application) or a standardized (for example , Application).

In addition, when it is determined that the input document is a standardized document (for example, an application) by the document determination unit 12, the paragraph separator 13 is configured according to a predetermined paragraph classification standard after document structure. It is structured to distinguish between core and additional paragraphs.

The pattern extracting unit 14 is configured to extract a pattern identical or similar to a preformed pattern in the core paragraph or the unstructured document (for example, the application) separated by the paragraph separating unit 13.

The sentence extractor 15 removes less important sentences using paragraph division, rhetorical structure information, and word air information from the pattern extracted from the pattern extractor 14, the core and additional paragraphs without the pattern, and the important sentence. It is configured to extract.

The document summary unit 16 is extracted from the sentence extraction unit 15 when the sentence extracted from the sentence extraction unit 15 is made with respect to a pattern extracted from an atypical document (for example, an application form). The sentence extracting unit 15 is summarized based on a sentence, and when the sentence extracted from the sentence extracting unit 15 is made with respect to a pattern extracted from a core paragraph of a standardized document (for example, an application). Summarize the document based on the pattern extracted from) and the important sentences extracted from the additional paragraphs, and if there are no patterns in the core paragraphs of the formalized document (eg application), then the important sentences extracted from the core and additional paragraphs Configured to summarize the document on the basis of.

As described above, the present invention is not only applicable to summarizing specialized documents such as patent documents, but also has an effect of providing a document summarization method and system that can improve summarization efficiency and readability by improving summarization efficiency. will be.

Claims (12)

In the method of summarizing a document, Performing pattern search on a document having a structural feature to preform a pattern structure; Dividing the stereotyped pattern into a core paragraph and an additional paragraph according to importance; Inputting a document to be summarized; Determining whether the input document is atypical or atypical; If the input document is atypical, searching for a pattern of the atypical document and extracting a pattern identical or similar to a preformed pattern; Summarizing a document by removing a less important sentence using paragraph division, rhetorical structure information, and word air information from the extracted pattern as described above; When the input document is a standardized document, classifying the document into a core paragraph and an additional paragraph according to the paragraph division criteria separated above; Determining the presence or absence of a pattern that is the same as or similar to the preformed pattern in the divided core paragraphs; Extracting a pattern when there is a pattern in the core paragraph; From the extracted patterns and additional paragraphs, paragraph sentences, rhetorical structure information, and word air information are used to remove less important sentences, and to extract important sentences and important sentences from the extracted patterns and important sentences in additional paragraphs. Summarizing the document using; And If there is no pattern identical or similar to the preformed pattern in the core paragraph, the sentence is extracted by removing less important sentences using paragraph division, rhetorical structure information, and word air information from the core paragraph and the additional paragraph. A document summary method comprising the steps of summarizing a document by using an important sentence in a core paragraph and an important sentence in an additional paragraph. The method of claim 1, wherein paragraph division comprises two stages of sentence division, one stage division of sentences uses a topic marker of a transition relationship, and two divisions of division uses a similarity of sentences between topics by using similarity between sentences. Document summary method, characterized in that to extract the important sentence by calculating the similarity between adjacent sentences for each physical segment The sentence extraction according to claim 1 or 2, wherein the sentence extraction based on the rhetorical structural information is given a distinctiveness according to relative importance by using a rank connection, a middle connection, and a conversion relationship in a paragraph, so that the weight of importance is summarized in the summary. Document summarization method characterized in that it is made in such a way as to give a high, parallel, example order to remove less important sentences The document summary method according to claim 1 or 2, wherein the extraction of the key sentence by the word air information is performed by readjusting the weight of the key word. The method according to claim 3, wherein the extraction of the key sentence by the word air information is performed by readjusting the weight of the key word. 6. The document according to claim 1, 2 or 5, wherein the document is a patent document, the atypical document is a document written not in accordance with the patent requirement, and the document is a document written in conformity with the patent requirement. Document summary method characterized in that The document summary method according to claim 3, wherein the document is a patent document, the atypical document is a document written not in accordance with the patent specification requirement, and the document is a document written in conformity with the patent specification requirement. The document summary method according to claim 4, wherein the document is a patent document, the atypical document is a document written not in accordance with the patent specification requirement, and the document is a document written in conformity with the patent requirement. In a system for summarizing documents, A document input unit for receiving a document to be summarized; Document determination unit for determining whether the document input to the document input unit is atypical or standardized: If it is determined by the document determination unit that the input document is a standardized paragraph separation section for dividing the core paragraph and additional paragraphs according to a predetermined paragraph classification criteria after document structure; A pattern extraction unit for extracting a pattern identical or similar to a preformed pattern in the core paragraph or the atypical document separated by the paragraph separator; A sentence extracting unit extracting an important sentence by removing a less important sentence using paragraph division, rhetorical structure information, and word air information in a pattern extracted from the pattern extracting unit, a core paragraph and an additional paragraph without a pattern; And If the sentence extracted by the sentence extractor is made of a pattern extracted from an unstructured document, the document is summarized based on the sentence extracted from the sentence extractor, and the sentence extracted from the sentence extractor is formatted. If the pattern is extracted from the core paragraph of the document, the document is summarized based on the pattern extracted from the sentence extraction unit and the important sentence extracted from the additional paragraph. If there is no pattern in the core paragraph of the standardized document, the core is summarized. A document summary system that includes a document summary that summarizes documents based on important sentences extracted from paragraphs and subparagraphs. 10. The document summary system of claim 9, wherein the document is a patent document, the atypical document is a document written not in accordance with the patent specification requirements, and the document summary system is a document written in accordance with the patent specification requirements. In a system for summarizing documents, A document input unit for receiving a document to be summarized; Document determination unit for determining whether the document input to the document input unit is atypical or standardized: If it is determined by the document determination unit that the input document is a standardized paragraph separation section for dividing the core paragraph and additional paragraphs according to a predetermined paragraph classification criteria after document structure; A short circuit determination unit for determining whether a short circuit divided by the short circuit division unit is a core short circuit or an additional short circuit; When the input document is judged to be an informal form or the paragraph divided by the paragraph separator is determined to be the core paragraph by the paragraph determination unit, the same pattern as the preformed pattern is used. Or a pattern extraction unit for extracting a similar pattern; A sentence extraction unit for removing less important sentences using paragraph division, rhetorical structure information, and word air information based on the pattern extracted document from the pattern extraction unit; A sentence extracting unit extracting a sentence using paragraph division, rhetorical structure information, and word air information when it is determined that the paragraph divided by the paragraph separator is an additional paragraph by the paragraph determination unit; And If the sentence extracted by the sentence extractor is made of a pattern extracted from an unstructured document, the document is summarized based on the sentence extracted from the sentence extractor, and the sentence extracted from the sentence extractor is formatted. In the case of the pattern extracted from the document, a document summary system including a document summary section for summarizing the document based on the core paragraph extracted from the sentence extraction unit and the additional paragraph extracted by the sentence extraction unit. 12. The document summary system of claim 11, wherein the document is a patent document, the atypical document is a document written not in accordance with the patent specification requirements, and the document summary system is a document written in conformity with the patent specification requirements.