JP2688020B2 - Derivative word processing method - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a natural language analysis device, and more particularly to a method for processing derivative words in language analysis in a machine translation device or the like. Conventional technology In machine translation etc., words that are not registered in the dictionary are usually processed as unknown words, but unknown words may have many part-of-speech possibilities or cannot be translated. In addition, processing such as analysis is made extremely difficult. That is, regarding the derivative word, since the part of speech of the derivative word, the semantic feature, the translated word, etc. are not conventionally estimated, the translation result of the translation does not convey the meaning of the original sentence. Therefore, (1) even when the wrong part of speech is taken into consideration As a result, many erroneous analyzes occur. (2) I can't get the translated words well. And the like. Purpose The present invention has been made in view of the above-mentioned circumstances, and in particular, estimates the grammatical property, the semantic property, or the translated word according to the conditions for those recognized as derivative words by affixes and the like. The purpose of this is to increase the certainty of the obtained analysis result and translation result. Structure In order to achieve the above object, the present invention is a translation device for translating an original language sentence into a target language sentence, which has a word dictionary and an affix dictionary separately, and uses the words in the original language sentence as a key, and the word dictionary And the word used as the key is not in the word dictionary, the affix dictionary is searched to enter the derivative word process, and at least the word for the original language sentence analysis is performed for the word recognized as the derivative word. The feature is that the grammatical information and the grammatical information for generating the object word sentence are added, and the contents are used for the source word sentence analysis and the object word sentence generation. Hereinafter, a description will be given based on examples of the present invention. FIG. 13 is a diagram for explaining the overall configuration of an English-Japanese automatic translation device to which the derivative word processing method according to the present invention is applied. The present invention, however, is an English-Japanese automatic translation device for translating English into Japanese. It goes without saying that, when translating one language into another language, it can be effectively applied to an analysis device of any language that analyzes a sentence of an input language. In FIG. 13, 1 is an input part, 2 is an English text, 3
Is a pre-editing unit, 4 is a morphological analysis unit, 5 is a syntactic analysis unit 1, 6
Is a syntax analysis II unit, 7 is an operation display unit, 8 is a word dictionary, 9 is an analysis rule file, 10 is a control unit, 11 is a structure conversion unit, 12
Is a translated sentence generating unit, 13 is a post-editing unit, 14 is an output unit, 15 is a Japanese sentence, and the translator has an input unit 1 as shown in the figure.
Thus, the English text 2 to be translated into Japanese is input. The input unit 1 is, for example, a keyboard having character keys such as alphanumeric keys and function keys, an optical character reader (OCR) for reading English text recorded on paper, and / or a recording medium such as a magnetic disk. It may include a file storage device or the like for reading the recorded English text. The English text input by the input unit 1 is read by the pre-editing unit 3 and pre-translation processing is performed. Here, processing of sentences, recognition, and unknown words is mainly performed. This functions as part of the morphological analysis. The pre-edited English sentence data is transferred to the morphological analysis unit 4 together with the information obtained by the pre-editing. In the morphological analysis unit 4, the word dictionary 8 is indexed and divided into sentences, the morphemes of English sentences are analyzed, various types of summarization such as unknown word processing, proper nouns, time expressions, and number expressions are performed, and additional questions are asked. , Process the entire sentence such as accreditation. The morphological analysis rules are stored in the analysis rule file 9. The English sentence data subjected to the morphological analysis is transferred to the syntax analysis unit 5 together with the dictionary information obtained by the morphological analysis. The syntactic analysis unit 5 is a functional unit that applies a grammar rule to English sentence data, analyzes the surface structure of the sentence, and finds all syntactic possibilities. The English sentence data parsed by the parsing I unit 5 is sent to the parsing II unit 6 together with the parsing information. Here, a solution is selected by applying a structural description from the surface analysis result obtained by the analysis I. This creates a probable parse tree of the English description, and its structure. These parsing rules are also stored in the parsing rule file 9. The parsed English sentence data is transferred to the structure conversion unit 11 as parse tree data. The structure conversion unit 11 creates a corresponding syntax tree of a Japanese sentence from a syntax tree that is an intermediate structure of the English sentence, and converts the Japanese sentence into a Japanese base structure that is easy to translate. The syntax tree data indicating the basic structure of Japanese thus structurally converted is sent to the translated sentence generation unit 12, and the translated sentence is generated in the latter. This is a functional unit that generates a Japanese sentence from the tree structure of the Japanese syntax tree. The translated sentence data, that is, the translated sentence data, is sent to the post-editing unit 13. The post-editing unit 13 uses the information used for the translation process to index the dictionary 8 to correct the translated sentence data, thereby completing a more natural Japanese sentence. The Japanese sentence data is transferred to the output unit 14, the translated Japanese sentence data is transferred to the output unit 14, and is output from the output unit 14 as a translated Japanese sentence 15. The output unit 14 includes a file storage device such as a printer, a display, and / or a magnetic disk. The flow of these series of translation processes is controlled by the control unit 10 which controls the entire control of the present apparatus. In the case of the illustrated example, the word dictionary 8 stores dictionary data for English and Japanese words, and describes not only vocabulary but also various kinds of information such as relations, that is, co-occurrence relations, meanings, singular words, parts of speech, and the like. Have been. The analysis rule file 9 stores rule data for morphological analysis and English sentence analysis. The operation display unit 7 is connected to the control unit 10. The operation display unit 7 gives various instructions from the operator to the apparatus. For example, it has operation keys such as a translation instruction key and a cursor key, a display and an indicator for visually displaying input English sentence text, translation result Japanese sentence, intermediate data such as dictionary information, various instructions to an operator, and the like.
Many of these operation display functions may be configured to be included in the keyboard when the input unit 1 includes a keyboard, and included in the display when the output unit 14 includes a display. According to the present invention, in the machine translation device as described above, when there is a derivative word in the English text 2, the derivative word is recognized as a derivative word by an affix or the like, according to the condition, in particular, the object word. By estimating a grammatical property or a semantic property, a translated word, or the like to which (translated word) generation grammatical information is added, the certainty of the obtained analysis result or translation result is increased. The affix dictionary is used to estimate dictionary information for unknown words in morphological analysis, but requires three types of processing: prefix processing, suffix processing, and suffix estimation processing. However, there are essentially two types of data: prefix estimation data and suffix estimation data. First, the prefix estimation data and the suffix estimation data will be described. (1) Prefix estimation data When the prefix part of a dictionary unregistered word matches the one described here and the rest is in the dictionary, the word is made to conform to the dictionary data of the stem part. In the dictionary data, it is possible to add an intrinsic feature to the original intrinsic feature sequence and a Japanese prefix to the original translated word. For example, as shown in the example below, the prefix entry [ele
dictionary information storage table entry [electroche] from ctro] and dictionary entry [chemical]
mical] can be based on dictionary data. As the dictionary information storage table entry, all dictionary information corresponding to the word stem is taken over. (2) Suffix estimation data When the suffix part of a dictionary unregistered word matches the one described here and the rest is in the dictionary, new dictionary data according to the information described in the suffix dictionary for that word. Give and register. At this time, the first translated word in the dictionary data corresponding to the word stem is extracted and used in the translated word of the new dictionary data. For example, as in the example below, for the input word [controler], the suffix entry [-(e) r] and the dictionary entry [co
ntrol] dictionary information storage table entry [controle
r] is registered. Next, an outline of processing of derivative words and processing of unknown words will be described. A Derived word processing a) Processing method If there is a prefix or suffix at the beginning or ending of a word that is not registered in the dictionary and the remaining part is registered in the dictionary, the dictionary information and Based on the affix information, English part-of-speech information, intrinsic features, and Japanese translations are synthesized. The prefix and suffix can be edited in a table format and can be edited as an external file format independently of the program. Attempt the prefix possibility first, and if that fails try the suffix possibility. If you have both, don't try. The word that fails here is subjected to the ending estimation process as an unknown word. B) Suffix dictionary a) Overview One file has both a prefix dictionary and a suffix dictionary. Arrange in order of prefix and suffix. Each is identified, for example, by the following heading record. "*** PREFIX" ... Prefix "*** SUFFIX" ... Suffixes Suffix entries for unknown words are mixed in the suffix dictionary. This file is in a format that can be edited with an editor such as Vi. b) Prefix dictionary "*** PREFIX" indicates the beginning of the prefix dictionary. Lines starting with # are valid, and other lines are comment lines. For each of the prefix, word base part-of-speech, intrinsic feature, and translation words, enter ":"
Separate. The effective character of each column is described immediately after "#" or ":". Space becomes a delimiter. At the end of the prefix dictionary, the first 9 characters are "*** SUFFIX"
Or judge by eof. The term "(-)" at the end of the prefix indicates that the term "(-)" may or may not be present at the end. The word base part-of-speech specifies either part-of-speech (N, V, ADJ, ADV, etc.) or "*" (anything is possible) as the part-of-speech information of the word excluding the prefix. When specifying multiple intrinsic features, separate them with "," without spaces and specify them continuously. The translation ends with Kanji space / Space / NL (NewLine). The correspondence of the code of the word base part of speech is as shown in the following example. c) Suffix dictionary "*** SUFFIX ***" indicates the beginning of the suffix dictionary. Lines starting with "#" or "=" are valid, and other lines are comment lines. Separate each column with ":". The valid characters in each column are described immediately after "#" and ":". Space becomes a delimiter. The end of the suffix dictionary is determined by eof. The word base part-of-speech and word-base change are information about the word in a format excluding the suffix part in the word. The former is the part-of-speech (N, V, ADJ, AD
V, etc.), the latter may be used as a base by supplementing a specific character string at the end of the string excluding the suffix,
Describe as "one character string". The part-of-speech specifies the English part-of-speech of the word. In the part-of-speech subclassification, the English part-of-speech subclassification of the word and its weight (&, &&, ...) Are successively specified. The intrinsic feature specifies the intrinsic feature of the word. If there is more than one, separate them with "," and specify continuously. The translated word {part of speech / advanced type} is the translated word information of the part corresponding to the suffix, the translated word is specified in the final form, and then in {},
Specify the part of speech of the translated word and the inflection type if necessary. That is, the translated word of the word is synthesized by {translated word of base part} + translated word. "{", "}", And "・ (chukoku)" are all described in Kanji code. The practical code correspondence is as follows. In addition, the utilization type code that does not correspond is 0. Since the same suffix may have another part of speech, subclass of part of speech, intrinsic feature, or translation, in that case, specify only “=” in the suffix column on the next and subsequent lines, and the word base part of speech and word base part of speech All columns below the part-of-speech are described as spaces. The entry for estimating the ending of an unknown word is the one whose word base part-of-speech column is "*" in this dictionary. Other fields are specified exactly as the suffix. When used both as a suffix and for estimating the ending, both must be specified. Correspondence codes of the parts of speech of the translated words are as follows. The code of the part of speech of the translated word that does not correspond is 0. C) Processing by prefix Under the following conditions, it is considered as a derivative of prefix. 1 ° The beginning of the word matches the entry word in the prefix dictionary. 2) The dictionary is searched in units excluding the prefix of the word to find the entry. The 3 ° dictionary entry matches the word base part-of-speech listed in the prefix dictionary. However, when the word base part-of-speech is "*", any part of speech is applicable because it can be anything. When the above conditions are met, all the relevant dictionary information is fetched, the intrinsic feature and the translated word are combined as follows, and registered as a new entry in the dictionary information storage table. Intrinsic feature ... OR (logical sum) with those specified in the prefix dictionary and those in the dictionary. Translated word: The translated word written in the prefix dictionary is embedded at the beginning of the translated word in the dictionary. D) Processing by suffix Under the following conditions, it is considered as a derivative of suffix. 1 ° The head part of the word matches the entry word in the suffix dictionary. 2 ° The dictionary is searched in units excluding the suffix of the word to find the entry. If the corresponding word is not found, the base change specified in the prefix dictionary is performed and the dictionary is searched again to find the entry. The 3 ° dictionary entry matches the word base part-of-speech listed in the suffix dictionary. When the above conditions are met, only the first dictionary information of the corresponding entry is fetched, the dictionary information is created as follows, and registered as a new entry in the dictionary information storage table. Part-of-speech ... Part-of-speech part-of-speech subclassification and weight specified in suffix dictionary ... Part-of-speech subclassification and weight specified in suffix dictionary Internal feature ... Intrinsic feature translation specified in suffix dictionary ... The part of speech of the translated word and the conjugation type ... The part of speech code of the translated word in the suffix dictionary and the conjugation type code B Processing of unknown words Even if the derivative word processing of the item A does not apply at all, in the affix dictionary The part-of-speech is estimated using the entry for estimating the ending of (see all items). When the ending part of the word matches the entry word of the ending estimation entry, create dictionary information as follows,
Register as a new entry in the dictionary information storage table. Part-of-speech ... Part-of-speech part-of-speech subclassification and weight specified for word-end estimation ... Part-of-speech subclassification and weight specified for word-end estimation Internal feature ... Intrinsic feature translation specified for word-end estimation ... Part of speech and active type of translated word ... Part of speech code and inflection type code specified in entry for estimating ending of a word Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram for explaining an embodiment of the present invention, and FIG. 2 is a flow chart for explaining the operation of FIG. 1, in which 20 is an input processing unit and 21 is a unit. A cutout unit, 22 is a delimit table, 23 is a derivative word processing unit, 24 is a reference dictionary, and 25 is a dictionary information storage table. First, in the input processing unit, an input device including an input document file or keyboard, OCR, etc. The English document is read from, then, in the cut-out unit of the dictionary lookup unit, the cut-out of the dictionary lookup unit is performed with reference to the delimit table, and if it is not the end (end), for example, the dictionary structure as shown in Table 1 is provided. A dictionary search is performed using the reference dictionary. If there is an entry as a result of the search, it is recorded in the dictionary information storage table, and if there is no entry, the derivative word is processed. In addition, in Table 1, the property information includes a part-of-speech subclassification and the like. FIG. 3 is a block diagram for explaining an example of derivative word processing by a prefix, and FIG. 4 is a flowchart for explaining the operation of FIG. 3, in which 30 is a head part and a prefix. Matching part of the dictionary, 31 is a prefix dictionary, 32 is a dictionary search part that searches the dictionary excluding the prefix part, 33 is a matching part between the word base part of speech of the prefix dictionary and the part of speech of the entry, and 34 is based on prefix estimation. A dictionary information creation unit 35 is a derivative word processing unit by a prefix, and in FIG. 4, a portion surrounded by a dotted line is a derivative word processing unit by a prefix, and the other portion, that is, a derivative word processing by a suffix is In short, it is used when the derivative processing by the prefix fails. First, the beginning part of the input word cut out as a unit is matched with the prefix dictionary, and when the beginning part matches the prefix, a dictionary search is performed excluding the prefix part, and if there is an entry,
The word base part-of-speech of the prefix dictionary and the part-of-speech of the entry are matched, and when they match, dictionary information is created by prefix estimation. For example, when the word cut out as a unit is anti-war, it is matched with the prefix anti (-), becomes one word with anti-war, and "anti" is added before the translated word and shown in column B of Table 2. Like Table 2 is a table showing an example of the contents of the dictionary storage table, and the table 2 shows movable anti-warpeaco.
ck is recorded, and the movable start position, or “m”
If the position of "6" is set, the end position of movable, that is, the position of "e" is "12", and if one space is left, the position of "a" of anti-war becomes "14", Thereafter, the start position and the end position are similarly determined. FIG. 5 is a flow chart showing the details of the dictionary information creation processing by the prefix processing shown in FIG. 4, but the word base refers to the part from which the prefix part has been removed, and the word base part of speech of the prefix dictionary. The dictionary information of the word base is read only for those that match. Next, the OR (logical sum) set of the property information (internal feature) of the word base part and the property information of the prefix part is taken (AUB). Next, the translated word information of the prefix is embedded in the leading portion of the translated word (information) of the word base portion. FIG. 6 is a block diagram for explaining an example of processing of a derivative word by a suffix, and FIG. 7 is a flowchart for explaining the operation of FIG. 6, in which 40 is a tail part and a suffix. Matching part of the word dictionary, 41 is a suffix dictionary, 42 is a complete (totally unknown) unregistered word processing part, 43 is a dictionary search part that searches the dictionary excluding the suffix part, and 44 is a suffix A matching unit between the word base part-of-speech and the entry part-of-speech, 45 is a dictionary information creation processing unit by suffix estimation, and 46 is a dictionary search unit that performs a dictionary search by adding the base change in the suffix to the part excluding the suffix part , 47 are unregistered word processing units by suffix estimation.
Match the end part with the suffix dictionary, and if it does not match, process it as a complete (totally unknown) unregistered word,
When there is a match, a dictionary search is performed using the part excluding the suffix part. If there is no entry as a result of the dictionary search, the dictionary search is performed by adding the word base change in the suffix dictionary to the part excluding the suffix part. As a result, if there is no entry, it is not registered by suffix estimation. Process words. If there is an entry, the part of speech of the entry is matched with the word base part of speech in the suffix dictionary, as in the case of the entry obtained as a result of the dictionary search in the part excluding the suffix part. Performs dictionary information creation processing by suffix estimation, and when unmatched, performs unregistered word processing by suffix estimation. For example, if the movable shown in the column A of Table 2 is taken as an example, it matches the suffix “able”. It does not match with mov, but since the base change is -e, it complements e and matches with move. Information such as translated words is formed by taking the word base part of speech V. FIG. 8 is a detailed flowchart of dictionary information creation processing by suffix estimation. Reading the dictionary information of the word base part
Perform only those that match the word base part of speech in the suffix dictionary. or,
To change the part-of-speech, etc., the part-of-speech (and its detailed information) and property information are changed to those in the suffix dictionary (however, if there is no corresponding information in the suffix dictionary, the original dictionary information is saved). Also, a translation word having a suffix to the end of the translation word information is added. Change the part-of-speech of the translated word, the usage information, etc. to that in the suffix dictionary. FIG. 9 is a flow chart showing details of unregistered word processing by suffix estimation. Information such as a part of speech is created by adding the part of speech (and its detailed information) and property information to that of the suffix dictionary, In the creation of information such as, the translated word is added to the word form, and the part of speech of the translated word, its utilization, etc. are set to those of the suffix. Fig. 10 is a flowchart showing the details of complete unregistered word processing (when it is completely unknown). For example, to create estimated information as a noun, part-of-speech ← noun (default detailed information) translated word ← English word form translation Part of speech ← Create like a noun. In addition, the estimated information as a verb is created as follows: Part of speech ← Verb (detailed detailed information) Translated word ← English form + “Suru” Part of speech of translated word ← Use of translated word ← Sahen For example, since peacock in column C of Table 2 is not in the dictionary, neither the prefix nor the suffix match. FIG. 11 is an overall detailed block diagram obtained by combining FIG. 1, FIG. 3, and FIG. 6 described above, and FIG. 12 is the complete (completely unknown case) shown in FIG. The unregistered word processing unit 42 is detailed according to the flowchart shown in FIG. 10, and the complete unregistered processing unit 42 shown in FIG. 11 includes the estimated information creation processing unit 50 as a noun and the estimated information as a verb. Creation processing section
It is a clarification that it consists of 51 and. However, since the respective parts of FIGS. 11 and 12 have already been described in detail, the description thereof will be omitted here. Effect As is clear from the above description, according to the present invention, grammatical information is added to a word recognized as a derivative word such as an affix, according to conditions, and in particular, grammatical information for generating an object (translated word) is also added. Since the property or the semantic property, the translated word, or the like is estimated, the certainty of the obtained analysis result or translation result can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram for explaining the overall configuration of the present invention, FIG. 2 is a flowchart for explaining the operation of FIG. 1, and FIG. 3 is a prefix. 4 is a block diagram for explaining an example of derivative word processing by FIG. 4, FIG. 4 is a flow chart for explaining the operation of FIG. 3, and FIG. 5 is dictionary information creation by prefix estimation shown in FIG. A detailed flowchart of the processing unit, FIG. 6 is a block diagram for explaining an example of derivative processing by a suffix, FIG. 7 is a flowchart for explaining the operation of FIG. 6, and FIG. A detailed flowchart of the dictionary information creation processing unit by suffix estimation shown in FIG. 7, FIG. 9 is a detailed flowchart of the unregistered word processing unit by suffix estimation shown in FIG. 7, and FIG.
A detailed flowchart of the unregistered word processing unit by suffix estimation shown in FIG. 7, FIG. 11 is an overall detailed block diagram combining FIG. 1, FIG. 3, and FIG. 6, and FIG. , 11th
FIG. 13 is a block diagram showing the complete unregistered word processing unit 42 in more detail in the figure, and FIG. 13 is a block diagram for explaining an example of an automatic translation apparatus to which the present invention is applied. 20 …… Input processing unit, 21 …… Unit cutout unit, 22 …… Delimit table, 23 …… Derivative word processing unit, 24 …… Reference dictionary, 25 …… Dictionary information storage table, 30 …… Matching unit, 31 ...... Prefix dictionary, 32 …… Dictionary search unit, 33 …… Matching unit, 34 …… Dictionary information creating unit, 35 …… Derivative word processing unit, 40 …… Matching unit, 41 …… Suffix dictionary, 42… … Unregistered word processing unit, 43 …… Dictionary search unit, 44 …… Matching unit, 45 …… Dictionary information creation processing unit, 46 …… Dictionary search unit, 47
...... Unregistered word processing unit, 50 …… Estimated information creation processing unit as a noun, 50 …… Estimated information creation processing unit as a verb.

Claims (1)

(57) [Claims] In the translation device that translates the original language sentence into the target language sentence,
It has a word dictionary and an affix dictionary separately, searches the word dictionary by using a word in the original sentence as a key, and if the word used as the key is not in the word dictionary, derives by searching the affix dictionary. The word processing is started, and at least grammatical information for source language sentence analysis and grammatical information for object sentence generation are added to the word recognized as a derivative word, and in the subsequent processing, the word is used as a known word in the original language sentence. A derivative word processing method characterized by performing analysis and object sentence generation.