JP6537996B2 - Unknown word detection device, unknown word detection method, program - Google Patents

Description

  TECHNICAL FIELD The present invention relates to speech recognition technology, and more particularly to technology for detecting, from an input speech signal, a word not included as a vocabulary of a speech recognition system.

  An utterance ranging from tens of minutes to several hours on a specific topic, such as speech of a lecture or lecture, is called a speech document. This speech document can be converted to text by speech recognition technology, and useful information can be quickly extracted from the speech document by language processing such as full text search and text mining.

  However, if misrecognition occurs during speech recognition of a speech document, words actually uttered will not appear in the recognition result. At this time, for example, there is a problem that a desired word can not be hit even if full-text search is performed. In particular, when a word not found in the vocabulary of the speech recognition system (hereinafter referred to as an unknown word) is repeatedly uttered in a speech document, the unknown word is always misrecognized and never appears as a recognition result. This problem is serious when considering the use of voice documents. Furthermore, since the words uttered repeatedly are often keywords in the voice document, it is difficult to extract useful information from the speech recognition result in which all the keywords are misrecognized.

  Non-Patent Document 1 discloses a technique for detecting a word uttered repeatedly as an unknown word as a countermeasure against the problem that the unknown word can not be recognized correctly. If a word uttered repeatedly can be detected as an unknown word, the speech document containing the unknown word can be correctly converted into a text by adding the unknown word to the vocabulary of the speech recognition system and performing speech recognition again. It becomes possible.

  Hereinafter, an outline of the unknown word detection device 800 disclosed in Non-Patent Document 1 will be described with reference to FIG. FIG. 8 is a block diagram showing the configuration of the unknown word detection device 800 of Non-Patent Document 1. As shown in FIG. As shown in FIG. 8, the unknown word detection device 800 includes an unknown word interval detection unit 810, an unknown word interval clustering unit 820, and a cluster selection unit 830.

  The unknown word interval detection unit 810 detects a plurality of intervals (hereinafter referred to as unknown word intervals) in which an unknown word in the voice document is uttered from the input voice document, and is uttered in each unknown word interval The phoneme string is estimated to generate an unknown word segment group with a phoneme string. Here, the method described in Non-Patent Document 2 is used as a method of estimating a phoneme string of an unknown word section and the unknown word section. The unknown word segment clustering unit 820 clusters the unknown word segment group with the phoneme sequence, which is the output of the unknown word segment detection unit 810, to generate an unknown word segment cluster group. As a distance measure used for clustering, the edit distance between phoneme strings of unknown word sections is used. If a large number of unknown word intervals are classified into the same cluster, the unknown word intervals belonging to the cluster are considered to be the same unknown word repeatedly uttered. Therefore, the cluster selection unit 830 selects unknown word segment clusters having a cluster size equal to or larger than a predetermined threshold from the unknown word segment cluster group output from the unknown word segment clustering unit 820, and includes unknown words that repeatedly appear. Generate unknown word group.

Long Qin and Alexander Rudnicky, “Finding Recurrent Out-of-Vocabulary Words”, in Proceedings of INTERSPEECH, pp. 2242-2246, 2013. Issam Bazzi and James R. Glass, "Modeling Out-of-Vocabulary Words for Robust Speech Recognition", in Proceedings of ICSLP, pp. 401-404, 2000.

  In the unknown word detection device 800, first, the unknown word period detection unit 810 detects an unknown word period. For this reason, whether or not the repeatedly uttered unknown word can be finally detected depends largely on the detection accuracy of the unknown word interval detection unit 810. In Non-Patent Document 2, which is a detection method in unknown word interval detection unit 810, when a word of pronunciation similar to the phoneme string of the unknown word uttered is included in the vocabulary of the speech recognition system, not the unknown word but the pronunciation As a result of assuming that similar known words are uttered, detection accuracy is reduced. In particular, a speech recognition system for lectures or lectures often has a large vocabulary exceeding 100,000 words, so that the detection accuracy of the unknown word section detection unit 810 is significantly reduced. As a result, the unknown word detection device 800 can not accurately detect the unknown word repeatedly uttered.

  Therefore, it is an object of the present invention to provide an unknown word detection device that detects unknown words repeatedly uttered in a speech document with high accuracy.

  One aspect of the present invention is an unknown word detection device for detecting an unknown word not included in the vocabulary of a speech recognition system from a speech document which is an input signal, which is a phoneme string of the speech document from the speech document. From a phoneme recognition unit that generates a document phoneme string, an unknown word candidate extraction unit that extracts a partial phoneme string from the voice document phoneme string, and extracts an unknown word candidate that is a similar partial phoneme string group; A speech recognition unit generating a speech recognition result including a language score; an unknown word feature calculated from a language score of a partial phoneme string included in the unknown word candidate from the unknown word candidate and the speech recognition result; An unknown word feature extraction unit for generating an unknown word candidate with unknown word features assigned to a candidate, and an unknown word feature of the unknown word candidate with the unknown word feature Unknown word candidates containing the unknown word determination unit determines whether the unknown word.

  According to the present invention, it is possible to detect an unknown word repeatedly uttered in an audio document with high accuracy.

FIG. 1 is a block diagram showing a configuration of an unknown word detection device 100. 6 is a flowchart showing the operation of the unknown word detection device 100. The figure which shows an example of an audio | voice document phoneme string. The figure which shows an example of an unknown word candidate group. The figure which shows the appearance of the extraction process of an unknown word candidate group. The figure which shows an example of a speech recognition result. The figure which shows an example of an unknown word feature-added unknown word candidate group. FIG. 10 is a block diagram showing the configuration of an unknown word detection device 800 of Non-Patent Document 1.

  Hereinafter, embodiments of the present invention will be described in detail. Note that components having the same function will be assigned the same reference numerals and redundant description will be omitted.

  Also in the following description, words that are not in the vocabulary of the speech recognition system are referred to as unknown words.

<The point of the present invention>
In the present invention, instead of detecting a section in which a phoneme string of an unknown word is uttered individually as an unknown word section as in Non-Patent Document 1, a section in which a similar phoneme string is uttered is detected Organize multiple sections. By putting together similar phoneme strings that appear repeatedly, the feature at the time of unknown word utterance that “consistently the language score of the speech recognition result is lowered in the section where the phoneme string corresponding to the unknown word is uttered” It becomes possible to capture. By determining the phoneme string indicating this feature as an unknown word, it is possible to accurately detect a word that has repeatedly appeared as an unknown word. The technique of “consistently lowering the language score” which appears over a plurality of sections in which similar phoneme strings are uttered can not be grasped only by separately detecting as an unknown word section, so the technique of Non-Patent Document 1 Is information that can not be used.

Embodiment
The unknown word detection device 100 will be described below with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing the configuration of the unknown word detection device 100. As shown in FIG. FIG. 2 is a flowchart showing the operation of the unknown word detection device 100. As shown in FIG. 1, the unknown word detection device 100 includes a phoneme recognition unit 110, an unknown word candidate extraction unit 120, a speech recognition unit 130, an unknown word feature extraction unit 140, and an unknown word determination unit 150.

  The unknown word detection apparatus 100 first performs phoneme recognition on a speech document, and converts the entire speech document into a phoneme string (speech document phoneme string) (S110). Next, a partial phoneme string that repeatedly appears in the obtained voice document phoneme string is extracted and used as an unknown word candidate (S120). Usually, a plurality of unknown word candidates are obtained. Speech recognition processing is performed on the same speech document (S130). In the section where each unknown word candidate appears, "whether or not the language score is always low" is calculated as a feature amount (S140), and it is determined using the feature amount whether the unknown word candidate is an unknown word or not (S140) S150).

  The processes of S110 and S120 and the process of S130 can be performed independently. That is, the order of S110 to S130 shown in FIG. 2 is an example.

  The input, output, and operation will be described below for each component.

<Phoneme Recognition Unit 110>
Input: voice document output: voice document phoneme string Phoneme recognition is performed on the input voice document to generate a voice document phoneme string which is a phoneme string of the entire voice document, and the voice document phoneme string is an unknown word candidate detection unit 120 It outputs to (S110).

  An example of a voice document phoneme string is shown in FIG. FIG. 3 shows a voice document phoneme string generated from the utterance content of "... Each phoneme is given an appearance interval (a start time and an end time with the start of the audio document as time 0) in the audio document.

<Unknown word candidate extraction unit 120>
Input: voice document phoneme string output: unknown word candidate group In the input voice document phoneme string, a similar partial phoneme string (hereinafter referred to as unknown word candidate) that appears repeatedly is extracted, and all extracted unknown word candidates (below, The unknown word candidate group is output to the unknown word feature extraction unit 140 (S120).

  An example of the unknown word candidate group is shown in FIG. Each unknown word candidate is a set of a plurality of similar partial phoneme strings, and each partial phoneme string is given an appearance interval in a speech document. For example, four partial phoneme sequences “rogumeru”, “ragumeru”, “rogumeru”, and “ragumeru” correspond to unknown word candidate 1. The start time and the end time of the appearance section of the first "rogumeru" coincide with the start time 4.31 of the phoneme "r" and the end time 4.88 of the phoneme "u" in FIG. 3, respectively.

  Extraction of the unknown word candidate group from the voice document phoneme string is performed, for example, by the following procedure.

1. From the voice document phoneme string (see FIG. 5A), all partial phoneme strings with a frequency of 2 or more having a length of 5 phonemes or more are listed (see FIG. 5B).

As a method of enumeration, for example, PrefixSpan method described in Non-Patent Document 3, which is an algorithm for enumerating subsequences having a length L or more and a frequency N or more from among sequences, is applied with L = 5 and N = 2. do it. Since it is rare that a short word of 4 phonemes or less will be an unknown word, it is preferable to set it to about L = 5. If N = 1, all five phoneme strings are extracted (that is, phoneme strings that do not repeatedly appear are also extracted). Therefore, N = 2 is set to extract a phoneme string that appears at least repeatedly.
(Reference Non-Patent Document 3: Jian Pei, Jiawei Han, Behzad Mortazavi-Asl and Helen Pinto, “PrefixSpan: Mining Sequential Pattern Efficient by Prefix-Projected Pattern Growth”, in Proceedings of ICDE, pp. 215-224, 2001.)

  Finally, a section from the start time of the top phoneme of each partial phoneme string obtained to the end time of the end phoneme is taken as the appearance section of the partial phoneme string.

2. Among the partial phoneme strings listed, clustering is performed so as to put together those which are close in editing distance between phoneme strings (see FIG. 5C).

Since it is unclear how many clusters it will be divided into, we use a clustering method that does not require specifying the number of clusters. Any existing method may be used as long as it is a clustering method that does not require designation of the number of clusters. For example, by applying the Chinese Whispers method described in Non-Patent Document 4 on the basis of the edit distance between phoneme strings, clustering can be performed such that partial phoneme strings close in edit distance are collected.
(Reference non-patent document 4: Chris Biemann, “Chinese Whispers-an Efficient Graph Clustering Algorithm and its Application to Natural Language Processing Problems,” in Proceedings of the first workshop on graph based methods for natural language processing, pp. 73-80, 2006.)

3. Each cluster of the clustering result is set as one unknown word candidate, and a set of obtained clusters is set as an unknown word candidate group.

  In addition to the above procedure, any method may be used as long as similar partial phoneme strings included in the audio document phoneme string can be extracted. For example, the BLAST algorithm or the like used when extracting similar partial sequences that repeatedly appear in the base sequence may be applied.

<Voice recognition unit 130>
Input: Speech document output: Speech recognition result Speech recognition is performed on the inputted speech document, and the generated speech recognition result is output to the unknown word feature detection unit 140 (S130).

  An example of the result of speech recognition is shown in FIG. In this example, "logmer" is an unknown word. Each word is given an appearance interval in the speech document and a language score calculated in the process of speech recognition. The language score is a real value representing likelihood as a sequence of words, and the higher the score, the more natural the sequence of words. In this example, the unknown word "log-meal" is recognized as two words "log" and "sleep". In other words, it is not recognized correctly at this stage.

  In addition, since it is judged whether it is an unknown word using the unknown word feature which shows whether the language score calculated by the speech recognition unit 130 is consistently low, the speech recognition unit 130 adds the detected unknown word. It is preferable to configure using a target speech recognition system.

<Unknown Word Feature Extraction Unit 140>
Input: Unknown word candidate group, Speech recognition result output: Unknown word candidate group with unknown word feature For each unknown word candidate of the inputted unknown word candidate group, unknown word feature referring to the speech recognition result inputted simultaneously Is calculated, and the calculated unknown word feature is assigned to the unknown word candidate. The generated unknown word feature-added unknown word candidate group is output to the unknown word determination unit 150 (S140).

  The unknown word feature is calculated from the language score group of the appearance point of the unknown word candidate (the appearance interval of the partial phoneme string included in the unknown word candidate), and is represented by a real value or a vector of multiple real values. It is If the unknown word feature has a low overall language score for multiple occurrences of the unknown word candidate (that is, is within a predetermined range indicating that the value is generally small), it is determined to be an unknown word . The determination method will be described later.

  For example, as in the example shown in FIG. 7, a two-dimensional vector in which the average and the standard deviation of the language scores at a plurality of appearance points of the unknown word candidate can be used as the unknown word feature. This is because the lower the mean and the lower the standard deviation, the lower the speech score consistently across multiple occurrences. The unknown word feature [0.003, 0.002] of unknown word candidate 1 in FIG. 7 is a set of the average and standard deviation of language scores of partial phoneme strings "rogumeru", "ragumeru", "rogumeru", and "ragumeru" .

  A value other than the standard deviation can also be used as a value representing the degree of variation of the language score at a plurality of occurrences of the unknown word candidate. For example, the difference between the maximum value and the minimum value of the language score of a plurality of occurrences may be used.

  In addition, as the unknown word feature, the maximum value of the language score of a plurality of appearance points may be used. In this case, they are represented by real numbers.

  The procedure for calculating the unknown word feature for one unknown word candidate in the case of using the language score average and the standard deviation as the unknown word feature is as follows.

1. The speech score of each partial phoneme string of the unknown word candidate is calculated with reference to the speech recognition result.

  Specifically, when the start time of the appearance section of the partial phoneme string of the unknown word candidate is T1 and the end time is T2, the language scores of all the words appearing in the section overlapping with T1 to T2 among the speech recognition results The average value of is taken as the language score of the partial phoneme string.

2. A two-dimensional vector obtained by arranging and calculating the average and standard deviation of the language scores of each partial phoneme string of the unknown word candidate is taken as the unknown word feature of the unknown word candidate.

<Unknown Word Determination Unit 150>
Input: Unknown word feature-added unknown word candidate group output: Unknown word group For each unknown word candidate of the unknown word feature-added candidate word group inputted, it is judged whether it is an unknown word or not based on the added unknown word feature Do. Of the unknown word candidates, those determined to be unknown words are output as the finally detected unknown word group (S150).

  As a method of judging whether it is an unknown word or not, a rule based on experience may be used, or a support vector machine or neural network etc. in which an unknown word feature calculated from an audio document in which an unknown word section is known is learned in advance. A classifier of may be used.

  When using the average and standard deviation values of the language score of unknown word candidate as the unknown word feature, for example, "If the average of language score is less than 0.01 and the standard deviation is less than 0.005, it is said as an unknown word" By using a rule, it is possible to detect a phoneme string whose language score is consistently low at a plurality of occurrences as an unknown word. In light of this rule, the unknown word candidate 1 in FIG. 7 is determined to be an unknown word.

  Moreover, when using the maximum value of the language score of the unknown word candidate as the unknown word feature, for example, by using the rule "If the maximum value of the language score is less than 0.02, it is regarded as an unknown word" It is possible to detect, as an unknown word, a phoneme string in which the language score is consistently lowered at a plurality of appearance points.

  According to the invention of this embodiment, the phoneme string repeatedly appearing by the unknown word candidate detection unit 120 is grouped as an unknown word candidate before determining whether it is an unknown word, and the unknown word feature extraction unit 140 The value representing whether or not the language score is consistently lowered in is calculated as an unknown word feature, and is assigned to the unknown word candidate. In order for the unknown word determination unit 150 to use this unknown word feature to determine whether an unknown word candidate is an unknown word, it is repeatedly performed with higher accuracy than in the prior art in which clustering is performed after each unknown word segment is detected. It becomes possible to detect unknown words that appear.

  In addition, since the system operator can accurately notice the occurrence of a serious problem that the keyword of the audio document does not appear in the recognition result by eliminating the omission of the unknown word detection, the unknown word has a vocabulary It is possible to appropriately implement measures for performing speech recognition again in addition. As a result, since accurate information can be obtained from a voice document by full-text search etc., the convenience for the system user is improved.

<Supplementary Note>
The apparatus according to the present invention is, for example, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected as a single hardware entity, or a communication device (for example, communication cable) capable of communicating outside the hardware entity. Communication unit that can be connected, CPU (central processing unit, cache memory, registers, etc. may be provided), RAM or ROM that is memory, external storage device that is hard disk, input unit for these, output unit, communication unit , CPU, RAM, ROM, and a bus connected so as to enable exchange of data between external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. Examples of physical entities provided with such hardware resources include general purpose computers.

  The external storage device of the hardware entity stores a program necessary for realizing the above-mentioned function, data required for processing the program, and the like (not limited to the external storage device, for example, the program is read) It may be stored in the ROM which is a dedicated storage device). In addition, data and the like obtained by processing of these programs are appropriately stored in a RAM, an external storage device, and the like.

  In the hardware entity, each program stored in the external storage device (or ROM etc.) and data necessary for processing of each program are read into the memory as necessary, and interpreted and processed appropriately by the CPU . As a result, the CPU realizes predetermined functions (each component requirement expressed as the above-mentioned,...

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. Further, the processing described in the above embodiment may be performed not only in chronological order according to the order of description but also may be performed in parallel or individually depending on the processing capability of the device that executes the processing or the necessity. .

  As described above, when the processing function in the hardware entity (the apparatus of the present invention) described in the above embodiment is implemented by a computer, the processing content of the function that the hardware entity should have is described by a program. Then, by executing this program on a computer, the processing function of the hardware entity is realized on the computer.

  The program describing the processing content can be recorded in a computer readable recording medium. As the computer readable recording medium, any medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, etc. may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (Rewritable), etc. as magneto-optical recording medium, MO (Magneto-Optical disc) etc., as semiconductor memory EEP-ROM (Electronically Erasable and Programmable Only Read Memory) etc. Can be used.

  Further, this program is distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD, a CD-ROM or the like in which the program is recorded. Furthermore, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, at the time of execution of the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of this program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer Each time, processing according to the received program may be executed sequentially. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to the computer It may be Note that the program in the present embodiment includes information provided for processing by a computer that conforms to the program (such as data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, in this embodiment, the hardware entity is configured by executing a predetermined program on a computer, but at least a part of the processing content may be realized as hardware.

100 unknown word detection device 110 phoneme recognition unit 120 unknown word candidate extraction unit 130 speech recognition unit 140 unknown word feature extraction unit 150 unknown word determination unit

Claims (5)

An unknown word detection device for detecting an unknown word not included in the vocabulary of a speech recognition system from an audio document which is an input signal, comprising:
A phoneme recognition unit that generates, from the voice document, a voice document phoneme string that is a phoneme string of the voice document;
An unknown word candidate extraction unit that extracts a partial phoneme string from the voice document phoneme string and extracts an unknown word candidate that is a similar partial phoneme string group;
A voice recognition unit that generates a voice recognition result including a language score from the voice document;
An unknown word feature added unknown word feature with the unknown word feature calculated from the language score of the partial phoneme string included in the unknown word candidate from the unknown word candidate and the speech recognition result Word feature extraction unit,
An unknown word detection device comprising: an unknown word determination unit that determines whether or not the unknown word candidate is an unknown word based on unknown word features of the unknown word candidate with the unknown word feature. The unknown word detection device according to claim 1, wherein
The unknown word feature is a combination of the average and standard deviation of language scores of partial phoneme strings included in the unknown word candidate, or the average of language scores of partial phoneme strings included in the unknown word candidate and the maximum of the language score Unknown word detection device which is a set of difference between value and minimum value. The unknown word detection device according to claim 1 or 2, wherein
The unknown word candidate extraction unit
A partial phoneme string that appears at a predetermined frequency or more is extracted from the voice document phoneme string, the extracted partial phoneme strings are clustered to obtain similar partial phoneme string groups, and the determined partial phoneme string groups are extracted as unknown word candidates Unknown word detection device. An unknown word detection apparatus including a phoneme recognition unit, an unknown word candidate extraction unit, a speech recognition unit, an unknown word feature extraction unit, and an unknown word determination unit converts a speech document as an input signal into a vocabulary of a speech recognition system An unknown word detection method for detecting an unknown word not included, comprising:
A phoneme recognition step in which the phoneme recognition unit generates a speech document phoneme sequence which is a phoneme sequence of the speech document from the speech document;
An unknown word candidate extraction step of the unknown word candidate extraction unit extracting a partial phoneme string from the voice document phoneme string and extracting an unknown word candidate that is a similar partial phoneme string group;
A voice recognition step in which the voice recognition unit generates a voice recognition result including a language score from the voice document;
The unknown word feature in which the unknown word feature extraction unit assigns the unknown word feature calculated from the language score of the partial phoneme string included in the unknown word candidate from the unknown word candidate and the speech recognition result to the unknown word candidate An unknown word feature extraction step of generating a tagged unknown word candidate;
An unknown word determining step of determining whether the unknown word candidate is an unknown word based on the unknown word feature of the unknown word candidate with the unknown word feature.   A program for causing a computer to function as the unknown word detection device according to any one of claims 1 to 3.

Images