TW201638931A - Speech recognition device and rescoring device - Google Patents

Abstract

A speech recognition device and a rescoring device are constructed that reduce recognition errors, that allow consideration for contexts longer than a discriminative language model and that are robust to unknown contexts to some extent. In the speech recognition device and the rescoring device utilizing a discriminatively trained language model, the discriminatively trained language model is trained based on alignment between a correct sequence and a hypothesis sequence and the discriminatively trained language model is constructed based on a recurrent neural network.

Description

Speech recognition device and adjustment device

The present invention relates to speech recognition devices and adjustment devices, and more particularly to the use of a language model based on a recurrent neural network.

In speech recognition, the performance improvement is well known by using the Recurrent Neural Network (RNN) in the Language Model (LM) (RNN-LM). For example, it is described in T. Mikolov, M. Karafiat, L. Burget, J. Cernocky and S. Khudanpur, "Recurrent neural network based language model", INTERSPEECH Journal 2010 1045-1048 .

Further, it is known that the recognition result candidate adjusted by the voice recognition is adjusted by the n-gram (N-linked) model as the basic recognition language model. For example, it is described in B. Roark, M. Saraclar, M. Collins, and M. Johnson, "Discriminative language modeling with conditional random fields and the perceptron algorithm (produced using a discriminative language model of conditional random fields and perceptron algorithms) ACL Journal 2004, pp. 47-54 and Japanese Patent Laid-Open Publication No. 2014-089247.

Using a language model that is usually n-gram, you can't consider long articles Text. In contrast, using RNN to LM, in principle, can maintain an infinite length of context. This mode is shown in Figure 1. The input x is a 1-of-N (1 of N) representation in a dictionary composed of N words. The output y is an afterthought probability corresponding to the N word. There is a low-dimensional vector s in the hidden layer. The projection row U links the input layer to the hidden layer. The projection row V connects the hidden layer to the output layer. Copying is in the hidden layer of the previous moment to the input layer, thereby maintaining the context. By using the LM using this configuration, it is possible to generate a better recognition hypotheses by considering a context longer than the context in which the LM of the n-gram can be considered. Also, since the mapping to the hidden layer is performed on a low-dimensional vector, the similarity between words is considered. For example, the words "dog" and "cat" can be replaced according to the context, in which case the consine (cosine) similarity between those vectors s becomes higher.

RNN-LM, compared to the n-gram method using the conventional comparison table, is mainly used for rescoring because of the long processing time. The structure used for rescoring is shown in Fig. 2. The recognition means 4 receives the speech 1 as an input, uses the acoustic model 2 and the language model 3 for recognition, reconciles the plural candidate columns, and provides a result of the rescoring as the output recognition result 5. On the other hand, the adjustment means 6 receives the recognition result 5 as an input, and uses the language model 7 for adjustment to return the recognition result 8 of the candidate of the descending order of the approximate descending order. The language model for adjustment 7 is RNN-LM. Using the language model 7 in which the long context can be considered, it can be expected that the recognition performance of the corrected recognition result 8 becomes better than the recognition result 5.

Further, since the word recognizable by the recognition means 4 has a possibility that the recognition result 5 appears, the term of the word to be recognized by the adjustment means 6 preferably covers the term of the recognition means 4. But by modeling the unknown (unk) as a group In addition, the number of terms of the adjustment means 6 can be reduced as compared with the identification means 4.

In the RNN-LM, the probability of the next word w _t+1 is calculated based on the word lines w ₁ , w ₂ , ... w _t so far. The word to be recognized contains the word | V |, and each word is given a different word number. The word number is denoted by n (but 1≦n≦| V |). Further, the word number may be given based on the result of classifying each word by a certain standard. In speech, when the word number of the word appearing at _{t is} provided as c _t , the evaluation function of the learning in the cross entropy (CE) reference is provided as in equation (1).

C is a word column (correct answer column) appearing in speech converted into a word number column, and C _t is a word number of the word of the tth. That is, C is a sequential column of rows c ₁ , c ₂ , c ₃ , . δ is the delta of Kronecker. y usually uses the Softmax function represented by equation (2).

However, a is activation, such as a = V‧s _t . The learning rule is obtained as a differential F ^CE as in equation (3).

At the time of learning, the probability of the next word y _t (n) obtained by inputting the current word x _t (c _t )=1 is calculated. Since the correct answer is provided by δ(n,c _t ), the difference between the correct answer δ(n,c _t ) and the probability of the current time point y _t (n) is the error ε _t (n), by inverse propagation, Update the parameters of the NU (neural network).

The parameters of the NN to be learned include at least one element of the projection row U and/or V of Fig. 1. Further, the parameters of the NN to be learned may include components representing the vector of the added offset as the projections of the projection rows U and V are generated. The inverse propagation is performed, for example, in order to find a parameter that minimizes the error ε _t (n). Further, as the specific method and calculation formula of the reverse propagation, well-known methods and calculation formulas can be used.

A specific example of the conventional adjustment means 6 is an example of a language model using recognition. This is based on learning materials, using the correct answer or N-best recognition results to learn. The N-best recognition result is a result of arranging the recognition results of the upper N candidates in order of the scores, for example, from all the candidates.

The score, for example, is expressed as a function of the acoustic model score and the language model score, such as the weighted sum of these. The language model is identified, and the candidate with the least recognition error among the correct answer column or the N-best recognition result is the correct answer, and the candidate with the most recognition error among the N-best recognition results is an incorrect answer, according to the respectively included n -gram, learning with (average) perceptron algorithm. An example of such a method is described in the above-mentioned Roark 2004 and Japanese Patent Laid-Open Publication No. 2014-089247.

The disadvantage of such a conventional method, first, is that it does not consider the point of exceeding the context of the n-gram. That is, the model of the bi-gram (binary model) does not consider the context length beyond the bi-gram, and the model of the tri-gram (ternary model) does not consider the context length exceeding the tri-gram.

Second, there is also a problem that the n-gram not shown by the N-best recognition result cannot be scored. Therefore, it is effective in the case where the learning materials are close to the identification areas of the evaluation materials, but in the case of these separations (for example, the learning materials are the news notes, the evaluation materials are the contents of the free e-mail (email) content production). Etc.), it may not work.

Thirdly, in the case of being used in combination with the RNN-LM, there is a problem that the second adjustment must be performed. That is, in addition to the adjustment (using the recognition language model) generated by the adjustment means 6, before or after, adjustment using the RNN-LM is necessary.

The invention is formed to solve the above problems, and aims to construct a speech recognition device and an adjustment device. Due to the effect of introducing recognition in the RNN-LM, the recognition error is reduced, and a context longer than the recognized language model can be considered, even for The unknown context is also somewhat robust.

In order to solve the above problems, the speech recognition apparatus according to the present invention is a speech recognition apparatus that memorizes the learned language model, and the language model of the recognition learning is learned based on the alignment in the word unit between the correct answer column and the candidate column. The language model for learning recognition is based on a recurrent neural network. The alignment can be determined, for example, by using a dynamic programming method to achieve maximum uniformity of character strings.

Further, according to the adjustment apparatus of the present invention, the language model of the recognition learning is used, the adjustment means for the candidate column of the speech recognition is adjusted, the language model of the learning is recognized, and the learning is performed based on the alignment in the word unit between the correct answer column and the candidate column. , the learning language model is based on the recurrent neural network Neural network).

In the adjustment device, a weighted average between the parameters of the original language model and the parameters of the language model for learning is obtained.

Each word in the candidate column can be individually attached with reliability. While learning to identify the language model of learning, recognizing the language model of learning can also make words with higher reliability more important to learn.

According to the original language model, the first result including the candidate column is obtained, and the second result including the candidate column is obtained based on the language model of the recognition learning, and the first result and the second result may be integrated.

According to the present invention, a speech recognition apparatus and an adjustment apparatus are provided to reduce recognition errors, and it is possible to consider a context longer than the recognition language model, even if it is somewhat robust to an unknown context.

1‧‧‧ voice

2‧‧‧Acoustic model

3‧‧‧ language model

4‧‧‧Recognition means

5‧‧‧N-best recognition results

6‧‧‧Adjustment means

7‧‧‧ language model

8‧‧‧ Rearranged N-best recognition results

10‧‧‧Voice recognition device

20‧‧‧ arithmetic means

21‧‧‧Recognition means

22‧‧‧Alignment means

23‧‧‧ Identification learning methods

24‧‧‧Adjustment means

25‧‧‧ Weighted means

26‧‧‧ Results integration

30‧‧‧ memory means

31‧‧‧Acoustic model

32‧‧‧1st language model

33‧‧‧N-best recognition results

34‧‧‧correct answer mark

35‧‧‧2nd language model

36‧‧‧ original language model

37‧‧‧Word reliability

38‧‧‧Incorrect answer alternate

40‧‧‧Voice input means

50‧‧‧ Results output means

60‧‧‧ voice

70‧‧‧ Rearranged N-best recognition results

121‧‧‧Recognition means

123‧‧‧ID learning means

224‧‧‧1st adjustment means

225‧‧‧2nd adjustment means

270‧‧‧ Rearranged N-best recognition results

271‧‧‧ Rearranged N-best recognition results

322‧‧‧Alignment means

323‧‧‧Model learning methods

325‧‧ ‧ weighting means

335‧‧‧2nd language model

421‧‧‧Recognition means

423‧‧‧ Identification learning methods

432‧‧‧Update language model

N-best‧‧‧ recognition results

[Fig. 1] is a diagram showing a language model according to a Recurrent neural network; [Fig. 2] is a functional block diagram of a conventional speech recognition apparatus; [Fig. 3] shows a correct answer column and An alignment diagram between candidate columns; [Fig. 4] is a hardware configuration example of the voice recognition device according to the first embodiment; [Fig. 5] is a flowchart of processing for learning by the voice recognition device of Fig. 4 [Fig. 6] The speech recognition apparatus of Fig. 4 implements a processing flow for application [Fig. 7] is a functional block diagram of the speech recognition apparatus of Fig. 4; [Fig. 8] is a functional block diagram of the speech recognition apparatus according to the second embodiment; [Fig. 9] is based on the third Functional block diagram of the speech recognition apparatus of the embodiment; [Fig. 10] is a functional block diagram of the speech recognition apparatus according to the fourth embodiment; [11] is a functional block diagram of the speech recognition apparatus according to the fifth embodiment And [12th] is a functional block diagram of the speech recognition apparatus according to the sixth embodiment.

Hereinafter, embodiments of the invention will be described based on additional drawings.

[First Embodiment]

In the first embodiment, the RNN-LM according to the identification reference is used. The object of the present invention is to improve recognition performance by learning to learn RNN-LM. Since one of the important purposes of the language model is to convert the speech to be recognized into the correct text, it is desirable to construct a language model that can correct the conventional speech recognition result.

Thus, in addition to the above-mentioned correct answer mark c _t , it is also considered to use the hypotheses h _t generated by the speech recognition to identify the constructed RNN-LM. The objective function at this time is considered to use the approximate ratio in the word level as in the following formula (4). In addition, an evaluation function that is sufficient for identifying learning and maximizing the amount of mutual data and the smallest phoneme error can also be used.

H is a sequence of rows h ₁ , h ₂ , h ₃ ..., and β is a discount factor. Similarly, when a is differentiated, the learning rule of the following formula (5) is obtained.

This sequence is specified using Figure 3. Now, the correct answer is listed as A, B, C, and D, considering the insertion (I), shedding (@), and replacement (S) errors. The correct answer column C starts to align with the speech recognition result H, and the correspondence relationship as shown in Fig. 3(a) is obtained.

In the normal RNN-LM learning, the weights of A, B, C, and D are 1, respectively, and the error ε is calculated, and the parameters of the RNN-LM are updated according to the equation (3). On the other hand, in the first embodiment, as shown in the third figure (b), in order to place and learn a higher weight of the recognition result of the incorrect answer than the recognition result of the correct answer, when the correct answer is discounted (this In the example, the weights of the words A and D). In this example, the weight of the correct answer is reduced by only the discount coefficient β from the weight 1 of the incorrect answer, whereby the learning weights of B and C can be awarded. This is the intent of equation (5).

At this time, special errors must be dealt with for insertion errors. For example, for the correct answer column of the third (a) figure, the candidate column of the ABCID in which the word I is erroneously inserted is obtained. In this case, the correct answer word corresponding to I does not exist. In this case, for example, I may be ignored as I, and the candidate column may be "ABCD" processing. For example, the third (b) diagram may be processed by considering the word C at the previous time.

The number of candidates is 2 or more (for example, the N-best recognition result). Same separately It is also possible to sample each candidate. For example, in the case of 2-best, the alignment processing of FIG. 3 is performed on the candidate of the first name, the parameters of the RNN-LM are updated, and the alignment processing of FIG. 3 is also performed for the candidates of the second name. , update the parameters of RNN-LM.

In Fig. 4, an example of the hardware configuration of the voice recognition device 10 of the first embodiment of the present invention is shown. The voice recognition device 10 can be constituted by, for example, a well-known computer. The speech recognition apparatus 10 includes an arithmetic means 20, a memory means 30, a voice input means 40, and a result output means 50. The computing means 20 includes a processor, and the memory means 30 includes a semiconductor memory and a memory medium such as an HDD (hard disk drive). The program (not shown) is stored in the memory means 30, and the program is executed by the arithmetic means 20 to realize the functions of the voice recognition device 10 described in the present specification. This program can also be recorded in a non-transitory information memory medium.

The voice input means 40 is, for example, a microphone, and accepts an input of a voice 60 including a word column. Alternatively, the voice input means 40 may be an electronic data input device, and the input of the voice 60 may be accepted as electronic data. The result output means 50 is, for example, a liquid crystal display, a printer, a network interface or the like, and outputs a rearranged N-best recognition result 70.

In the fifth and sixth figures, a flowchart showing the processing performed by the voice recognition device 10 is shown.

Figure 5 is a flow chart of learning. When the voice recognition device 10 operates in accordance with the flow of FIG. 5, the voice recognition device 10 may be a voice recognition learning device. First, the voice recognition device 10 receives an input of the voice 60 for training (step S1). Next, the voice recognition device 10 performs voice recognition on the voice 60. The N-best recognition result is obtained (step S2). Next, the speech recognition apparatus 10 aligns each candidate column included in the N-best recognition result with the correct answer column (step S3). Next, the voice recognition device 10 recognizes the learning language model based on the result of the alignment (step S4). Next, the voice recognition device 10 outputs a language model for learning recognition (step S5). Further, it is common to use a plurality of correct answer columns for learning, but the present invention can be implemented if there are at least one correct answer column and at least one candidate column.

Figure 6 is a flow chart of an application. When the voice recognition device 10 operates in accordance with the flowchart of FIG. 6, the voice recognition device 10 may be referred to as an adjustment device. First, the voice recognition device 10 accepts the input of the voice 60 to be recognized (step S6). Next, the voice recognition device 10 performs voice recognition processing on the voice 60 to obtain an N-best recognition result (step S7). Next, the speech recognition apparatus 10 adjusts each candidate column included in the N-best recognition result based on the language model of the recognition learning (step S8). Next, the voice recognition device 10 outputs the rearranged N-best recognition result 70 based on the result of the adjustment (step S9). Further, in general, a plurality of candidate columns are output, but at least one candidate column is output, and the present invention can be applied to the present invention.

In Fig. 7, a functional block diagram of the voice recognition device 10 is shown. The arithmetic means 20 of the voice recognition device 10 functions as the recognition means 21, the alignment means 22, the recognition learning means 23, and the adjustment means 24. Further, in the memory means 30 of the voice recognition device 10, the acoustic model 31, the first language model 32, the N-best recognition result 33, the correct answer mark 34, and the second language model 35 can be stored. The first language model 32 is, for example, a language model for speech recognition, and the second language model 35 is, for example, a language model for adjustment.

The identification means 21, the acoustic model 31, and the first language model 32 may be of a conventional configuration. In other words, the recognition means 4 of Fig. 2 may use the acoustic model 2 and the language model 3 for recognition.

In the configuration of Fig. 7, the correct answer mark 34, the alignment means 22, the recognition learning means 23, and the second language model 35 are added to the conventional configuration of Fig. 2.

The alignment means 22 aligns the N-best recognition result 33 with the correct answer mark 34. The term "to align" means, for example, each word included in the correct answer column and each word included in the candidate column. For example, in the example of the third (a) diagram, the words A, B, and D corresponding to the candidate column are respectively assigned to the words A, B, and D in the correct answer column. Also, regarding the words that are not corresponding, it is considered that insertion or dropping occurs. For example, in the example of Fig. 3(a), the word C falls off and the word I is inserted. Alignment, for example, using dynamic programming, can find the maximum agreement.

The recognition learning means 23 performs the recognition learning to generate or update the second language model 35 based on the result of the alignment processing. The second language model 35 is constructed based on RNN. The identification learning of the second language model 35 is performed, for example, by inverse propagation using the above equation (5), thereby updating the parameters of the RNN. This can be done in the same way as the reverse propagation in the conventional learning. Then, the second language model 35 is learned based on the alignment between the correct answer column and the candidate column.

The adjustment means 24 adjusts the N-best recognition result 33 based on the second language model 35, and obtains the rearranged N-best recognition result 70. The term "adjustment" refers to, for example, re-rating a candidate column for a score. The initial adjustment is based on the score generated by the recognition means 21 in the first embodiment.

For example, the scores of each candidate are composed of acoustic model scores and language models. When the score is expressed, the adjustment means 24 replaces the language model score of each candidate included in the N-best recognition result 33, and becomes a language model score estimated using NN. Alternatively, obtain a weighted average of the scores of the original language model. Then, by using the second language model 35 for recognition learning, adjustment of the error tendency in consideration of the speech recognition in the recognition means 21 can be performed.

For example, in a car navigation system, when a newsletter service generated based on a voice recognition technology is created, a more accurate text data can be obtained by learning the error tendency for a specific user in advance. Alternatively, if the predetermined language, vocabulary or syntactically restricted speech is used to create the second language model 35 with a suitable command, the first language model 32 may have the advantage of using a common object.

As described above, since the RNN-LM obtains the effect of the recognition learning, the N-best recognition result can be corrected more effectively than the conventional composition.

Compared to the case of combining the recognition language modules in the conventional configuration, it is possible to analogize the context that does not appear in the learning material in the present invention, for example, considering that the difference in the area becomes more robust. For example, the words "dog" and "cat" can be replaced according to the context, but in the case of mapping such words to the low-order vector s, the consine (cosine) similarity between those vectors becomes high. Therefore, the learning effect when the "dog" appears in the learning material becomes similar to the learning effect when the "cat" appears, and an effect similar to the proximity of the context including the exchangeable word can be obtained. This effect cannot be obtained with the conventional recognition language model. Moreover, the specific dimension of the vector s can generally be appropriately designed to be smaller than |V|.

Moreover, compared with the conventional configuration in which the RNN-LM and the recognition language model are used in combination, there is an advantage that the adjustment is completed once. Of course, in the latter part, the use of other recognition language models can improve performance. For example, the adjustment means 24 In the subsequent stage, an additional adjustment means is provided, and the additional adjustment means may perform adjustment of the rearranged N-best recognition result 70 based on another identification language model.

Further, in each of the embodiments of the present specification, learning and application are performed using a single device, but learning and application may be performed using different devices (different computers, etc.). For example, the device for learning may not have the adjustment means 24, and the device for application may not include the alignment means 22 or the recognition learning means 23. Further, the device for application may be, for example, a conventional speech recognition device (such as the device shown in Fig. 2) (however, the adjustment uses the second language model 35).

[Second embodiment]

The first embodiment maintains the second language model 35 using the recognition learning. In the second embodiment, the weighted average parameter between the original language model 36 and the second language model 35 is used. According to such a configuration, the influence of over-learning can be reduced.

The original language model 36 is the second language model 35 before the NN parameter update generated by the recognition learning means 23, that is, the same as the second language model 35 in the initial state. In other words, the second language model 35 is generated for the original language model 36 by performing recognition learning.

The configuration of the second embodiment is shown in Fig. 8. The weighting means 25 is added. The arithmetic means 20 of the speech recognition apparatus 10 functions as a weighting means 25. The weighting means 25, the parameters of the weighted average original language model 36 and the parameters of the second language model 35. For example, in the configuration of Fig. 1, the equation (6) is shown.

{ U , V }← τ { U ^CE , V ^CE }+(1- τ ){ U ^LR , V ^LR } (6)

U ^CE and V ^CE are parameters of a model learned by cross entropy, and U ^LR and V ^LR are parameters of the model to be learned. τ system smoothing coefficient. Further, in general, each language model includes a plurality of parameters, but if a language model including at least one parameter is used, weighted averaging can be performed.

As described above, by obtaining the average of the original language model 36 and the second language model 35 of the recognition learning, the influence of over-learning which is easily caused in the recognition learning is reduced, and a more stable recognition learning effect is produced.

[Third embodiment]

In the third embodiment, an RNN-LM based on an identification criterion using word reliability is used.

The configuration of the third embodiment is shown in Fig. 9. In this example, instead of the identification means 21 of the first and second embodiments, the identification means 121 is provided, and the identification learning means 123 is provided instead of the identification learning means 23 of the first and second embodiments. The arithmetic means 20 of the speech recognition apparatus 10 functions as the identification means 121 and the identification learning means 123.

The recognition means 121 outputs the N-best recognition result 33, and each word included in the N-best recognition result 33 requires reliability, and the output is the word reliability 37. The word reliability 37 is, for example, stored in the memory means 30 of the voice recognition device 10. The recognition learning means 123 generates or updates the second language model 35 based on the word reliability 37 based on the result of the entire column processing.

Most methods for obtaining word reliability are well known. For example, the approximate likelihood of a particular candidate at a certain time can be used as the word reliability of the specific candidate for the proportion of the total candidate in the total candidate. For example, when each word candidate is assumed to be w _t ⁱ (1≦i≦I) at time t, the approximate p(w _t ⁱ ) of each word candidate can be expressed as:

Because the error of considering high word reliability is more serious than the error of low word reliability, the discount ratio can be changed according to the high reliability of the word. For example, it is calculated by the following formula (7).

ν _t is the word reliability, 0≦ ν _t ≦1.

When the word of the incorrect answer has the highest reliability (for example, ν _t =1), the greatest weight (for example, 1) is learned. On the other hand, when the word of the incorrect answer has the lowest reliability (for example, ν _t =0), since the learning according to the word has no effect, the weight (e.g., 1-β) having the same discount as the correct answer is learned.

Therefore, in the third embodiment, each word of the candidate column has reliability, and the second language model 35 is more focused on learning than words having higher reliability.

According to such a configuration, even if the same word error has different weight learning, an important word can be learned with a greater weight. As described above, by using the learning of the word reliability, it is possible to learn according to the significance of the recognition error.

Further, in the ninth embodiment, the weighting means 25 and the original language model 36 are provided in the same manner as the second embodiment, but the same as in the first embodiment, these are not provided.

[Fourth embodiment]

In the first and second embodiments, the results of the learning are integrated at the language model level. On the other hand, in the fourth embodiment, the result of the learning is integrated by the recognition result level.

The configuration according to the fourth embodiment is shown in Fig. 10. The first adjustment means 224 and the second adjustment means 225 are provided instead of the adjustment means 24 of the first and second embodiments. The arithmetic means 20 of the voice recognition device 10 may function as the first adjustment means 224 and the second adjustment means 225.

The first adjustment means 224 adjusts the N-best recognition result 33 based on the original language model 36 to obtain the rearranged N-best recognition result 270 (first result). The second adjustment means 225 adjusts the N-best recognition result 33 based on the second language model 35 of the recognition learning to obtain the rearranged N-best recognition result 271 (second result). The rearranged N-best recognition results 270 and 271 may be stored in the memory means 30 of the speech recognition apparatus 10.

Further, in the fourth embodiment, the result integration means 26 is set. The arithmetic means 20 of the speech recognition apparatus 10 may function as the result integration means 26. The resulting integration means 26 integrates the rearranged N-best identification results 270 and 271 to obtain a final rearranged N-best recognition result 70.

For example, it is also possible to compare the candidates according to the scores and select the candidates for the high scores.

Or, integration can be done according to a minority majority. A small number of specific application methods that can be applied to a large number can be arbitrarily designed. For example, a minority of three or more systems can be used for a large number of subordinates. If each system outputs a different candidate, the score can be compared.

Also, at the time of integration, it is also possible to appropriately discount the scores of any language model. For example, for a language model that is less reliable, a weight smaller than 1 (for example, 0.8) is applied, and each candidate can be integrated based on the score comparison.

Of course, such integration processing can also be applied to the composition using word reliability as in the third embodiment.

As described above, it is more robust to make adjustments independently using a complex language model than in the case of using a single (or averaged) language model.

[Fifth Embodiment]

In the fifth embodiment, in the recognition learning of the language model, only the composition of the incorrect answer hypothesis is used.

In the first to fourth embodiments, both the candidate of the correct answer and the candidate of the incorrect answer are used for learning. However, in order to make it easier to identify learning, consider using a language model that only learns based on incorrect answer assumptions.

The configuration according to the fifth embodiment is shown in Fig. 11. Instead of the alignment means 22 of the second embodiment, an alignment means 322 is provided. The alignment means 322 extracts the incorrect answer candidate 38 from the N-best recognition result 33 and aligns it.

The model learning means 323 is provided instead of the identification learning means 23 of the second embodiment. The model learning means 323 learns by using the incorrect answer candidate 38 based on the result of the alignment, and generates or updates the second language model 335. This learning process body does not have to be implemented according to identification methods. For example, the model learning means 323 learns by updating the parameters of the NN according to the equation (3).

Further, instead of the weighting means 25 of the second embodiment, the weighting means 325 is provided. Weighting means 325, in order to punish the incorrect answer candidate, the weighting is flat The parameters of the original language model 36 and the parameters of the second language model 335. For example, the weighting means 325 weights the parameter of the second language model 335 to a negative number (that is, τ of the equation (6) becomes larger than 1).

Here, even if the learning ontology of the language model is not recognized, the speech recognition apparatus 10 can perform the recognition learning by combining the original language model and the language model based on the non-correct candidate learning.

The calculation means 20 of the speech recognition apparatus 10 may function as the alignment means 322, the model learning means 323, and the weighting means 325. Further, the incorrect answer candidate 38 and the second language model 335 may be stored in the memory means 30 of the voice recognition device 10.

As described above, in addition to the original language model 36, by using the second language model 335 that is only learned based on the incorrect answer, the method of learning the language model remains unchanged, and the effect of simple recognition learning can be obtained.

[Sixth embodiment]

In the first embodiment, the first language model 32 for speech recognition does not become an object of recognition learning. On the other hand, in the sixth embodiment, the language model for speech recognition is learned using RNN-LM.

The configuration according to the sixth embodiment is shown in Fig. 12. In the sixth embodiment, the identification learning means 423 is provided instead of the identification learning means 23 of the first embodiment. The recognition learning means 423 performs the recognition learning update language model 432 based on the result of the alignment processing. Further, in place of the identification means 21 of the first embodiment, the identification means 421 is provided. The recognition means 421 performs the speech recognition output N-best recognition result 33 based on the language model 432 of the recognition learning.

According to such a configuration, as in the first embodiment, it is also possible to obtain To identify the effects of learning.

S1‧‧‧ input voice

S2‧‧‧According to speech recognition, obtain N-best recognition result

S3‧‧ Aligned with N-best recognition results

S4‧‧‧ Implementation of identification learning

S5‧‧‧ Language model for output recognition learning

Claims (8)

A speech recognition device for memorizing and learning a language model; wherein the language model of the identification learning is learned according to alignment in a word unit between a correct answer column and a candidate column, and the language model of the identification learning is based on a recurrent neural network It is composed of (circular neural network). The speech recognition apparatus according to claim 1, wherein a weighted average between a parameter of the original language model and a parameter of the language model for the learned learning is obtained. The speech recognition apparatus according to claim 1, wherein each of the words in the candidate column has reliability; and the language model of the recognition learning further focuses on a word having higher reliability. The speech recognition apparatus according to claim 1, wherein the first result including the candidate column is obtained based on the original language model, and the second result including the candidate column is obtained based on the language model of the identification learning; The first result and the second result described above. An adjusting device, which uses a language model of recognition learning to adjust a candidate column of speech recognition; wherein the language model of the identification learning is learned according to alignment in a word unit between a correct answer column and a candidate column; and the above-mentioned identification learning The language model is constructed based on a recurrent neural network. The adjusting device according to claim 5, wherein the weighted average between the parameters of the original language model and the parameters of the language model for the learned learning is obtained. The adjustment device according to claim 5, wherein each of the words in the candidate column has reliability; and the language model of the recognition learning further focuses on a word with higher reliability. The adjustment device according to claim 5, wherein the first result including the candidate column is obtained based on the original language model; the second result including the candidate column is obtained based on the language model of the identification learning; and the first 1 result and the above 2nd result.