WO2022143723A1 - Voice recognition model training method, voice recognition method, and corresponding device - Google Patents

Abstract

Disclosed is a voice recognition model training method, a voice recognition method, and a corresponding device. In the present solution, a migration model to be trained in a target domain can be created on the basis of an original voice recognition model trained in a source domain, a feature data set is extracted from sample data of the target domain, each piece of feature data in the feature data set carrying environmental factors capable of reflecting similar possibilities of different sample data in the same voice physical attribute, and a training set in the feature data set is substituted into the migration model as an input parameter for iterative training, so as to obtain a target voice recognition model.

Description

Speech recognition model training method, speech recognition method and corresponding device

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims the priority of Chinese patent application CN202011629211.3, filed on December 31, 2020, entitled "Speech Recognition Model Training Method, Speech Recognition Method and Corresponding Device", the entire contents of which are incorporated into this disclosure by reference.

technical field

The present disclosure relates to the technical field of artificial intelligence, and in particular, to a speech recognition model training method, a speech recognition method and a corresponding device.

Background technique

With the advent of the 5G era with large bandwidth and low latency, the application of real-time media such as audio and video will definitely become the mainstream in the future, and the massive data generated will be of great help to the analysis of user habits.

Under the traditional machine learning framework, the basic process is to learn a classification model on the basis of given sufficient training data, and then use this model to complete the classification and prediction of the test data. However, with the advent of the 5G era, various real-time media-related applications have exploded, which has brought about the refinement of various fields. It is very difficult to obtain a large amount of training data in these newly divided fields. How to use these limited and refined field data to effectively analyze user habits and make targeted optimization to improve user experience is a huge challenge before contact center products.

At present, the traditional training process of machine learning requires a large amount of training data, and it is difficult to obtain sufficient training data for some sub-fields (such as announcement sound). Even if a large amount of training data is obtained, a lot of manpower and material resources are needed to label the data. There are many offices in the contact center. For different offices, the announcement sound data is different, which will cause the existing model to have a low recognition rate for new scenarios. The previous practice was to start training from scratch by adding new announcement sounds on the basis of the previous training data. Not only did the training time take a long time, but also because the newly added data had less obvious characteristics than the previous massive data, the recognition effect was average. Although the training data of the new scenario is more or less different from the source data, there are still some instances in the source training data that are suitable for the new application scenario. Directly discarding the existing training model and starting the training from scratch results in the duplication of some data. Huge waste of training and resources.

Therefore, how to improve the training efficiency of the speech recognition model and how to improve the speech recognition efficiency has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

The purpose of one or more embodiments of the present disclosure is to provide a speech recognition model training method, speech recognition method and corresponding apparatus, which can reduce the training cost and training time of the speech recognition model, and improve the training efficiency and model recognition rate.

To solve the above technical problems, one or more embodiments of the present disclosure include the following first to eighth aspects.

A first aspect provides a speech recognition model training method, comprising: creating a migration model to be trained in a target domain based on an original speech recognition model trained in a source domain, wherein the output layer of the migration model is different from the original speech recognition model The output layer of the model; extracts a feature data set from the sample data of the target domain, wherein each feature data in the feature data set carries environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; The centralized training set is substituted into the migration model as input parameters for iterative training, and the target speech recognition model is obtained; wherein, during the iterative training process, the output layer of the migration model adjusts the weights randomly, and the weights of other layers of the migration model remain unchanged.

In a second aspect, a speech recognition method is proposed, including: determining speech data to be recognized; extracting characteristic data from the speech data, the characteristic data carrying an environment that can reflect the similar possibility of different sample data in terms of the same speech physical properties factor; the feature data is substituted into the target speech recognition model for speech recognition, and the target speech recognition model is trained based on the speech recognition model training method of the first aspect.

In a third aspect, a speech recognition model training device is proposed, including: a creation module for creating a migration model to be trained in the target domain based on the original speech recognition model trained in the source domain, wherein the output layer of the migration model is Different from the output layer of the original speech recognition model; the extraction module is used to extract the feature data set from the sample data of the target domain, wherein each feature data in the feature data set carries information that can reflect the physical properties of the same voice in different sample data. The environmental factor of similar possibility; the training module is used to substitute the training set in the feature data set as the input parameter into the migration model for iterative training to obtain the target speech recognition model; wherein, in the iterative training process, the output layer of the migration model is randomly adjusted Weights, the weights of other layers of the transfer model remain unchanged.

In a fourth aspect, a speech recognition device is proposed, comprising: a determination module for determining speech data to be recognized; an extraction module for extracting characteristic data from the speech data, wherein the characteristic data carries data that can reflect different sample data in The environmental factor of the similar possibility in terms of the physical properties of the same speech; the recognition module is used to substitute the feature data into the target speech recognition model for speech recognition, and the target speech recognition model is trained based on the speech recognition model training method of the first aspect.

In a fifth aspect, an electronic device is proposed, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being executed by the processor: based on the original speech recognition obtained by training in the source domain model, create a migration model to be trained in the target domain, where the output layer of the migration model is different from the output layer of the original speech recognition model; extract a feature dataset from the sample data in the target domain, where each feature data in the feature dataset is Both carry environmental factors that can reflect the similar possibility of different sample data in terms of the same physical attributes of the same speech; the training set in the feature data set is used as the input parameter to be substituted into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process , the output layer of the migration model adjusts the weights randomly, and the weights of other layers of the migration model remain unchanged.

In a sixth aspect, a computer-readable storage medium is provided, the computer-readable storage medium stores one or more programs, and the one or more programs, when executed by a server including a plurality of application programs, cause the server to perform the following operations: based on The original speech recognition model trained in the source domain creates a migration model to be trained in the target domain, where the output layer of the migration model is different from the output layer of the original speech recognition model; the feature dataset is extracted from the sample data in the target domain, Among them, each feature data in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical attributes of the same speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech is obtained. Recognition model; wherein, in the iterative training process, the output layer of the migration model adjusts the weights randomly, and the weights of other layers of the migration model remain unchanged.

In a seventh aspect, an electronic device is proposed, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being executed by the processor: determining speech data to be recognized; The feature data is extracted from the feature data, and the feature data carries environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; the feature data is substituted into the target speech recognition model for speech recognition, and the target speech recognition model is based on the first aspect. The speech recognition model training method is trained.

In an eighth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores one or more programs, and the one or more programs, when executed by a server including a plurality of application programs, cause the server to perform the following operations: determine The voice data to be recognized; extract feature data from the voice data, and the feature data carries environmental factors that can reflect the similar possibility of different sample data in terms of the same physical attributes of the same voice; the feature data is substituted into the target voice recognition model for voice recognition, the target The speech recognition model is obtained by training based on the speech recognition model training method of the first aspect.

Description of drawings

In order to more clearly illustrate the technical solutions in one or more embodiments or some situations of the present disclosure, the following briefly introduces the accompanying drawings that are needed in the description of one or more embodiments or some situations. Obviously, the following The drawings in the description are only some embodiments described in the present disclosure, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a schematic diagram of steps of a speech recognition model training method provided by the present disclosure.

FIG. 2 is a schematic diagram of steps of a speech recognition method provided by the present disclosure.

3a and 3b are schematic diagrams of recognition results before and after training provided by the present disclosure, respectively.

FIG. 4 is a flowchart of the training and application of the speech recognition model provided by the present disclosure by taking the announcement tone as an example.

FIG. 5 is a schematic structural diagram of a speech recognition model training apparatus provided by the present disclosure.

FIG. 6 is a schematic structural diagram of a speech recognition device provided by the present disclosure.

FIG. 7 is a schematic structural diagram of an electronic device provided by the present disclosure.

Detailed ways

In order to make those skilled in the art better understand the technical solutions in the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in one or more embodiments of the present disclosure. It is apparent that the described embodiment or embodiments are only some, but not all, of the embodiments of the present disclosure. Based on one or more embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

Before that, the technical terms that the present disclosure may involve are briefly introduced.

Frozen layers: The parameters of a fixed layer remain unchanged during training.

Retrain layers: Remove the last few layers of the neural network so that it is trained on the new dataset.

Learning rate: Used to control the speed of parameter updates during gradient descent. If the learning rate is too small, the convergence can be guaranteed, but it will greatly reduce the optimization speed of the model. If the learning rate is too large, it may lead to overshooting the global optimal point, resulting in parameter "oscillation".

Announcement Tone: The prompt tone that appears during a phone call.

The main purpose of the present disclosure is to provide a solution that can speed up the generation of an effective training model by using the transfer learning method in the case of insufficient training data for different application scenarios, reduce costs, improve training efficiency, and at the same time, During training, the feature data used in the retraining layer carries environmental factors that can affect the physical properties of speech. Therefore, the target speech recognition model obtained by training has good scalability and high recognition accuracy.

It should be noted that the training and recognition scheme of the speech recognition model involved in the present disclosure is not limited to the recognition scenario of the announcement sound, but can also be applied to other speech recognition scenarios with differences in the physical properties of speech.

Example 1

Referring to FIG. 1 , which is a schematic diagram of steps of a speech recognition model training method provided in the present disclosure, the method may include the following steps 102 to 106 .

Step 102: Create a migration model to be trained in the target domain based on the original speech recognition model trained in the source domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model.

The source domain, which can be the feature range formed by the sample data on which the original speech recognition model is trained. The sample data in the range of these features is known and has a sufficient number to be well trained to generate a high-quality model.

The target domain can be understood as the domain to be learned. The number of sample data in this domain is limited or relatively small, and it is impossible to train a high-quality model alone.

Here, step 102 uses the original speech recognition model trained in the source domain to perform transfer learning, so that a small amount of sample data in the target domain can be trained based on the transferred model parameters.

In an exemplary embodiment, in step 102, when creating a transfer model to be trained in the target domain based on the original speech recognition model trained in the source domain, the following two methods can be used: one and two.

method one

1.1. Extract the parameters of the original speech recognition model trained in the source domain, wherein the parameters include at least the weight of each layer in the neural network model.

1.2. Create a new speech recognition model based on the neural network, and transfer the weights of other fully connected layers except the output layer in the extracted parameters to the new speech recognition model as the migration model to be trained in the target domain.

It should be understood that the speech recognition model involved in the present disclosure is a deep neural network model. When implementing, you can first obtain the parameters of the original speech recognition model and load the model, mainly the weight matrix of each layer of the neural network. Then, re-initialize to create a new target speech recognition model, put the weights obtained earlier into the new model, and remove the parameters of the last layer.

Method 2

2.1. Obtain the original speech recognition model trained in the source domain.

2.2. Eliminate the weight of the output layer in the original speech recognition model to obtain the transfer model to be trained in the target domain.

The difference from the first method is that the speech recognition model is not newly created, but is directly processed on the basis of the obtained original speech recognition model, and the original speech recognition model with the weight of the output layer removed is used as the target speech recognition model.

It should be understood that, in the obtained target speech recognition model, the output layer is equivalent to a retraining layer, which is mainly based on a small number of speech samples in the target domain for migration training. The other fully connected layers act as frozen layers, and no parameter changes occur during model training.

Step 104: Extract a feature data set from the sample data in the target domain, wherein each feature data set in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical properties of speech.

The physical attributes of speech mainly include four types of attribute elements: pitch, intensity, length, and timbre. Generally speaking, it can be understood as intonation, accent, etc.

In the present disclosure, when step 104 extracts the feature dataset from the sample data of the target domain, it includes:

The first step is to preprocess the sample data of the target domain separately to obtain the MFCC.

When preprocessing the sample data, it can be implemented according to the following steps 1 to 5.

Step 1. Prepare the translation file translation.txt of the original announcement sound and the corresponding dictionary lexicon.txt. The translation file needs to be segmented.

Step 2: Pre-emphasis, framing and windowing are performed on the announcement tone data.

Step 3: Perform discrete Fourier transform on the original announcement sound data, convert the time domain sequence of the voice stream into a frequency spectrum sequence, and extract the sound spectrum information.

Step 4: Configure a triangular filter bank and calculate the output after filtering the signal amplitude spectrum by each triangular filter.

Step 5. Perform logarithmic operation on the outputs of all filters, and further perform discrete cosine transform to obtain MFCC; wherein, MFCC: Mel Frequency Cepstrum Coefficient, Mel frequency cepstrum coefficient.

Generally speaking, the MFCC feature extraction of speech recognition is basically completed here, but in order to minimize the impact of external conditions on the recognition results, we propose an environmental factor based on MFCC, temporarily named MFCC-p.

The second step is to determine the environmental factor of the MFCC based on the MFCC and the likelihood of the MFCC.

In an exemplary embodiment, the likelihood of the MFCC may be determined according to the MFCC and the likelihood function; and then the set of corresponding weights when the likelihood converges to an optimal value is determined as the environmental factor of the MFCC.

The likelihood of the MFCC can determine that the MFCC of any sample data is X, where X={x ₁ , x ₂ ,...x _T }, X contains T feature components; according to the following likelihood function formula, calculate the likelihood of the MFCC;

为高斯模型的参数，p _k为高斯混合模型中各个高斯所占的权重，u _k和Σ _k分别为各个高斯权重的均值和方差。 p(x _t |λ) is the likelihood value of MFCC, w _k is the weight of the k-th feature component,

are the parameters of the Gaussian model, p _k is the weight of each Gaussian in the Gaussian mixture model, and u _k and Σ _k are the mean and variance of each Gaussian weight, respectively.

The third step is to combine MFCC and its corresponding environmental factor MFCC-p into feature data. When combining, you can combine the two dimensions together.

In the fourth step, the characteristic data of all sample data is composed to obtain a characteristic data set.

In fact, the second - fourth step can include:

The MFCC feature corresponding to an extracted piece of speech data is X, where X={x ₁ , x ₂ ,...x _T }, and assuming the dimension is D, we can calculate its corresponding likelihood, the likelihood function formula for:

It is known from GMM (Gaussian Mixture Model) that we can obtain the extracted features by weighting by k Gaussian density functions. The mean value u _k and covariance Σ _k of each Gaussian component are 1×D and D×D respectively. , the set of weights λ={w _k , _ui , ∑ _k }. Give λ an initial value set, a basic learning rate, and use a neural network to make the Euclidean distance between the Gaussian component weighted and the original feature continuously decrease, and finally we can get a new feature MFCC-p. Adding our newly extracted impact factor MFCC-p feature to the original MFCC is the feature we need for the training below.

The MFCC feature is based on the difference in the sensitivity of the human ear to low-frequency and high-frequency sounds to achieve the recognition effect. And because the same sentence, even if it is spoken by the same person, has different characteristics in different situations, for example, the intonation, speed and accent of dialect spoken by different people may be different. In order to better refine and accurately recognize speech, we can consider combining the likelihood function, through accurate analysis of different intonation, speech speed, accent and other attributes, increase environmental factors, and perform feature correction on MFCC. Compared with MFCC, MFCC The -p feature can just reflect these effects. Combined with the original MFCC feature, it can recognize speech more accurately and in detail, especially the announcement sound and other similar speech.

Step 106: Substitute the training set in the feature data set as an input parameter into the migration model for iterative training to obtain a target speech recognition model.

Among them, in the iterative training process, the weights of the output layer of the migration model are adjusted randomly, and the weights of other layers of the migration model remain unchanged.

For datasets of different sizes, set different learning rates. For the frozen layers, since they are obtained based on huge speech data, a normal learning rate is set for them, while the newly added Retrain layers are trained based on smaller announcement data, and a smaller learning rate needs to be set for them. . Therefore, when iteratively training the transfer model, the learning rate set for the output layer is lower than the learning rate set for the other layers.

It should be understood that different learning rates are adjusted in each iteration. In the entire iterative training process, after each iteration is completed, the error rate of the model test after this iteration can be compared with the error rate of the model test after the previous iteration; if the error rate of this iteration is greater than the previous iteration If the error rate is less than or equal to the previous error rate, the learning rate will be increased by the preset rate. Wherein, the preset amplitude may be about 5%.

In the iterative process, the recognition error rate is continuously calculated. When the error rate is lower than the preset accuracy, the training is terminated and a new training model is obtained.

It can be seen from the above technical solutions that a migration model to be trained in the target domain is created based on the original speech recognition model trained in the source domain, and a feature data set is extracted from the sample data in the target domain. An environmental factor that can reflect the similarity of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process, the migration The output layer of the model adjusts the weights randomly, and the weights of the other layers of the transfer model remain unchanged. Therefore, in the case of insufficient training data in the new scene, the use of transfer learning can speed up the generation of an effective training model, reduce costs, and improve training efficiency. At the same time, during training, the feature data used by the training layer There are environmental factors that can affect the physical properties of speech, so the target speech recognition model obtained by training has good scalability and high recognition accuracy.

Embodiment 2

Referring to FIG. 2, which is a schematic diagram of steps of a speech recognition method provided by the present disclosure, the method may include:

Step 202: determine the voice data to be recognized;

Step 204: extracting characteristic data from the speech data, and the characteristic data carries environmental factors that can reflect the similar possibility of different sample data in terms of the same physical property of speech;

Step 206: Substitute the feature data into the target speech recognition model to perform speech recognition, and the target speech recognition model is obtained by training based on the method of the first embodiment.

The method of extracting characteristic data in step 204 may refer to Embodiment 1, which is the description of extracting characteristic data from sample data. I won't go into details here.

Compared with the traditional transfer learning process, in order to improve the recognition rate of the announcement sound, this solution uses the language model generated by training to adjust the hyperparameters of the speech recognition model. Typically, the language model can be used to implement functions such as correcting homophone typos, for example, correcting "Dialing" in the recognition result to "Dialing", which greatly improves the coupling between the speech recognition model and the announcement sound field.

The recognition results before and after training are shown in Figure 3a and Figure 3b: a total of 40 announcement sounds that were sent back by the original training model could not be recognized, 33 of them could be recognized by the new training model, and the speech recognition efficiency was significantly improved.

To sum up, the present disclosure allows engineers to spend less time to obtain a training model with improved recognition rate when the existing announcement sound data is insufficient and the existing model has a low recognition rate for the announcement sound. There is no need to spend manpower to label a large amount of training data, no need to relabel expired data, and no need to train from scratch on trained data. Engineers only need to perform feature extraction on the newly added announcement sound data, and on the basis of the original model, train on the announcement sound feature data to obtain a better training model. Since the output layer is only trained for the newly added data, the training time can be effectively shortened. Moreover, this method is also very flexible compared to traditional methods. When the current training model has poor recognition results for a specific field, the traditional method is to add the data of the specific field on the basis of the original data, and retrain to obtain the model. The results obtained after spending a lot of time training are not necessarily better. With the method provided by the present disclosure, not only a model can be obtained quickly, but also a better recognition result can be obtained by flexibly adjusting the initial weight of the output layer. From the perspective of engineers, the training process is simplified and the training efficiency is improved; from the perspective of users, when the existing solution is not perfect, a mature solution can be obtained in less time, which greatly improves the efficiency of training. Improved user experience.

Embodiment 3

Referring to FIG. 4 , taking the announcement sound as an example, the overall flow of model training and application is introduced.

model training

--MFCC+MFCC-p from which the announcement tone sample data can be extracted. For the extraction method, refer to the introduction in the first embodiment.

--Remove the last layer of the fully connected layer of the original model to create a migration model.

In fact, the extraction of MFCC+MFCC-p and the creation of the migration model can be reversed in order or performed simultaneously without interfering with each other.

--The weights of the frozen layer remain unchanged, and the weights of the retraining layer are adjusted randomly.

--Train the transfer model using the extracted training set in MFCC+MFCC-p.

-- Determine whether the model test error rate for each iteration is less than the threshold, if so, end the training to obtain the target speech recognition model, otherwise, continue the iterative training.

Subsequently, the obtained target speech recognition model can be sent to the model application link.

Model application

--Extract the MFCC+MFCC-p of the announcement tone pending data.

-- Send the extracted MFCC+MFCC-p into the target speech recognition model trained in the above link for speech recognition.

In the training phase, the feature extraction is performed on the speech samples of the original model and the announcement sound recognition model, and these feature data are mapped, so that the features of the original speech and announcement sound can be mapped to a new common feature space, and the dimension of these data is guaranteed. Controlled within a certain range (to prevent the curse of dimensionality), and then iteratively optimizes the pseudo-labels of the announcement domain samples within this new feature space until convergence. This pre-training model is based on a large data set. Since there is a great similarity in characteristics between ordinary speech and announcement speech, its corresponding structure and weight can be directly used, and the DNN without the output layer can be regarded as a The fixed feature extraction machine sets a small learning rate for the migration layer, and the newly added output layer is updated with a normal learning rate and applied to the new data set. Predict and score to get the final model. For this newly generated model, if the recognition result is not ideal, the weights of some layers at the beginning of the original model can also be kept unchanged, and the subsequent layers can be retrained to obtain new weights. In this process, the user can make multiple attempts, so that the best training model can be obtained.

Embodiment 4

Referring to FIG. 5 , which is a schematic structural diagram of a speech recognition model training apparatus provided in the present disclosure, the speech recognition model training apparatus 500 may include: a creation module 502 , an extraction module 504 and a training module 506 .

The creation module 502 is configured to create a migration model to be trained in the target domain based on the original speech recognition model trained in the source domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model.

The extraction module 504 is configured to extract a feature data set from the sample data of the target domain, wherein each feature data set in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical property of speech.

The training module 506 is used for substituting the training set in the feature data set as an input parameter into the migration model for iterative training to obtain the target speech recognition model;

Among them, in the iterative training process, the weights of the output layer of the migration model are adjusted randomly, and the weights of other layers of the migration model remain unchanged.

An achievable solution, the creation module 502 is used to extract the parameters of the original speech recognition model trained in the source domain when creating the migration model to be trained in the target domain based on the original speech recognition model obtained by training in the source domain, Among them, the parameters include at least the weight of each layer in the neural network model; create a new speech recognition model based on the neural network, and transfer the weights of other layers except the output layer in the extracted parameters to the new speech recognition model, as the target Domain transfer model to be trained.

In another achievable solution, the creation module 502 is used to obtain the original speech recognition model trained in the source domain when creating the migration model to be trained in the target domain based on the original speech recognition model trained in the source domain; The weight of the output layer in the original speech recognition model is removed to obtain the transfer model to be trained in the target domain.

In yet another achievable solution in the present disclosure, when extracting the feature data set from the sample data of the target domain, the extraction module 504 is used to preprocess the sample data of the target domain respectively to obtain the Mel frequency cepstral coefficient MFCC; MFCC and its likelihood, determine the environmental factors of MFCC; combine MFCC and its corresponding environmental factors into feature data; the feature data set of all sample data is composed of feature data sets.

In yet another achievable solution in the present disclosure, when determining the environmental factor of the MFCC based on the likelihood of the MFCC, the extraction module 504 is used to determine the likelihood of the MFCC according to the MFCC and the likelihood function; the likelihood is converged to the optimum When the value is taken, the corresponding weight set is determined as the environmental factor of the MFCC.

In yet another achievable solution in the present disclosure, when the extraction module 504 determines the likelihood of the MFCC according to the MFCC and the likelihood function, it is used to determine that the MFCC of any sample data is X, where X={x ₁ ,x ₂ ,...x _T }, X contains T eigencomponents; According to the following likelihood function formula, calculate the likelihood of this MFCC;

为高斯模型的参数，p _k为高斯混合模型中各个高斯所占的权重，u _k和Σ _k分别为各个高斯权重的均值和方差。 p(x _t |λ) is the likelihood value of MFCC, w _k is the weight of the k-th feature component,

are the parameters of the Gaussian model, p _k is the weight of each Gaussian in the Gaussian mixture model, and u _k and Σ _k are the mean and variance of each Gaussian weight, respectively.

In yet another achievable solution in the present disclosure, when the transfer model is iteratively trained, the learning rate set for the output layer is lower than the learning rate set for other layers.

In yet another achievable solution in the present disclosure, in the iterative training process, after each iteration is completed, it further includes: a comparison module, configured to compare the error rate of the model test after this iteration with the model after the previous iteration The error rate of the test is compared; if the error rate of this time is greater than the previous error rate, the learning rate is decreased by the preset range; otherwise, the learning rate is increased by the preset range.

In yet another achievable solution in the present disclosure, during the iterative training process, when the error rate is lower than the threshold, the training is ended after the current iteration is completed.

It can be seen from the above technical solutions that a migration model to be trained in the target domain is created based on the original speech recognition model trained in the source domain, and a feature data set is extracted from the sample data in the target domain. An environmental factor that can reflect the similarity of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process, the migration The output layer of the model adjusts the weights randomly, and the weights of the other layers of the transfer model remain unchanged. Therefore, in the case of insufficient training data in the new scene, the use of transfer learning can speed up the generation of an effective training model, reduce costs, and improve training efficiency. At the same time, during training, the feature data used by the training layer There are environmental factors that can affect the physical properties of speech, so the target speech recognition model obtained by training has good scalability and high recognition accuracy.

Embodiment 5

Referring to FIG. 6 , which is a schematic structural diagram of a speech recognition apparatus provided in the present disclosure, the apparatus 600 may include: a determination module 602 , an extraction module 604 , and an identification module 606 .

The determining module 602 is used for determining the speech data to be recognized.

The extraction module 604 is used for extracting characteristic data from the speech data, where the characteristic data carries environmental factors that can reflect the similar possibility of different sample data in terms of the same speech physical property.

The recognition module 606 is used for substituting the feature data into the target speech recognition model for speech recognition, and the target speech recognition model is obtained by training based on the method of the first embodiment.

It can be seen from the above technical solutions that a migration model to be trained in the target domain is created based on the original speech recognition model trained in the source domain, and a feature data set is extracted from the sample data in the target domain. An environmental factor that can reflect the similarity of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process, the migration The output layer of the model adjusts the weights randomly, and the weights of the other layers of the transfer model remain unchanged. Therefore, in the case of insufficient training data in the new scene, the use of transfer learning can speed up the generation of an effective training model, reduce costs, and improve training efficiency. At the same time, during training, the feature data used by the training layer There are environmental factors that can affect the physical properties of speech, so the target speech recognition model obtained by training has good scalability and high recognition accuracy.

Embodiment 6

The present disclosure also provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is executed by the processor: based on the original speech recognition model trained in the source domain, Create a migration model to be trained in the target domain, where the output layer of the migration model is different from the output layer of the original speech recognition model; extract a feature dataset from the sample data in the target domain, where each feature data in the feature dataset carries There are environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as the input parameter to be substituted into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process, The output layer of the transfer model adjusts the weights randomly, and the weights of other layers of the transfer model remain unchanged.

At the same time, a computer-readable storage medium is also provided. The computer-readable storage medium stores one or more programs, and the one or more programs, when executed by a server including a plurality of application programs, cause the server to perform the following operations: The original speech recognition model obtained by domain training is used to create a migration model to be trained in the target domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model; the feature dataset is extracted from the sample data of the target domain, wherein, Each feature data in the feature data set carries environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech recognition model is obtained. ; Among them, in the iterative training process, the output layer of the migration model adjusts the weights randomly, and the weights of other layers of the migration model remain unchanged.

It can be seen from the above technical solutions that a migration model to be trained in the target domain is created based on the original speech recognition model trained in the source domain, and a feature data set is extracted from the sample data in the target domain. An environmental factor that can reflect the similarity of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process, the migration The output layer of the model adjusts the weights randomly, and the weights of the other layers of the transfer model remain unchanged. Therefore, in the case of insufficient training data in the new scene, the use of transfer learning can speed up the generation of an effective training model, reduce costs, and improve training efficiency. At the same time, during training, the feature data used by the training layer There are environmental factors that can affect the physical properties of speech, so the target speech recognition model obtained by training has good scalability and high recognition accuracy.

Embodiment 7

The present disclosure also provides another electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being executed by the processor: determining speech data to be recognized; Extract feature data, which carries environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; substitute the feature data into the target speech recognition model for speech recognition, and the target speech recognition model is based on the method of the first embodiment Trained to get.

At the same time, another computer-readable storage medium is also provided. The computer-readable storage medium stores one or more programs, and the one or more programs, when executed by a server including a plurality of application programs, cause the server to perform the following operations: Recognized speech data; extract characteristic data from speech data, which carries environmental factors that can reflect the similarity of different sample data in terms of the same physical properties of speech; substitute characteristic data into the target speech recognition model for speech recognition, the target speech The recognition model is obtained by training based on the method of the first embodiment.

It can be seen from the above technical solutions that a migration model to be trained in the target domain is created based on the original speech recognition model trained in the source domain, and a feature data set is extracted from the sample data in the target domain. An environmental factor that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; the training set in the feature data set is used as an input parameter into the migration model for iterative training, and the target speech recognition model is obtained; among them, in the iterative training process, the migration The output layer of the model adjusts the weights randomly, and the weights of the other layers of the transfer model remain unchanged. Therefore, in the case of insufficient training data in the new scene, the use of transfer learning can speed up the generation of an effective training model, reduce costs, and improve training efficiency. At the same time, during training, the feature data used by the training layer There are environmental factors that can affect the physical properties of speech, so the target speech recognition model obtained by training has good scalability and high recognition accuracy.

For the electronic devices involved in the above embodiments, reference may be made to the schematic structural diagram shown in FIG. 7 .

In conclusion, the above are only preferred embodiments of the present disclosure, and are not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

The systems, devices, modules or units described in one or more of the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. In an exemplary embodiment, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer , wearable devices, or a combination of any of these devices.

Computer-readable storage media includes both persistent and non-permanent, removable and non-removable media, and storage of information can be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

The various embodiments in the present disclosure are described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the partial descriptions of the method embodiments.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims (16)

A speech recognition model training method, comprising: Create a migration model to be trained in the target domain based on the original speech recognition model trained in the source domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model; Extracting a feature data set from the sample data of the target domain, wherein each feature data set in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; Substitute the training set in the feature data set as an input parameter into the migration model for iterative training to obtain a target speech recognition model; Wherein, in the iterative training process, the weight of the output layer of the migration model is adjusted randomly, and the weights of other layers of the migration model remain unchanged. The speech recognition model training method as claimed in claim 1, based on the original speech recognition model obtained by training in the source domain, creating a migration model to be trained in the target domain, comprising: Extract the parameters of the original speech recognition model trained in the source domain, wherein the parameters include at least the weight of each layer in the neural network model; Create a new speech recognition model based on the neural network, and transfer the weights of other layers except the output layer in the extracted parameters to the new speech recognition model as the transfer model to be trained in the target domain. The speech recognition model training method as claimed in claim 1, based on the original speech recognition model obtained by training in the source domain, creating a migration model to be trained in the target domain, comprising: Obtain the original speech recognition model trained in the source domain; The weight of the output layer in the original speech recognition model is eliminated to obtain a migration model to be trained in the target domain. The speech recognition model training method according to any one of claims 1-3, extracting a feature data set from the sample data of the target domain, comprising: The sample data of the target domain are respectively preprocessed to obtain the Mel frequency cepstral coefficient MFCC; determining an environmental factor for the MFCC based on the MFCC and the likelihood of the MFCC; combining the MFCC and the environmental factors corresponding to the MFCC into characteristic data; The feature data of all sample data is composed to obtain a feature data set. The speech recognition model training method according to claim 4, based on the MFCC and the likelihood of the MFCC, determining the environmental factor of the MFCC, comprising: determining the likelihood of the MFCC from the MFCC and a likelihood function; The set of corresponding weights when the likelihood converges to the best value is determined as the environmental factor of the MFCC. The speech recognition model training method as claimed in claim 5, determining the likelihood of the MFCC according to the MFCC and the likelihood function, comprising: Determine the MFCC of any sample data to be X, where X={x ₁ , x ₂ ,...x _T }, and X contains T feature components; Calculate the likelihood of the MFCC according to the following likelihood function formula;

The p(x _t |λ) is the likelihood value of the MFCC, the w _k is the weight of the k-th feature component, and the

are the parameters of the Gaussian model, p _k is the weight of each Gaussian in the Gaussian mixture model, and u _k and Σ _k are the mean and variance of each Gaussian weight, respectively. The speech recognition model training method according to any one of claims 1 to 3, when performing iterative training on the transfer model, the learning rate set for the output layer is lower than the learning rate set for other layers. The speech recognition model training method according to any one of claims 1-3, in the iterative training process, after each iteration is completed, the method further comprises: Compare the error rate of the model test after this iteration with the error rate of the model test after the previous iteration; If the error rate of this time is greater than the previous error rate, the learning rate will be reduced according to the preset range; If the error rate of this time is less than or equal to the error rate of the last time, the learning rate will be increased according to the preset range. The speech recognition model training method according to any one of claims 1-3, in the iterative training process, when the error rate is lower than the threshold, the training is ended after the current iteration is completed. A speech recognition method, comprising: Determine the speech data to be recognized; extracting feature data from the voice data, the feature data carrying environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of the voice; Substitute the feature data into a target speech recognition model to perform speech recognition, and the target speech recognition model is obtained by training based on the method described in any one of claims 1-9. A speech recognition model training device, comprising: A creation module for creating a migration model to be trained in the target domain based on the original speech recognition model trained in the source domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model; The extraction module is used to extract a feature data set from the sample data of the target domain, wherein each feature data in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical properties of speech ; A training module, used for substituting the training set in the feature data set as an input parameter into the migration model for iterative training to obtain a target speech recognition model; Wherein, in the iterative training process, the weight of the output layer of the migration model is adjusted randomly, and the weights of other layers of the migration model remain unchanged. A voice recognition device, comprising: A determination module, used to determine the speech data to be recognized; an extraction module, used for extracting feature data from the voice data, the feature data carrying environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of the voice; The recognition module is used for substituting the feature data into a target speech recognition model for speech recognition, and the target speech recognition model is obtained by training based on the method of any one of claims 1-9. An electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being executed by the processor: Create a migration model to be trained in the target domain based on the original speech recognition model trained in the source domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model; Extracting a feature data set from the sample data of the target domain, wherein each feature data set in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; Substitute the training set in the feature data set as an input parameter into the migration model for iterative training to obtain a target speech recognition model; Wherein, in the iterative training process, the weight of the output layer of the migration model is adjusted randomly, and the weights of other layers of the migration model remain unchanged. A computer-readable storage medium storing one or more programs that, when executed by a server including a plurality of application programs, cause the server to perform the following operations: Based on the original speech recognition model obtained by training in the source domain, create a migration model to be trained in the target domain, wherein the output layer of the migration model is different from the output layer of the original speech recognition model; Extracting a feature data set from the sample data of the target domain, wherein each feature data set in the feature data set carries an environmental factor that can reflect the similar possibility of different sample data in terms of the same physical properties of speech; Substitute the training set in the feature data set as an input parameter into the migration model for iterative training to obtain a target speech recognition model; Wherein, in the iterative training process, the weight of the output layer of the migration model is adjusted randomly, and the weights of other layers of the migration model remain unchanged. An electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being executed by the processor: Determine the speech data to be recognized; extracting feature data from the voice data, the feature data carrying environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of the voice; Substitute the feature data into a target speech recognition model to perform speech recognition, and the target speech recognition model is obtained by training based on the method described in any one of claims 1-9. A computer-readable storage medium storing one or more programs that, when executed by a server including a plurality of application programs, cause the server to perform the following operations: Determine the speech data to be recognized; extracting feature data from the voice data, the feature data carrying environmental factors that can reflect the similar possibility of different sample data in terms of the same physical properties of the voice; Substitute the feature data into a target speech recognition model to perform speech recognition, and the target speech recognition model is obtained by training based on the method described in any one of claims 1-9.

Images