WO2022121155A1 - Meta learning-based adaptive speech recognition method and apparatus, device and medium - Google Patents

Abstract

A meta learning-based adaptive speech recognition method and apparatus, a device and a medium, relating to the technical field of artificial intelligence and capable of solving the problems that, when an adaptive speaker adjustment is made to a speech recognition model, adjustment criteria needs to be manually designed beforehand according to different speaker types, the design process is tedious, onerous and cannot cover all parameter adjustment scenarios, and thus the speech recognition effect is poor. The method comprises: training a speech recognition model and a meta learning adaptation model by means of pre-processed sample speech data (101); adjusting, based on the meta learning adaptation model, initial model parameters of the speech recognition model to target model parameters which match a target speech type (102); and recognizing a target speech of the target speech type by means of the speech recognition model configured with the target model parameters (103).

Description

Meta-learning-based adaptive speech recognition method, device, device and medium

This application claims the priority of the Chinese patent application filed on December 10, 2020 with the application number 202011434900.9 and the invention titled "Meta-learning-based adaptive speech recognition method, device, equipment and medium", all of which are The contents are incorporated herein by reference.

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to a meta-learning-based adaptive speech recognition method, apparatus, device and medium.

Background technique

With the development of deep learning, automatic speech recognition systems have achieved remarkable results and are used in various scenarios of daily life. The most widely used speech recognition application today is the intelligent assistant, which allows users to communicate with machines naturally through speech to assist work. However, most scenarios for intelligent assistant services are single-user, and their speech recognition is aimed at a single speaker. At this time, the performance of the automatic speech recognition system can improve the recognition effect by adjusting the parameters of the acoustic model to compensate for the mismatch between the training and testing conditions. This method of adjusting existing parameters for unknown speakers is called speaker adaptation.

The inventors have realized that an effective speaker adaptation method relies on the selection of suitable acoustic model parameters and suitable parameter update rules to avoid overfitting on less training data. In order to effectively adjust the parameters of the acoustic model according to different speakers, the manual design adjustment criteria (such as the number of design adjustment steps, the learning rate, etc.) are mainly used at present. However, this adjustment method requires manual design of adjustment criteria according to different speaker types in advance. The design process is cumbersome and the workload is large, and it cannot cover all parameter adjustment situations, which may easily lead to poor speech recognition results.

SUMMARY OF THE INVENTION

In view of this, the present application provides a meta-learning-based adaptive speech recognition method, device, equipment and medium, which mainly solves the problem that when the speaker adaptively adjusts the speech recognition model, it needs to be manually adjusted in advance according to different speaker types. To design the adjustment criteria, the design process is cumbersome and the workload is large, and it cannot cover all parameter adjustment situations, which leads to the problem of poor speech recognition effect.

According to an aspect of the present application, a meta-learning-based adaptive speech recognition method is provided, the method comprising:

Use the preprocessed sample speech data to train the speech recognition model and the meta-learning adaptation model;

Based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type;

The target speech under the target speech type is recognized by using the speech recognition model configured with the target model parameters.

According to another aspect of the present application, a meta-learning-based adaptive speech recognition device is provided, comprising:

The training module is used to train the speech recognition model and the meta-learning adaptation model by using the preprocessed sample speech data;

an adjustment module for adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type based on the meta-learning adaptation model;

The recognition module is used for recognizing the target speech under the target speech type by using the speech recognition model configured with the target model parameter.

According to yet another aspect of the present application, there is provided a storage medium on which a computer program is stored, and when the program is executed by a processor, the following methods are implemented:

Use the preprocessed sample speech data to train the speech recognition model and the meta-learning adaptation model;

Based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type;

The target speech under the target speech type is recognized by using the speech recognition model configured with the target model parameters.

According to yet another aspect of the present application, a computer device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, the processor implements the following method when executing the program :

Use the preprocessed sample speech data to train the speech recognition model and the meta-learning adaptation model;

Based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type;

The target speech under the target speech type is recognized by using the speech recognition model configured with the target model parameters.

In this application, the meta-learning adaptation model is used to realize the self-adaptive adjustment of the model parameters in the speech recognition model, and the artificial intelligence technology is used, which not only reduces the instability of manual design, but also enables the model parameter update for different application scenarios. This ensures the accuracy of speech recognition.

Description of drawings

The drawings described herein are used to provide further understanding of the application, and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the local application. In the attached image:

1 shows a schematic flowchart of a meta-learning-based adaptive speech recognition method provided by an embodiment of the present application;

2 shows a schematic flowchart of another meta-learning-based adaptive speech recognition method provided by an embodiment of the present application;

3 shows a schematic flowchart of a meta-learning adaptation model training process provided by an embodiment of the present application;

FIG. 4 shows a schematic flowchart of a meta-learning-based adaptive speech recognition system provided by an embodiment of the present application;

FIG. 5 shows a schematic structural diagram of a meta-learning-based adaptive speech recognition device provided by an embodiment of the present application;

FIG. 6 shows a schematic structural diagram of another meta-learning-based adaptive speech recognition device provided by an embodiment of the present application.

Detailed ways

Hereinafter, the present application will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict.

The technical solution of the present application relates to the technical field of artificial intelligence, for example, it may specifically relate to speech processing technology to realize speech recognition. Optionally, the data involved in this application, such as sample speech data and/or target speech, etc., may be stored in a database, or may be stored in a blockchain, which is not limited in this application.

For the current speaker adaptive adjustment of the speech recognition model, it is necessary to manually adjust the design of the criteria according to different speaker types in advance. The design process is cumbersome, the workload is large, and it cannot cover all parameter adjustment situations, which leads to speech recognition. For the problem of poor effect, an embodiment of the present application provides a meta-learning-based adaptive speech recognition method. As shown in FIG. 1 , the method includes:

101. Use the preprocessed sample speech data to train a speech recognition model and a meta-learning adaptation model.

Among them, the sample speech data corresponds to a preset number of unlabeled speech data. Before using the sample speech data to train the speech recognition model and the meta-learning adaptation model, it is necessary to use a preset speech processing tool (such as the Kaldi ASR tool) to process the sample speech data. Data preprocessing. Data preprocessing can include pre-emphasis, framing, windowing and other operations. Through data preprocessing, aliasing and high-order harmonic distortion caused by defects in human vocal organs and acquisition equipment can be eliminated. The effect of voice signal quality. In addition, in order to facilitate the analysis of speech features, it is also necessary to use feature extraction tools (such as Mel-Frequency Cepstral Coefficients (MFCCs) or Filter-Bank features, etc.) to perform feature extraction. Specifically, it is necessary to extract the speech features. and text features.

In addition, when using the sample speech data to train the meta-learning adaptation model, because the test set data cannot be used in the training process, it is also necessary to preprocess the sample speech data according to the meta-learning method. The data is sampled or divided into multiple data blocks, so that during the training process, the current data block can be used for adaptive training, and the loss reduction can be achieved on the next data block corresponding to the current data block.

The executive body of the present application can be a speech recognition system for realizing speaker adaptation, in which a pre-trained speech recognition model is configured, and the speech recognition model can be adjusted according to the adaptation data of different speech types. Meta-learning to adapt the model parameters to the model.

102. Based on the meta-learning adaptation model, adjust the initial model parameters of the speech recognition model to target model parameters matching the target speech type.

The target speech type is the same speech type as the speaker type to be recognized, and the speech type can be customized and divided according to actual application requirements. The following are some examples of the classification of speech types: Because the audio and tones corresponding to different age groups or different genders are often different, the speaker types can be divided according to age or gender. Adults, teenagers, infants and young children, or according to gender, the voice types are divided into men and women; correspondingly, because even in the same age group, different people will have different timbres due to different vocal cords, so each person can be used as a voice. An independent voice type; in addition, due to the influence of the environment, people in different regions will have different types of accents, so the definition of voice types can be divided according to the environmental region, such as Northeastern dialect, Sichuan dialect, Shandong dialect, Cantonese dialect, etc. .

Correspondingly, for this embodiment, the target voice types can be divided and selected according to actual application requirements. For example, when it involves voice recognition tasks for different age groups (such as infant education, etc.), the target voice types can be divided and selected according to age groups. ; If it involves one-to-one intelligent voice service, each person can be regarded as an independent group type; if it involves popular intelligent voice service, the target voice type can be divided and selected according to the environmental area. Further, after the target group type is selected, the initial model parameters of the speech recognition model can be adjusted to the target model parameters matching the target speech type by using the meta-learning adaptation model.

103. Use the speech recognition model configured with the target model parameter to recognize the target speech under the target speech type.

For this embodiment, in a specific application scenario, after it is determined that the target model parameters matching the target population type are obtained, the target speech under the target speech type can be targeted based on the speech recognition model configured with the target model parameters. Thereby, the recognition accuracy is improved.

Through the adaptive speech recognition method based on meta-learning in this embodiment, compared with the way of performing speech adaptive recognition by manually designing and adjusting criteria, the present application can firstly pre-train the speech recognition model through sample speech data, and further Taking the speaker adaptation task as a task in meta-learning, a meta-learning adaptation model is designed to adjust the model parameters in the speech recognition model according to different speech types. Then, when the target speech type is determined, the initial model parameters of the speech recognition model can be adjusted to the target model parameters matching the target speech type based on the meta-learning adaptation model, and then the speech recognition model configured with the target model parameters can be used to realize the target speech type. Targeted and precise recognition of target speech. In this application, the meta-learning adaptation model is used to realize the self-adaptive adjustment of the model parameters in the speech recognition model, and the artificial intelligence technology is used, which not only reduces the instability of manual design, but also enables the model parameter update for different application scenarios. This ensures the accuracy of speech recognition.

Further, as a refinement and extension of the specific implementation of the above-mentioned embodiment, in order to fully describe the specific implementation process in this embodiment, another adaptive speech recognition method based on meta-learning is provided, as shown in FIG. Methods include:

201. Preprocess the sample speech data, and mark the first speech feature and the first text feature corresponding to the sample speech data, and the preprocessing at least includes pre-emphasis processing, frame segmentation processing, and windowing processing.

The purpose of pre-emphasizing the sample speech data is to emphasize the high-frequency part of the speech, remove the influence of lip radiation, and increase the high-frequency resolution of the speech. Generally, pre-emphasis is realized by a first-order FIR high-pass digital filter with a transfer function of H(z)=1-az ^-1 , where a is a pre-emphasis coefficient, 0.9<a<1.0. Assuming that the speech sample value at time n is x(n), the result after pre-emphasis processing is y(n)=x(n)-ax(n-1), where a=0.98 is taken.

In addition, since the characteristics of the speech signal remain basically unchanged in a short time range, that is, the speech signal has short-term stability, so the analysis of the sample speech should be based on the "short-term" basis. Windowing and framing processing are performed, and the sample speech is divided into sections to analyze its characteristics, and each section becomes a "frame". Since the frame is too large to obtain the time-varying characteristics of the speech signal, and the frame is too small to extract the characteristics of the speech signal, each frame can be divided into 20-30ms size. After the sample speech is windowed and divided into frames, A vector of speech features corresponding to each frame of sample speech can be obtained. The speech features of the entire sample data can be obtained by framing, that is, integrating the speech feature vectors of each frame based on the frame serial number, and the size of the frame serial number indicates the time sequence of the corresponding speech of each frame.

For this example, by considering the speech recognition task as a sequence-to-sequence problem. Specifically, the speech features in the sample speech are calculated frame by frame, and finally the speech features can be integrated based on the frame number corresponding to each frame, and the speech sequence characteristics of the sample speech can be further obtained, which are expressed as:

X _t =[x ₀ ,...,x _T ]

Among them, T is the total number of frames of the speech sequence, which is expressed as the speech feature contained in the t-th frame.

Correspondingly, the existing voice conversion algorithm can be used to obtain the text features corresponding to each frame of sample voice, and further integration can be performed to obtain the text sequence features corresponding to the sample voice, which are expressed as:

Yu _u =[x ₀ ,...,y _U ]

Among them, U is the total length of the text corresponding to the speech, which is represented as the u-th text feature.

202. Train a speech recognition model conforming to the first training standard based on the first speech feature and the first text feature.

Among them, in this embodiment, the speech recognition model obtained by pre-training with sample speech data can be applied to basic speech recognition tasks, but when performing targeted speech recognition (such as speech recognition with accents, speech recognition of infants and young children) Recognition, etc.), the recognition effect is often not accurate enough, so it is necessary to perform the model parameter correction process in the subsequent steps, so that the speech recognition model can achieve targeted recognition. For the speech recognition model of this embodiment, a deep neural network model based on Connectionist Temporal Classification (CTC) can be used, and the CTC-based model is trained by predicting the output of each frame in the speech sequence and comparing the real sample labels to calculate Model training error. In this application, the speech recognition model uses the first speech feature and the first text feature corresponding to the sample speech data for training, and the network structure can generally adopt the structure of LSTM/CNN/GRU. Fewer network structure layers. Correspondingly, in order to train a speech recognition model that meets the first training standard according to the first speech feature and the first text feature, step 202 in the embodiment may specifically include: inputting the first speech feature into the speech recognition model to obtain a text output result; A first loss function is calculated from the text output result and the first text feature; if it is determined that the first loss function is smaller than the first preset threshold, it is determined that the speech recognition model meets the first training standard. The first preset threshold may be set according to actual training accuracy requirements, and should be a value greater than 0 and less than 1. The larger the set value, the higher the training accuracy of the speech recognition model.

203. Use the sample speech data and the speech recognition model to train a meta-learning adaptation model that meets the second training standard.

In this embodiment, the meta-learning adaptation model is trained based on a meta-learning technique, wherein the meta-learning is an algorithm that uses a model related to a learning task instead of manually designing an adjustment criterion. Under the setting of this scheme, the task of the meta-learning adaptation model is to adjust the parameters of the original model with the help of a small amount of adaptive speech data, so that its effect on speech recognition is better. When using the sample speech data to train the meta-learning adaptation model, because the test set data cannot be used in the training process, it is necessary to preprocess the data blocks of the sample speech data in the way of meta-learning. Specifically, sampling from the sample speech data can be used. Or the method of dividing and converting into multiple data blocks, so that in the training process, the current data block can be used for adaptive training, and the loss reduction can be obtained on the next data block corresponding to the current data block. Correspondingly, when using the sample speech data and the speech recognition model to perform the training of the meta-learning adaptation model, step 203 in the embodiment may specifically include: dividing the sample speech data into a preset number of data blocks, and extracting the second data block of each data block. The voice feature and the second text feature; according to the second voice feature, the second text feature, and the voice recognition model, a meta-learning adaptation model that meets the second training standard is trained. It should be noted that the second voice feature and the second text feature in the steps of this embodiment are different from the first voice feature and the first text feature in the step 202 of the embodiment, and the second voice feature and the second text feature both correspond to sample voices The feature sequence of each data block divided by the data, and the first voice feature and the first text feature correspond to the feature sequence of the entire sample voice data.

In the process of training the meta-learning adaptation model according to the second speech feature, the second text feature and the speech recognition model, please refer to the schematic flowchart of the training process of the meta-learning adaptation model shown in FIG. 3 , if the current data block is divided into In the first data block, the initial model parameters of the pre-trained speech recognition model (pre-training model 1) and the second speech feature and the second text feature of the data block 1 can be used to calculate the meta-learning adaptation model in the data block 1. The loss value, loss gradient and new model parameters in , replace the new model parameters with the initial model parameters in the pre-training model 1, and obtain the model 2 corresponding to the speech recognition model. Further according to the model parameters corresponding to the model 2 and the second speech feature and the second text feature of the data block 2, the loss value, the loss gradient and the new model parameters of the meta-learning adaptation model in the data block 2 are calculated, and the new model parameters obtained at this time are calculated. The model parameters are replaced into model 2 to obtain model 3 corresponding to the speech recognition model. Further according to the model parameters corresponding to the model 3 and the second speech feature and the second text feature of the data block 3, the loss value, the loss gradient and the new model parameters of the meta-learning adaptation model in the data block 3 are calculated, and the new model parameters obtained at this time are calculated. The model parameters are replaced into model 3, and the model 4 corresponding to the speech recognition model is obtained, until the training of all data blocks for the meta-learning adaptation model is completed. The loss function determines whether the meta-learning adaptive model has passed the training according to the loss function. When the training is passed, the model parameters determined by the last data block can be determined as the new model parameters of the speech recognition model in the testing phase.

Correspondingly, according to the second speech feature, the second text feature and the speech recognition model, the specific implementation process of the training element learning adaptation model may be: extracting the initial model parameters of the speech recognition model; if it is determined that the current data block is the first divided data block, calculate the loss value, loss gradient and new model parameters of the meta-learning adaptive model in the first data block according to the initial model parameters and the second speech feature and the second text feature of the current data block; if the current data block is determined If the block is not the first data block, the loss value, loss gradient and New model parameters; if it is determined that all data blocks have been trained, the loss value, loss gradient and new model parameters calculated by each data block are used to determine the second loss function of the meta-learning adaptation model; if it is determined that the second loss function is smaller than the second loss function If the preset threshold is set, it is determined that the meta-learning adaptation model complies with the second training standard. The second preset threshold may be set according to the actual training accuracy requirement, and should be a value greater than 0 and less than 1. The larger the set value, the higher the training accuracy of the speech recognition model.

其中，J为第二损失函数，y _c+1为c+1数据块的第二文本特征，x _c+1为c+1数据块的第二语音特征，θ′为元学习适应模型在c数据块计算出的新模型参数，L(y _c+1,f(x _c+1；θ′))为元学习适应模型在c+1数据块计算出的损失值。 Among them, the feature description of the second loss function calculation formula is:

Among them, J is the second loss function, y _c+1 is the second text feature of the c+1 data block, x _c+1 is the second speech feature of the c+1 data block, θ′ is the meta-learning adaptation model in c The new model parameters calculated by the data block, L(y _c+1 , f(x _c+1 ; θ′)) is the loss value calculated by the meta-learning adaptation model in the c+1 data block.

经过第一层LSTM，可以得到输入的隐层表示h _t。将h _t输入第二层LSTM后，可以得到其中的遗忘门参数f _t和输入门参数i _t，结合原参数θ _t和对应的梯度

可以得到新模型的参数： In this application, the network structure of the meta-learning adaptation model may adopt a two-layer Long Short-Term Memory (LSTM). The input of the first layer of LSTM at time t includes the original model parameter θ _t , the cross-entropy loss L _t on the data block and its corresponding gradient

After the first layer of LSTM, the input hidden layer representation h _t can be obtained. After inputting h _t into the second layer of LSTM, the forgetting gate parameter f _t and the input gate parameter i _t can be obtained, combined with the original parameter θ _t and the corresponding gradient

The parameters of the new model can be obtained:

204. Based on the meta-learning adaptation model, adjust the initial model parameters of the speech recognition model to target model parameters matching the target speech type.

For this embodiment, after training a meta-learning adaptation model that meets the preset training standard, the meta-learning adaptation model can be used to adaptively adjust the model parameters of the speech recognition model according to the actual application scenario. Specifically, in order to obtain the target model parameters that match the target speech type, it is necessary to extract a small amount of adaptive speech data that matches the target speech type, and input the initial model parameters of the speech recognition model and the loss and loss gradient on the adaptive speech data into the meta-learning In the adaptation model, the target model parameters in the speech recognition model that match the target speech type can be obtained. The adaptation data needs to belong to the same speech type as the target speech to be recognized. For example, if the speech recognition of the present application is applied to infant education, because the target speech to be recognized is an infant, the meta-learning adaptation model is used to analyze the speech When the model parameters of the recognition model are adjusted, a small amount of infant voice data can be used as adaptive voice data to further determine the target model parameters that match the voice type of infants and young children, so as to obtain high accuracy in infant voice recognition. speech recognition model. For another example, if the speech recognition of the present application is applied to a popular-oriented intelligent speech service, because the target speech to be recognized is a group of people in various regions, in order to avoid the influence of regional accents on the speech recognition effect, it can be selected according to the region to be recognized. For the corresponding adaptive voice data, if the area to be recognized is the Northeast region, a small amount of Northeast voice data can be selected as the adaptive voice data, so as to train a voice recognition model that can eliminate accent interference.

205. Use the speech recognition model configured with the target model parameter to recognize the target speech under the target speech type.

Wherein, after it is determined that the target model parameters matching the adaptive speech data are obtained, the target model parameters can be updated to the speech recognition model, and then the updated speech recognition model can be used to achieve accurate recognition of the target speech under the target speech type, thereby It can get better speech recognition effect than pre-trained speech recognition model.

With the above-mentioned meta-learning-based adaptive speech recognition method, the present application can firstly pre-train the speech recognition model through sample speech data, and further regard the speaker adaptation task as a task in the meta-learning, which is designed to be used according to different speech sounds. Type, a meta-learning adaptation model that adjusts model parameters in a speech recognition model. Then, when the target speech type is determined, the initial model parameters of the speech recognition model can be adjusted to the target model parameters matching the target speech type based on the meta-learning adaptation model, and then the speech recognition model configured with the target model parameters can be used to realize the target speech type. Targeted and precise recognition of target speech. In this application, the meta-learning adaptation model is used to realize the self-adaptive adjustment of the model parameters in the speech recognition model, which not only reduces the instability of manual design, but also enables the model parameters to be updated for different application scenarios, thereby ensuring the accuracy of speech recognition. Accuracy. And when using the meta-learning adaptation model to determine the model parameters of the speech recognition model, only a small amount of adaptive speech data is needed, and less adaptive speech data will make the updated model parameters easier to overfit in the application scenario, so the parameters can also be reduced. The updated risk of overfitting.

For this embodiment, in a specific application scenario, the speech recognition system may refer to the schematic flowchart of the meta-learning-based adaptive speech recognition system shown in FIG. 4 . Specifically, after the sample speech data is determined and preprocessed, the Feature extraction, using the extracted first voice feature and the first text feature to pre-train the voice adaptation model, and then according to the second voice feature corresponding to the sample voice data, the second text feature and the original model parameter training element learning corresponding to the voice recognition model Fit the model. Then, using the adaptive voice data matching the target voice type, after extracting the voice features and text features, input them together with the initial model parameters of the voice recognition model into the trained meta-learning adaptation model, and then obtain the matching target voice type. The initial model parameters of the speech recognition model are updated to the target model parameters, and then the updated speech recognition model (speaker adaptation model) can be used to recognize the target speech under the target speech type.

Further, as a specific implementation of the methods shown in FIG. 1 and FIG. 2 , an embodiment of the present application provides an adaptive speech recognition device based on meta-learning. As shown in FIG. 5 , the device includes: a training module 31 , an adjustment module 32. Identification module 33;

The training module 31 can be used to train the speech recognition model and the meta-learning adaptation model by using the preprocessed sample speech data;

The adjustment module 32 can be used to adjust the initial model parameters of the speech recognition model to the target model parameters matching the target speech type based on the meta-learning adaptation model;

The recognition module 33 can be used to recognize the target speech under the target speech type by using the speech recognition model configured with the target model parameters.

In a specific application scenario, in order to obtain a speech recognition model and a meta-learning adaptation model through pre-training, as shown in FIG. 6 , the training module 31 may specifically include: a processing unit 311 , a first training unit 312 , and a second training unit 313 ;

The processing unit 311 can be used to preprocess the sample voice data, and mark the first voice feature and the first text feature corresponding to the sample voice data, and the preprocessing at least includes pre-emphasis processing, framing processing, and windowing processing;

a first training unit 312, which can be used to train a speech recognition model that meets the first training standard based on the first speech feature and the first text feature;

The second training unit 313 can be configured to use the sample speech data and the speech recognition model to train a meta-learning adaptation model that meets the second training standard.

Correspondingly, the first training unit 312 can be specifically configured to input the first speech feature into the speech recognition model to obtain the text output result; calculate the first loss function according to the text output result and the first text feature; if it is determined that the first loss function is smaller than the first loss function; A preset threshold value determines that the speech recognition model complies with the first training standard.

In a specific application scenario, the second training unit 313 can be specifically configured to divide the sample voice data into a preset number of data blocks, and extract the second voice feature and the second text feature of each data block; according to the second voice feature , a second text feature, and a speech recognition model to train a meta-learning adaptation model that meets the second training criteria.

Correspondingly, when training a meta-learning adaptive model that meets the second training standard according to the second voice feature, the second text feature, and the voice recognition model, the second training unit 313 can specifically be used to extract the initial model parameters of the voice recognition model; If it is determined that the current data block is the first divided data block, according to the initial model parameters and the second speech feature and the second text feature of the current data block, the loss value, Loss gradient and new model parameters; if it is determined that the current data block is not the first data block, the meta-learning adaptation model is calculated based on the new model parameters of the previous data block and the second speech feature and second text feature of the current data block. The loss value, loss gradient and new model parameters in the current data block; if it is determined that all data blocks have completed training, the loss value, loss gradient and new model parameters calculated by each data block are used to determine the second loss of the meta-learning adaptation model function; if it is determined that the second loss function is smaller than the second preset threshold, it is determined that the meta-learning adaptive model meets the second training standard.

Among them, the feature description of the second loss function calculation formula is:

其中，J为第二损失函数，y _c+1为c+1数据块的第二文本特征，x _c+1为c+1数据块的第二语音特征，θ′为元学习适应模型在c数据块计算出的新模型参数，L(y _c+1,f(x _c+1；θ′))为元学习适应模型在c+1数据块计算出的损失值。

Among them, J is the second loss function, y _c+1 is the second text feature of the c+1 data block, x _c+1 is the second speech feature of the c+1 data block, θ′ is the meta-learning adaptation model in c The new model parameters calculated by the data block, L(y _c+1 , f(x _c+1 ; θ′)) is the loss value calculated by the meta-learning adaptation model in the c+1 data block.

In a specific application scenario, in order to determine the target model parameters matching the target speech type based on the meta-learning adaptation model, as shown in FIG. 6 , the adjustment module 32 may specifically include: an extraction unit 321 and an acquisition unit 322;

Extraction unit 321, for extracting adaptive voice data matching the target voice type;

The obtaining unit 322 is configured to input the initial model parameters of the speech recognition model and the adaptive speech data into the meta-learning adaptation model, and obtain the target model parameters matching the target speech type.

Correspondingly, the recognition module 33 can be specifically configured to update the initial model parameters of the speech recognition model to the target model parameters, so as to use the updated speech recognition model to recognize the target speech under the target speech type.

It should be noted that, for other corresponding descriptions of the functional units involved in the meta-learning-based adaptive speech recognition apparatus provided in this embodiment, reference may be made to the corresponding descriptions in FIG. 1 to FIG. 2 , which will not be repeated here.

Based on the above methods as shown in FIGS. 1 to 2 , correspondingly, the present embodiment further provides a storage medium on which computer-readable instructions (or computer programs) are stored, and the readable instructions (or computer programs) are When executed by the processor, the above-mentioned meta-learning-based adaptive speech recognition method shown in FIG. 1 to FIG. 2 is implemented.

Optionally, the storage medium involved in this application may be a readable storage medium, or may be referred to as a computer-readable storage medium. The storage medium, such as a readable storage medium, may be non-volatile, such as a non-volatile readable storage medium; or, may also be volatile, such as a volatile readable storage medium.

Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.), including several The instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present application.

Based on the methods shown in FIG. 1 to FIG. 2 and the virtual device embodiments shown in FIG. 5 and FIG. 6 , in order to achieve the above purpose, this embodiment also provides a computer device, the computer device includes a storage medium and a processing The storage medium may be a non-volatile storage medium for storing a computer program; a processor for executing the computer program to implement the above-mentioned meta-learning-based adaptive speech recognition method shown in FIG. 1 to FIG. 2 . For example, the processor may be configured to perform: using the preprocessed sample speech data to train a speech recognition model and a meta-learning adaptation model; based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to match the target speech type matching target model parameters; using the speech recognition model configured with the target model parameters to recognize the target speech under the target speech type. Optionally, when the processor executes, other steps of the method in the foregoing embodiment may also be implemented, which will not be repeated here.

Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (Radio Frequency, RF) circuit, a sensor, an audio circuit, a WI-FI module, and the like. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, and the like. Optional network interfaces may include standard wired interfaces, wireless interfaces (such as WI-FI interfaces), and the like.

Those skilled in the art can understand that the structure of a computer device provided in this embodiment does not constitute a limitation on the physical device, and may include more or less components, or combine some components, or arrange different components.

The non-volatile storage medium may further include an operating system and a network communication module. An operating system is a program that manages the hardware and software resources of the above-mentioned computer equipment, and supports the operation of information processing programs and other software and/or programs. The network communication module is used to realize the communication between various components inside the non-volatile storage medium, and communicate with other hardware and software in the information processing entity device.

From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, and can also be implemented by hardware.

By applying the technical solution of the present application, compared with the current prior art, the present application can firstly pre-train the speech recognition model through sample speech data, and further regard the speaker adaptation task as a task in meta-learning, which is designed for According to different speech types, the meta-learning adaptation model of the model parameters in the speech recognition model is adjusted. Then, when the target speech type is determined, the initial model parameters of the speech recognition model can be adjusted to the target model parameters matching the target speech type based on the meta-learning adaptation model, and then the speech recognition model configured with the target model parameters can be used to realize the target speech type. Targeted and precise recognition of target speech. In this application, the meta-learning adaptation model is used to realize the self-adaptive adjustment of the model parameters in the speech recognition model, which not only reduces the instability of manual design, but also enables the model parameters to be updated for different application scenarios, thereby ensuring the accuracy of speech recognition. Accuracy. And when using the meta-learning adaptation model to determine the model parameters of the speech recognition model, only a small amount of adaptive speech data is needed, and less adaptive speech data will make the updated model parameters easier to overfit in the application scenario, so the parameters can also be reduced. The updated risk of overfitting.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred implementation scenario, and the modules or processes in the accompanying drawing are not necessarily necessary to implement the present application. Those skilled in the art can understand that the modules in the device in the implementation scenario may be distributed in the device in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the implementation scenario with corresponding changes. The modules of the above implementation scenarios may be combined into one module, or may be further split into multiple sub-modules.

The above serial numbers in the present application are only for description, and do not represent the pros and cons of the implementation scenarios. The above disclosures are only a few specific implementation scenarios of the present application, however, the present application is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present application.

Claims (20)

A meta-learning-based adaptive speech recognition method, comprising: Use the preprocessed sample speech data to train the speech recognition model and the meta-learning adaptation model; Based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type; The target speech under the target speech type is recognized by using the speech recognition model configured with the target model parameters. The method according to claim 1, wherein the use of the preprocessed sample speech data to train a speech recognition model and a meta-learning adaptation model specifically includes: Preprocessing the sample voice data, and marking the first voice feature and the first text feature corresponding to the sample voice data, and the preprocessing at least includes pre-emphasis processing, frame segmentation processing, and windowing processing; Train a speech recognition model that meets the first training standard based on the first speech feature and the first text feature; Using the sample speech data and the speech recognition model, a meta-learning adaptation model that meets the second training criterion is trained. The method according to claim 2, wherein the training of a speech recognition model conforming to a first training standard based on the first speech feature and the first text feature specifically comprises: Inputting the first speech feature into the speech recognition model to obtain a text output result; calculating a first loss function according to the text output result and the first text feature; If it is determined that the first loss function is less than a first preset threshold, it is determined that the speech recognition model meets the first training standard; The use of the sample speech data and the speech recognition model to train a meta-learning adaptation model that meets the second training standard specifically includes: dividing the sample speech data into a preset number of data blocks, and extracting the second speech feature and the second text feature of each of the data blocks; According to the second speech feature, the second text feature, and the speech recognition model, a meta-learning adaptation model that meets the second training standard is trained. The method according to claim 3, wherein, according to the second speech feature, the second text feature and the speech recognition model, training a meta-learning adaptation model that meets the second training standard, specifically comprising: extracting initial model parameters of the speech recognition model; If it is determined that the current data block is the first divided data block, then according to the initial model parameters and the second speech feature and the second text feature of the current data block, calculate the meta-learning adaptation model in the first data block. loss value, loss gradient, and new model parameters; If it is determined that the current data block is not the first data block, then according to the new model parameters of the previous data block and the second speech feature and the second text feature of the current data block, calculate the meta-learning adaptation model in the current data block. loss value, loss gradient, and new model parameters in ; If it is determined that all data blocks have completed training, then the second loss function of the meta-learning adaptation model is determined by using the loss value, loss gradient and new model parameters calculated by each data block; If it is determined that the second loss function is smaller than a second preset threshold, it is determined that the meta-learning adaptation model meets the second training standard. The method according to claim 4, wherein the feature description of the second loss function calculation formula is:

Among them, J is the second loss function, y _c+1 is the second text feature of the c+1 data block, x _c+1 is the second speech feature of the c+1 data block, and θ′ is the meta-learning adaptation model The new model parameters calculated in the c data block, L(y _c+1 , f(x _c+1 ; θ′)) is the loss value calculated by the meta-learning adaptation model in the c+1 data block. The method according to claim 5, wherein, adjusting the initial model parameters of the speech recognition model to the target model parameters matching the target speech type based on the meta-learning adaptation model specifically includes: Extract adaptive speech data matching the target speech type; The initial model parameters of the speech recognition model and the adapted speech data are input into the meta-learning adaptation model to obtain target model parameters matching the target speech type. The method according to claim 6, wherein the identifying the target voice under the target voice type by using the voice recognition model configured with the target model parameters specifically includes: The initial model parameters of the speech recognition model are updated to the target model parameters, so as to use the updated speech recognition model to recognize the target speech under the target speech type. A meta-learning-based adaptive speech recognition device, comprising: The training module is used to train the speech recognition model and the meta-learning adaptation model by using the preprocessed sample speech data; an adjustment module for adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type based on the meta-learning adaptation model; The recognition module is used for recognizing the target speech under the target speech type by using the speech recognition model configured with the target model parameter. A storage medium on which a computer program is stored, wherein, when the program is executed by a processor, the following methods are implemented: Use the preprocessed sample speech data to train the speech recognition model and the meta-learning adaptation model; Based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type; The target speech under the target speech type is recognized by using the speech recognition model configured with the target model parameters. The storage medium according to claim 9, wherein performing the training of the speech recognition model and the meta-learning adaptation model using the preprocessed sample speech data comprises: Preprocessing the sample voice data, and marking the first voice feature and the first text feature corresponding to the sample voice data, and the preprocessing at least includes pre-emphasis processing, frame segmentation processing, and windowing processing; Train a speech recognition model that meets the first training standard based on the first speech feature and the first text feature; Using the sample speech data and the speech recognition model, a meta-learning adaptation model that meets the second training criterion is trained. The storage medium according to claim 10, wherein performing the training of a speech recognition model conforming to a first training standard based on the first speech feature and the first text feature comprises: Inputting the first speech feature into the speech recognition model to obtain a text output result; calculating a first loss function according to the text output result and the first text feature; If it is determined that the first loss function is less than a first preset threshold, it is determined that the speech recognition model meets the first training standard; Executing the using the sample speech data and the speech recognition model to train a meta-learning adaptation model that meets the second training standard, including: dividing the sample speech data into a preset number of data blocks, and extracting the second speech feature and the second text feature of each of the data blocks; According to the second speech feature, the second text feature, and the speech recognition model, a meta-learning adaptation model that meets the second training standard is trained. The storage medium according to claim 11, wherein performing the training of a meta-learning adaptation model that meets the second training standard according to the second speech feature, the second text feature, and the speech recognition model comprises: extracting initial model parameters of the speech recognition model; If it is determined that the current data block is the first divided data block, then according to the initial model parameters and the second speech feature and the second text feature of the current data block, calculate the meta-learning adaptation model in the first data block. loss value, loss gradient, and new model parameters; If it is determined that the current data block is not the first data block, then according to the new model parameters of the previous data block and the second speech feature and the second text feature of the current data block, calculate the meta-learning adaptation model in the current data block. loss value, loss gradient, and new model parameters in ; If it is determined that all data blocks have completed training, then the second loss function of the meta-learning adaptation model is determined by using the loss value, loss gradient and new model parameters calculated by each data block; If it is determined that the second loss function is smaller than a second preset threshold, it is determined that the meta-learning adaptation model meets the second training standard. The storage medium according to claim 12, wherein the feature description of the second loss function calculation formula is:

Among them, J is the second loss function, y _c+1 is the second text feature of the c+1 data block, x _c+1 is the second speech feature of the c+1 data block, and θ′ is the meta-learning adaptation model The new model parameters calculated in the c data block, L(y _c+1 , f(x _c+1 ; θ′)) is the loss value calculated by the meta-learning adaptation model in the c+1 data block. The storage medium according to claim 13, wherein, performing the adaptation model based on the meta-learning, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type, comprising: Extract adaptive speech data matching the target speech type; The initial model parameters of the speech recognition model and the adapted speech data are input into the meta-learning adaptation model to obtain target model parameters matching the target speech type. A computer device, comprising a storage medium, a processor and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the following methods when executing the program: Use the preprocessed sample speech data to train the speech recognition model and the meta-learning adaptation model; Based on the meta-learning adaptation model, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type; The target speech under the target speech type is recognized by using the speech recognition model configured with the target model parameters. The computer device according to claim 15, wherein performing the training of the speech recognition model and the meta-learning adaptation model using the preprocessed sample speech data comprises: Preprocessing the sample voice data, and marking the first voice feature and the first text feature corresponding to the sample voice data, and the preprocessing at least includes pre-emphasis processing, frame segmentation processing, and windowing processing; Train a speech recognition model that meets the first training standard based on the first speech feature and the first text feature; Using the sample speech data and the speech recognition model, a meta-learning adaptation model that meets the second training criterion is trained. The computer device according to claim 16, wherein performing the training of a speech recognition model conforming to a first training criterion based on the first speech feature and the first text feature comprises: Inputting the first speech feature into the speech recognition model to obtain a text output result; calculating a first loss function according to the text output result and the first text feature; If it is determined that the first loss function is less than a first preset threshold, it is determined that the speech recognition model meets the first training standard; Executing the using the sample speech data and the speech recognition model to train a meta-learning adaptation model that meets the second training standard, including: dividing the sample speech data into a preset number of data blocks, and extracting the second speech feature and the second text feature of each of the data blocks; According to the second speech feature, the second text feature, and the speech recognition model, a meta-learning adaptation model that meets the second training standard is trained. The computer device according to claim 17, wherein performing the training of a meta-learning adaptive model that meets the second training standard according to the second speech feature, the second text feature, and the speech recognition model comprises: extracting initial model parameters of the speech recognition model; If it is determined that the current data block is the first divided data block, then according to the initial model parameters and the second speech feature and the second text feature of the current data block, calculate the meta-learning adaptation model in the first data block. loss value, loss gradient, and new model parameters; If it is determined that the current data block is not the first data block, then according to the new model parameters of the previous data block and the second speech feature and the second text feature of the current data block, calculate the meta-learning adaptation model in the current data block. loss value, loss gradient, and new model parameters in ; If it is determined that all data blocks have completed training, then the second loss function of the meta-learning adaptation model is determined by using the loss value, loss gradient and new model parameters calculated by each data block; If it is determined that the second loss function is smaller than a second preset threshold, it is determined that the meta-learning adaptation model meets the second training standard. The computer device according to claim 18, wherein the feature description of the second loss function calculation formula is:

Among them, J is the second loss function, y _c+1 is the second text feature of the c+1 data block, x _c+1 is the second speech feature of the c+1 data block, and θ′ is the meta-learning adaptation model The new model parameters calculated in the c data block, L(y _c+1 , f(x _c+1 ; θ′)) is the loss value calculated by the meta-learning adaptation model in the c+1 data block. The computer device according to claim 19, wherein, performing the adaptation model based on the meta-learning, adjusting the initial model parameters of the speech recognition model to target model parameters matching the target speech type, comprising: Extract adaptive speech data matching the target speech type; The initial model parameters of the speech recognition model and the adapted speech data are input into the meta-learning adaptation model to obtain target model parameters matching the target speech type.

Images