WO2019037382A1 - Emotion recognition-based voice quality inspection method and device, equipment and storage medium - Google Patents

Abstract

A voice quality inspection method, device, device and storage medium based on emotion recognition. The voice quality detection method based on emotion recognition includes: acquiring voice data to be tested; sending the voice data to be tested to the voice emotion recognition platform for emotion recognition; and transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result. When the emotion recognition method based on emotion recognition performs emotion recognition, it has the advantages of high efficiency and low labor cost.

Description

Speech quality inspection method, device, device and storage medium based on emotion recognition

This patent application is based on the Chinese invention patent application filed on August 24, 2017, with the application number of 201710734303.X, entitled "Speech quality inspection method, device, device and storage medium based on emotion recognition", and requires priority right.

Technical field

The present invention relates to the field of voice quality inspection technology, and in particular, to a voice quality inspection method, apparatus, device and storage medium based on emotion recognition.

Background technique

In the banking, securities, insurance and P2P (peer-to-peer lending, P2T), the service organization has a customer service voice question answering system. The agent of the service organization provides services to the customer through the customer service voice question answering system. Voice quality inspection is to monitor the call between the agent and the customer service to evaluate the quality of the call, quality of service, quality of service and satisfaction, so as to improve the quality of service based on the quality inspection results. At present, the quality of voice quality inspection is mainly carried out by means of manual sampling, which has the problems of low sampling efficiency, untimely strain and a large amount of manpower and material resources.

Summary of the invention

The present application provides a method, device, device and storage medium for voice quality detection based on emotion recognition, so as to solve the problem that the current voice quality inspection adopts manual sampling mode.

In a first aspect, the present application provides a voice quality verification method based on emotion recognition, including:

Obtaining voice data to be tested;

Performing feature extraction on the voice data to be tested to obtain a voice feature;

Identifying the voice feature by using an emotion recognition model to obtain an emotion recognition result;

Transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result.

In a second aspect, the present application provides a voice quality verification device based on emotion recognition, including:

a voice data acquiring module to be used for acquiring voice data to be tested;

a voice data feature acquiring module, configured to perform feature extraction on the voice data to be tested, and acquire a voice feature;

An emotion recognition result obtaining module, configured to identify the voice feature by using an emotion recognition model, and obtain an emotion recognition result;

And an emotion recognition result feedback module, configured to send the emotion recognition result to the associated terminal, so that the association terminal displays the emotion recognition result.

In a third aspect, the present application provides a terminal device including a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, the processor executing the computer readable instructions The following steps are implemented:

Obtaining voice data to be tested;

Performing feature extraction on the voice data to be tested to obtain a voice feature;

Identifying the voice feature by using an emotion recognition model to obtain an emotion recognition result;

Transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result.

In a fourth aspect, the present application provides a computer readable storage medium storing computer readable instructions that, when executed by a processor, implement the following steps:

Obtaining voice data to be tested;

Performing feature extraction on the voice data to be tested to obtain a voice feature;

Identifying the voice feature by using an emotion recognition model to obtain an emotion recognition result;

Transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result.

In the speech recognition-based voice quality inspection method, device, device and storage medium provided by the present application, feature extraction is performed on the voice data to be measured to acquire a voice feature, and then the voice recognition model is used to identify the voice feature to obtain the emotion recognition. As a result, the emotion recognition result is displayed by the associated terminal, so that the user can understand the speaker emotion corresponding to the voice data to be tested by viewing the emotion recognition result. The method, apparatus, device and storage medium for voice recognition based on emotion recognition can realize intelligent recognition of the voice data to be measured to obtain the emotion recognition result, and the recognition process has high processing efficiency, and can implement the test corresponding to the speaker. Timely and comprehensive sampling of voice data, without manual intervention, is conducive to saving labor costs.

DRAWINGS

In order to more clearly illustrate the technical solutions of the present application, the drawings to be used in the description of the present application will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present application. For ordinary technicians, other drawings can be obtained based on these drawings without paying for creative labor.

1 is a flow chart of a method for voice quality detection based on emotion recognition in Embodiment 1.

FIG. 2 is a specific flowchart of step S30 in FIG. 1.

FIG. 3 is another specific flowchart of step S30 in FIG. 1.

4 is another specific flowchart of the voice quality-based voice quality checking method in Embodiment 1.

FIG. 5 is a schematic block diagram of a voice quality checking apparatus based on emotion recognition in Embodiment 2.

Figure 6 is a schematic diagram of a terminal device in Embodiment 4.

Detailed ways

The technical solutions in the present application will be clearly and completely described in the following with reference to the drawings in the present application. It is obvious that the described embodiments are a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

Example 1

FIG. 1 shows a voice quality inspection method based on emotion recognition in the present embodiment. The emotion recognition-based voice quality inspection method is applied to a financial institution such as a bank, a securities, an insurance, and a peer-to-peer lending (P2P), or a terminal device of another institution that needs to perform emotion recognition, for use in a terminal device of a financial institution such as a peer-to-peer lending (P2P) The speaker's voice data to be tested is identified to determine the emotion of the speaker. Among them, the terminal equipment includes but is not limited to a PC, a smart phone, a tablet computer, and a customer service voice question answering system. In this embodiment, the terminal device is a customer service voice question answering system. As shown in FIG. 1, the emotion recognition based voice quality inspection method comprises the following steps:

S10: Acquire voice data to be tested.

The voice data to be tested refers to the voice data of the speaker collected by the terminal device. The voice data to be tested may be voice data in wav, mp3 or other formats. It can be understood that each voice data to be tested carries a data source ID, which is an identifier for uniquely identifying the speaker of the voice data to be tested. When the agent communicates with the customer, the recording module integrated on the terminal device or the recording device connected to the terminal device collects the voice data to be tested.

In this embodiment, the manner in which the voice data to be tested is obtained includes online real-time acquisition and offline acquisition. The online real-time acquisition refers to recording the content of the call between the client and the agent during the conversation between the client and the agent to obtain the voice data to be tested. Offline acquisition refers to acquiring the voice data to be tested that has been saved in the database from the background of the system connected to the terminal device. It can be understood that each voice data to be tested carries a data source ID, and the speaker corresponding to the data source ID may be a client or an agent. Therefore, the data source ID may be a customer ID for uniquely identifying the customer, or may be an agent ID for uniquely identifying the agent. The customer ID may be the customer's ID number, mobile phone number, or the account number when the organization handles the business. The agent ID can be the number of the agent inside the organization.

S20: Feature extraction of the voice data to be measured, and acquiring voice features.

It will be appreciated that speech features include, but are not limited to, prosodic features, phonological features, spectral features, lexical features, and voiceprint features. Among them, the prosodic feature, also known as the super-sound quality feature or the super-segment feature, refers to the change in pitch, pitch length and sound intensity in the speech other than the sound quality feature. The prosody features include, but are not limited to, the pitch frequency, the utterance duration, the utterance amplitude, and the utterance rate in the present embodiment. Sound quality features include, but are not limited to, formants F1-F3, band energy distribution, harmonic signal to noise ratio, and short-term energy jitter in this embodiment. Spectral characteristics, also known as vibrational spectral features, refer to the decomposition of complex oscillations into resonant waveforms of different amplitudes and frequencies, and the amplitudes of these resonant oscillations are arranged in a frequency pattern. The spectral features are combined with prosodic features and sound quality features to improve the anti-noise effect of the characteristic parameters. In this embodiment, the spectral features are Mel-Frequency Cepstral Coefficients (MFCC), which can reflect the auditory characteristics of the human ear. The vocabulary feature is a part of speech feature for embodying words in the speech data to be tested, including but not limited to positive words and negative words in the embodiment. The part-of-speech feature is combined with other phonetic features to facilitate the recognition of the speaker's emotion corresponding to the speech data to be tested. The voiceprint feature (i.e., i-vector feature) is a speaker-related feature that, combined with other phonetic features, can more effectively improve the accuracy of recognition in the speech recognition process.

Specifically, the process of performing feature extraction on the voice data to be tested specifically includes pre-emphasis processing on the voice data to be tested, performing frame division and windowing operations on the processed voice data, and then performing fast Fourier transform and logarithmic operation processing. Finally, the discrete cosine transform is performed to obtain the above-mentioned speech features. Then, the above speech features are spliced into feature vectors, and the feature vectors are used as input emotion recognition models for emotion recognition.

S30: The emotion recognition model is used to identify the voice features, and the emotion recognition result is obtained.

Among them, the emotion recognition model is a pre-trained model for identity verification in the organization. The results of the emotion recognition include, but are not limited to, recognizing emotions such as happiness, anger, sadness, trouble, and calmness of the speaker corresponding to the voice feature. Specifically, the terminal device uses the pre-trained emotion recognition model to identify the voice feature, so as to obtain the emotion recognition results such as happiness, anger, sadness, trouble, and calmness carried by the speaker when speaking. In this embodiment, the emotion recognition model is used to intelligently identify the voice features to obtain the emotion recognition result, and the recognition process has high processing efficiency, and can realize timely and comprehensive sampling inspection of the voice data corresponding to the speaker, without manual intervention. Conducive to saving labor costs.

In a specific implementation manner, in S30, the emotion recognition model is used to identify the voice feature, and the emotion recognition result is obtained, which specifically includes the following steps, as shown in FIG. 2:

S311: Using a neural network-based emotion recognition model to identify the voice features and determine whether the accurate recognition result can be output.

Among them, the neural network-based emotion recognition model is a model for identifying emotions in the voice data obtained by training the training voice data using the neural network model. The emotion recognition model based on the neural network includes an input layer, a hidden layer and an output layer, and the voice feature acquired in step S20 is input into the input layer of the emotion recognition model based on the neural network, and the voice feature is identified and processed through the hidden layer, and then passed through the output layer. The recognition result is output, and the recognition result includes an accurate recognition result and a fuzzy recognition result. The accurate recognition result is a recognition result for indicating that the voice feature corresponds to a specific emotion; and the fuzzy recognition result is a recognition result for indicating that the voice feature cannot correspond to a specific emotion.

Specifically, the training process of the emotion recognition model based on the neural network is as follows: First, the training voice data is acquired, and the training voice data is emotionally labeled, so that the training voice data carries the emotion tag. Among them, the training voice data is voice data for training the emotion recognition model. Each training emotional data corresponding to each emotional label is selected 500, so that the training speech data corresponding to the five emotions of happiness, anger, sadness, trouble, and calm are proportionally, so as to avoid over-fitting in the training process of the emotion recognition model. . Then, feature extraction is performed on the training speech data carrying the emotional tag, and the training speech feature carrying the emotional tag is obtained, which is specifically represented by the training speech feature x and the corresponding emotional tag y, such as (training speech feature 1, happy), (training speech) Feature 2, anger)... (training speech feature x, emotion y). Then, using the logistic regression algorithm in the neural network model, iteratively calculates all the training speech features to extract the features corresponding to emotions such as happiness, anger, sadness, trouble and calmness from the training speech features. After 20,000 iterations, the loss of the trained model converges, then the training is stopped, and the finally trained neural network-based emotion recognition model is obtained, which makes the neural network-based emotion recognition model have strong nonlinear fitting ability. It can map complex nonlinear relationships and has strong robustness and memory ability. Further, the neural network of this embodiment is specifically a Deep Neural Networks (DNN).

When the emotion recognition model based on the neural network is used to identify the speech features corresponding to the speech data, the speech features corresponding to the speech data to be tested are input into the input layer of the neural network based emotion recognition model, and the emotion recognition model based on the neural network is used. The hidden layer performs emotion recognition according to the characteristics learned in the pre-training process, acquires the corresponding emotion recognition result, and outputs the emotion recognition result from the output layer. In the process of identifying the voice data to be measured based on the neural network-based emotion recognition model, the hidden layer calculates the probability of acquiring the voice data to be tested as happy, angry, sad, troublesome or calm, and compares the highest probability with the second highest. Whether the difference of the probability is greater than the preset probability difference. If the difference between the highest probability and the second highest probability is greater than the preset probability difference, the emotion corresponding to the highest probability is output as an accurate recognition result. If the difference between the highest probability and the second highest probability is not greater than the preset probability difference, the neural network based emotion recognition model cannot output an accurate recognition result, but outputs a fuzzy recognition result. The preset probability difference is a value set in advance for evaluating whether the emotion category can be determined.

If the preset probability difference is 20%, if the speech characteristics are identified as happy, angry, sad, troubled or calm, respectively, 2%, 60%, 15%, 20% and 3%, the highest probability is 60%. The second highest probability is 20%, and the difference between the highest probability and the second highest probability is 40%, which is greater than the preset probability difference, and the emotion corresponding to the highest probability is output as the recognition result. Conversely, if the speech characteristics are identified as happy, angry, sad, troubled, or calm, respectively, 2%, 40%, 20%, 35%, and 3%, the highest probability is 40%, and the second highest probability is 35%. The highest probability and the second highest probability of the impairment are less than the preset probability difference, and the accurate recognition result cannot be output, but the fuzzy recognition result is output.

S312: If an accurate recognition result can be output, the accurate recognition result is used as the emotion recognition result.

In this embodiment, if the emotion recognition model based on the neural network can output an accurate recognition result, the highest probability that the speaker emotion corresponding to the voice data to be tested belongs to happiness, anger, sadness, trouble, and calm is much greater than the second highest probability. If the difference between the two is greater than the preset probability difference, the recognition result output by the emotion recognition model based on the neural network is more accurate, and the accurate recognition result can be directly output as the emotion recognition result.

S313: If the accurate recognition result cannot be output, the emotion recognition model based on the support vector machine is used to identify the voice feature, and the emotion recognition result is obtained.

In this embodiment, if the emotion recognition model based on the neural network cannot output an accurate recognition result, that is, the emotion recognition model based on the neural network outputs a fuzzy recognition result, indicating that the speaker emotion corresponding to the voice data to be tested belongs to happiness, anger, sadness, The difference between the highest probability far and the second high probability in the noisy and calm is not greater than the preset probability difference, that is, the speaker does not accurately recognize a specific emotion. Therefore, the emotion recognition model based on support vector machine is needed to further identify the speech features to obtain the emotion recognition results, thereby further improving the accuracy of emotion recognition.

The emotion recognition model based on the support vector machine is a model for identifying emotions in the voice data obtained by training the training voice data by using the support vector machine model. Among them, the support vector machine (SVM) is a classifier that supports vector operations, and the support vector machine can realize linear classification and nonlinear classification. The emotion recognition model of support vector machine has less computational complexity, and can determine the final result according to a few support vectors. It helps to grasp key samples and eliminate redundant samples in the training process, which has better robustness.

The training process of the emotion recognition model of the support vector machine is as follows: SVM training is to separate different types of emotional attributes on a hyperplane, including five emotions of happiness, anger, sadness, irritability and calmness. In this embodiment, the hyperplane is divided into five-dimensional hyperplanes according to the five kinds of emotional attributes, and the dividing lines dividing the five emotions are searched in the five-dimensional hyperplane, and the expression of the dividing line is obtained to complete the training of the SVM. . Specifically, by inputting the speech features of the training speech data (ie, prosodic features, sound quality features, spectral features, lexical features, and voiceprint features), audio feature extraction is performed on the data to be measured, and an optimal solution space corresponding to the feature is obtained, and the space is To express the space for the corresponding emotions, complete the emotional classification and judgment of the input training speech data.

The SVM is a two-category model, and its implementation may be in the form of a binary tree, that is, each attribute is separately judged to determine whether it belongs to the emotional attribute or does not belong to the emotional attribute. Since the SVM is a two-category model, the SVM-based happy recognition model, the anger recognition model, the sadness recognition model, the irritability recognition model, and the calm recognition model are separately created in the emotion recognition model based on the support vector machine in this embodiment. When the speech recognition model based on the support vector machine is used to identify the speech features corresponding to the speech data, the speech features need to be identified by the happy recognition model, the angry recognition model, the sad recognition model, the irritability recognition model and the calm recognition model, respectively. Obtain the corresponding emotional scores; then compare the five emotional scores and select the highest-scoring emotion as the emotional recognition result.

In this embodiment, the neural network-based emotion recognition model is used to identify the voice features corresponding to the voice data, and the voice features with clearer emotional attributes are more accurately identified, and the recognition process is faster; then the support vector machine is used. The emotion recognition model further recognizes the speech features corresponding to the speech data to be tested that cannot be outputted by the neural network based emotion recognition model, which is beneficial to improve the accuracy of the speech.

In a specific implementation, the data source ID is used by the terminal device to obtain the data source ID of the voice data to be tested, and the data source ID is used to indicate the speaker of the voice data to be tested. As shown in FIG. 3, in the emotion recognition-based voice quality inspection method, in step S30, the emotion recognition model is used to identify the voice feature, and the emotion recognition result is obtained, which specifically includes the following steps:

S321: Acquire a target emotion recognition model associated with the data source ID based on the data source ID of the voice data to be tested.

The target emotion recognition model is an emotion recognition model for training training voice data carrying the same data source ID. The target emotion recognition model may be an emotion recognition model trained according to the emotion recognition model training method mentioned in the embodiment, and the emotion recognition model has its own data source ID, that is, the target emotion recognition model may be based on a neural network. The emotion recognition model may also be a support vector machine based emotion recognition model, which is different from the emotion recognition model in steps S311-S313 in that the training voice data carries the same data source ID. It can be understood that the target emotion recognition model may be an emotion recognition model that is preliminarily trained in the training voice data carrying the same data source ID and stored in the database. In the process of emotion recognition, the terminal device queries the database according to the data source ID in the received voice data to be tested, and determines whether there is a target emotion recognition model associated with the data source ID in the database; if the target emotion recognition model exists, Step S322 is performed; if the target emotion recognition model does not exist, steps S311-S313 are performed to perform voice emotion recognition using the emotion recognition model not associated with the data source ID, that is, the emotion recognition model in steps S311-S313 can be applied to all The voice data to be tested corresponding to the speaker is identified.

S322: Identifying a voice feature by using a target emotion recognition model to obtain an emotion recognition result.

Since the target emotion recognition model is an emotion recognition model trained by the same data source ID, it is an emotion recognition model for a specific speaker, and the speech data to be tested carries the same emotion recognition model, and therefore, the target emotion recognition model is adopted. When the voice features corresponding to the voice data are identified, the emotion recognition result can be more accurate. It can be understood that the target emotion recognition model only identifies the voice data to be tested carrying the same data source ID, and is highly targeted and the recognition result is more accurate.

In a specific embodiment, since the target emotion recognition model associated with the data source ID is applied in steps S321 and S322, the voice recognition quality-based quality inspection method includes pre-training and before step S321. The step of the target emotion recognition model associated with the data source ID. As shown in FIG. 4, pre-training the target emotion recognition model associated with the data source ID specifically includes the following steps:

S331: Acquire training voice data associated with the data source ID in the database based on the data source ID.

The training voice data may be voice data collected by a recording module integrated on the terminal device or a recording device connected to the terminal device when the agent communicates with the client, and the training voice data is stored in a database connected to the terminal device, and Stored in association with the data source ID. When the target emotion recognition model associated with the data source ID needs to be trained, the database needs to be queried to obtain all the training voice data associated with the data source ID.

S332: Determine whether the number of training voice data reaches the emotional model training threshold.

The emotional model training threshold is a quantity in which the training speech data required to train the emotion recognition model is set in advance. If the number of training speech data reaches the emotional model training threshold, the database is stored with training speech data capable of training a target emotion recognition model associated with the data source ID. If the number of training speech data does not reach the emotional model training threshold, the target emotion recognition model cannot be trained.

S333: If the training speech data reaches the emotional model training threshold, the emotion recognition model training is performed based on the training speech data associated with the data source ID, and the target emotion recognition model is obtained.

In this embodiment, the training process of the emotion recognition model training based on the training voice data associated with the data source ID and the neural network based emotion recognition model mentioned in steps S311-S313 and/or the support vector machine based emotion recognition model The training process is the same. To avoid repetition, we will not repeat them here. The target emotion recognition model acquired by using the training voice data corresponding to the data source ID for the emotional model training is more suitable for the speaker's emotion corresponding to the data source ID, so that the trained target emotion recognition model carries the same data source ID. The corresponding emotion recognition of the voice data to be tested is more accurate, and the error caused by the training voice data of different speakers can be effectively avoided.

Since the client and the agent are in the process of talking, the recording module integrated on the terminal device or the recording device connected to the terminal device records the call process, and stores the acquired voice data in a database, each voice data and data source. The ID is associated with the storage, and the data source ID may be a customer ID or an agent ID. The database periodically counts the number of voices corresponding to each data source ID. When the number of voice data corresponding to any data source ID reaches the emotion model training threshold, step S333 is performed to obtain a target emotion recognition model corresponding to the data source ID. . When identifying the voice data to be tested, first searching whether a target emotion recognition model associated with the data source ID carried by the voice data to be tested is stored in the database; if the target emotion recognition model exists, performing steps S321-S322 to The accuracy of the emotion recognition is guaranteed; if there is no target emotion recognition model, steps S311-S313 are performed.

S40: Send the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result.

The associated terminal may be a terminal that performs human-computer interaction with the agent or the quality inspector, wherein the quality inspector is a person who performs quality inspection on the agent service of the institution. The terminal includes, but is not limited to, a terminal such as a smartphone, a PC, a tablet, etc., which can display the result of the emotion recognition. In this embodiment, the terminal device of the financial institution or other institution that needs to perform the emotion recognition sends the acquired emotion recognition result to the associated terminal after the emotion recognition is performed, so that the associated terminal displays the emotion recognition result, so that the associated terminal is used. The agent or the quality inspector can know the emotion of the speaker corresponding to the voice data to be tested according to the emotion recognition result.

In this embodiment, if the speaker of the voice data to be tested is a client, the terminal device sends the emotion recognition result to the agent who is in a call with the client, so that the agent adjusts the communication manner according to the customer's emotion (eg, when the customer is angry, Remind the staff to properly appease), thereby improving the quality of service and improving customer satisfaction with the organization's services. If the speaker of the voice data to be tested is an agent, the terminal device sends the emotion recognition result to the quality inspector, so that the quality inspector monitors the communication process between the agent and the client, and evaluates the working state of the agent, and implements Reward and punishment, prompting agents to better serve customers.

The method for voice quality detection based on emotion recognition provided by the present application extracts features from the voice data to be acquired to obtain voice features, and then uses the emotion recognition model to identify the voice features to obtain the emotion recognition result, and displays the same through the associated terminal. The emotion recognition result is such that the user can understand the speaker emotion corresponding to the voice data to be tested by viewing the emotion recognition result. In this embodiment, by causing the associated terminal to display the emotion recognition result corresponding to the voice data to be tested, it helps the assistant agent to improve the service quality of the customer service, thereby improving the customer's service satisfaction with the organization. In the voice quality detection method based on emotion recognition, the voice data to be tested can be intelligently recognized to obtain the emotion recognition result, and the recognition process is highly efficient, and the voice data to be tested corresponding to the speaker can be timely and comprehensively sampled. No manual intervention is required, which is conducive to saving labor costs.

In a specific implementation manner, in the voice recognition based voice quality checking method, the step S10 specifically includes: acquiring the voice data to be tested collected by the calling terminal in real time.

In this embodiment, the calling terminal may be a terminal that performs voice communication with a client or a terminal that performs voice communication with an agent. The calling terminal can be a voice call device such as a fixed telephone, a mobile phone or a walkie-talkie. The calling terminal is connected to the terminal device of the financial institution or other institution that needs to perform the emotion recognition, so that the terminal device can obtain the voice data to be tested collected by the calling terminal in real time, so as to facilitate real-time monitoring of the voice data to be tested. Specifically, the terminal device acquires the voice data to be tested collected by the calling terminal in real time, and the terminal device obtains the voice data to be tested collected in real time during the call between the client and the agent, so as to implement the emotion of the client or the agent who is calling. Monitor.

Step S40 specifically includes: transmitting the emotion recognition result to the associated terminal in real time, so that the associated terminal displays the emotion recognition result.

In this embodiment, the terminal device sends the acquired emotion recognition result to the associated terminal in real time, so that the associated terminal can display the emotion of the speaker corresponding to the voice data to be tested in real time, and prompts the agent to adjust the communication mode, thereby improving the client to the agent. Even the agency's service satisfaction. The voice recognition based voice quality detection method adopts artificial intelligence recognition mode, and the processing efficiency is high, and the process does not need to be equipped with professional quality inspection personnel for sampling inspection, which can save labor cost and reduce fraud risk.

The method for voice quality detection based on emotion recognition provided by the present application obtains the voice data to be tested collected by the calling terminal in real time, and then extracts the feature data of the voice data to obtain the voice feature, and then uses the emotion recognition model to identify the voice feature. The emotion recognition result is obtained, and the emotion recognition result is sent to the associated terminal in real time, and the emotion recognition result is displayed by the associated terminal, so that the agent or the quality inspection personnel corresponding to the associated terminal can view the emotion to be tested by viewing the emotion recognition result. The speaker's sentiment corresponding to the data and the adjustment of communication methods can help improve the service quality of the organization and thus improve the customer's service satisfaction with the organization. In the voice quality detection method based on emotion recognition, the voice data to be tested can be intelligently recognized to obtain the emotion recognition result, and the recognition process is highly efficient, and the voice data to be tested corresponding to the speaker can be timely and comprehensively sampled. No manual intervention is required, which is conducive to saving labor costs.

It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution order, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the present application.

Example 2

Corresponding to the voice recognition based voice quality inspection method in Embodiment 1, FIG. 5 is a schematic block diagram showing the emotion recognition based voice quality inspection apparatus corresponding to the emotion recognition based voice quality inspection method in Embodiment 1. As shown in FIG. 5, the emotion recognition based voice quality inspection apparatus includes a voice data acquisition module 10 to be tested, a voice data feature acquisition module 20, an emotion recognition result acquisition module 30, and an emotion recognition result feedback module 40. The steps of the voice data acquisition module 10, the voice data feature acquisition module 20, the emotion recognition result acquisition module 30, and the emotion recognition result feedback module 40 are compared with the voice quality detection method based on the emotion recognition in the embodiment. Correspondingly, in order to avoid redundancy, the present embodiment will not be described in detail.

The voice data acquiring module 10 is configured to acquire voice data to be tested.

The voice data feature acquisition module 20 is configured to perform feature extraction on the voice data to be measured, and acquire voice features.

The emotion recognition result obtaining module 30 is configured to identify the voice feature by using the emotion recognition model, and obtain the emotion recognition result.

The emotion recognition result feedback module 40 is configured to send the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result.

Preferably, the emotion recognition result acquisition module package 30 includes a recognition result output determination unit 311, a first recognition result processing unit 312, and a second recognition result processing unit 313.

The recognition result output judging unit 311 is configured to recognize the speech feature by using the neural network-based emotion recognition model, and determine whether the accurate recognition result can be output.

The first recognition result processing unit 312 is configured to use the accurate recognition result as the emotion recognition result when the accurate recognition result can be output.

The second recognition result processing unit 313 is configured to identify the voice feature by using the emotion recognition model based on the support vector machine to obtain the emotion recognition result when the accurate recognition result cannot be output.

Preferably, the emotion recognition result acquisition module 30 includes a target model acquisition unit 321 and a recognition result acquisition unit 322.

The target model obtaining unit 321 is configured to acquire a target emotion recognition model associated with the data source ID based on the data source ID of the voice data to be tested.

The recognition result obtaining unit 322 is configured to identify the voice feature by using the target emotion recognition model, and obtain the emotion recognition result.

Preferably, the emotion recognition based voice quality checking device further comprises a target model training module 50.

The target model training module 50 is configured to pre-train the target emotion recognition model associated with the data source ID.

Preferably, the target model training module 50 includes a training voice data acquiring unit 51, a number determining unit 52, and a target model training unit 53.

The training voice data acquiring unit 51 is configured to acquire training voice data associated with the data source ID in the database based on the data source ID.

The quantity determining unit 52 is configured to determine whether the number of training voice data reaches the emotional model training threshold.

The target model training unit 53 is configured to perform the emotion recognition model training based on the training voice data associated with the data source ID to obtain the target emotion recognition model when the training voice data reaches the emotion model training threshold.

Preferably, the to-be-tested voice data acquisition module 10 is configured to acquire the voice data to be tested collected by the associated terminal in real time.

The emotion recognition result feedback module 40 is configured to send the emotion recognition result to the associated terminal in real time, so that the associated terminal displays the emotion recognition result.

In the voice quality-checking device based on the emotion recognition provided by the embodiment, the voice data acquiring module 10 can obtain the voice data to be tested in real time online, and can also obtain the voice data stored in the database offline, and meet different voice data acquisition requirements. The voice data feature acquisition module 20 is configured to perform feature extraction on the voice data to be measured, and acquire a voice feature. The voice feature extraction method is a combination of the neural network method and the support vector machine recognition method. The emotion recognition result acquisition module 30 uses the emotion recognition model to identify the voice features and obtain the emotion recognition results. The emotion recognition model is established by using the neural network based emotion recognition model training and the support vector machine emotion recognition model training. Make the output more accurate and realistic. In addition, the emotion recognition result obtaining module 30 may further acquire the target emotion recognition model associated with the data source ID through the data source ID of the voice data to be tested for emotion recognition. The emotion recognition result feedback module 40 is configured to send the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result, and the associated terminal can display the test result in real time, and remind the agent to adjust the dialogue strategy with the customer according to the customer's emotion in time to ensure The call is pleasant and smooth, and the quality inspectors can also complete the random inspection of the agents.

Example 3

The embodiment provides a computer readable storage medium on which computer readable instructions are stored, and when the computer readable instructions are executed by the processor, the voice recognition based voice quality detection method in Embodiment 1 is implemented. Avoid repetition, no more details here. Alternatively, when the computer readable instructions are executed by the processor, the functions of the modules/units in the voice recognition based voice quality checking apparatus in Embodiment 2 are implemented. To avoid repetition, details are not described herein again.

Example 4

FIG. 6 is a schematic diagram of a terminal device according to an embodiment of the present application. As shown in FIG. 6, the terminal device 60 of this embodiment includes a processor 61, a memory 62, and computer readable instructions 63 stored in the memory 62 and operable on the processor 61. The processor 61 implements various steps of the emotion recognition based speech quality inspection method in Embodiment 1 when the computer readable instructions 63 are executed, such as steps S10 to S13 shown in FIG. Alternatively, when the processor 61 executes the computer readable instructions 63, the functions of the modules/units in the foregoing device embodiments are implemented, such as the voice data acquisition module 10 to be tested, the voice data feature acquisition module 20, and the emotion recognition result shown in FIG. 5. The module 30 and the emotion recognition result feedback module 40 are obtained.

Illustratively, computer readable instructions 63 may be partitioned into one or more modules/units, one or more modules/units being stored in memory 62 and executed by processor 61 to complete the application. The one or more modules/units may be an instruction segment of a series of computer readable instructions 63 capable of performing a particular function for describing the execution of computer readable instructions 63 in the terminal device 60. For example, the computer readable instructions 63 may be divided into the to-be-tested speech data acquisition module 10, the speech data feature acquisition module 20, the emotion recognition result acquisition module 30, the emotion recognition result feedback module 40, and the target model training module 50 in Embodiment 2. The function of the function is described in detail in Embodiment 2, and details are not described herein.

The terminal device 60 can be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device may include, but is not limited to, a processor 61, a memory 62. It will be understood by those skilled in the art that FIG. 6 is only an example of the terminal device 60, and does not constitute a limitation on the terminal device 60, and may include more or less components than those illustrated, or combine some components, or different components. For example, the terminal device may further include an input/output device, a network access device, a bus, and the like.

The processor 61 may be a central processing unit (CPU), or may be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.

The memory 62 may be an internal storage unit of the terminal device 60, such as a hard disk or memory of the terminal device 60. The memory 62 may also be an external storage device of the terminal device 60, such as a plug-in hard disk provided on the terminal device 60, a smart memory card (SMC), a Secure Digital (SD) card, and a flash memory card (Flash). Card) and so on. Further, the memory 62 may also include both an internal storage unit of the terminal device 60 and an external storage device. The memory 62 is used to store computer readable instructions as well as other programs and data required by the terminal device. The memory 62 can also be used to temporarily store data that has been or will be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the division of each functional unit and module described above is exemplified. In practical applications, the above functions may be assigned to different functional units as needed. The module is completed, dividing the internal structure of the device into different functional units or modules to perform all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the integrated unit may be implemented by hardware. Formal implementation can also be implemented in the form of software functional units. In addition, the specific names of the respective functional units and modules are only for the purpose of facilitating mutual differentiation, and are not intended to limit the scope of protection of the present application. For the specific working process of the unit and the module in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the above embodiments, the descriptions of the various embodiments are different, and the parts that are not detailed or described in the specific embodiments may be referred to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of this application.

In the embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be used. Combinations can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the processes in the foregoing embodiments, and may also be implemented by computer readable instructions, which may be stored in a computer readable storage medium. The computer readable instructions, when executed by a processor, may implement the steps of the various method embodiments described above. Wherein, the computer readable instructions comprise computer readable instruction code, which may be in the form of source code, an object code form, an executable file or some intermediate form or the like. The computer readable medium can include any entity or device capable of carrying the computer readable instruction code, a recording medium, a USB flash drive, a removable hard drive, a magnetic disk, an optical disk, a computer memory, a read only memory (ROM, Read-Only) Memory), random access memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in a jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, computer readable media It does not include electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still implement the foregoing embodiments. The technical solutions described in the examples are modified or equivalently replaced with some of the technical features; and the modifications or substitutions do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in Within the scope of protection of this application.

Claims (20)

A voice quality inspection method based on emotion recognition, which is characterized in that it comprises: Obtaining voice data to be tested; Performing feature extraction on the voice data to be tested to obtain a voice feature; Identifying the voice feature by using an emotion recognition model to obtain an emotion recognition result; Transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result. The method for detecting a voice quality based on emotion recognition according to claim 1, wherein the using the emotion recognition model to identify the voice feature and obtaining the emotion recognition result comprises: The speech recognition feature is identified by using a neural network based emotion recognition model to determine whether an accurate recognition result can be output; If the accurate recognition result can be output, the accurate recognition result is used as the emotion recognition result; If the accurate recognition result cannot be output, the voice feature is identified by using a support vector machine based emotion recognition model, and the emotion recognition result is obtained. The emotion recognition based speech quality inspection method according to claim 2, wherein the emotion recognition based speech quality is performed before the step of recognizing the speech feature by using a neural network based emotion recognition model The method further includes: training the neural network based emotion recognition model; The training the neural network based emotion recognition model comprises: Obtaining training voice data, and performing emotional labeling on the training voice data, so that the training voice data carries an emotional label; Feature extraction of training speech data carrying emotional tags, and acquiring training speech features carrying emotional tags; The training speech features are iteratively calculated using a logistic regression algorithm in a neural network model to obtain a neural network based emotion recognition model. The method for detecting a voice quality based on emotion recognition according to claim 1, wherein the voice data to be tested carries a data source ID; The using the emotion recognition model to identify the voice feature and obtaining the emotion recognition result includes: Obtaining a target emotion recognition model associated with the data source ID based on the data source ID of the voice data to be tested; The speech feature is identified by the target emotion recognition model, and the emotion recognition result is obtained. The emotion recognition based speech quality inspection method according to claim 4, wherein the emotion recognition based speech quality inspection method further comprises: preliminating a target emotion recognition model associated with the data source ID; The pre-training the target emotion recognition model associated with the data source ID includes: And acquiring training voice data associated with the data source ID in the database based on the data source ID; Determining whether the number of the training voice data reaches the emotional model training threshold; And if the training voice data reaches the emotion model training threshold, performing emotion recognition model training based on the training voice data associated with the data source ID, and acquiring the target emotion recognition model. The method for detecting a voice quality based on the emotion recognition according to claim 1, wherein the acquiring the voice data to be tested comprises: acquiring the voice data to be tested collected by the calling terminal in real time; Transmitting the sentiment recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result, including: transmitting the emotion recognition result to the associated terminal in real time, so that the associated terminal displays the location Describe the results of emotional recognition. A voice quality detecting device based on emotion recognition, comprising: a voice data acquiring module to be used for acquiring voice data to be tested; a voice data feature acquiring module, configured to perform feature extraction on the voice data to be tested, and acquire a voice feature; An emotion recognition result obtaining module, configured to identify the voice feature by using an emotion recognition model, and obtain an emotion recognition result; The emotion recognition result feedback module is configured to send the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result. The emotion recognition-based voice quality inspection apparatus according to claim 7, wherein the emotion recognition result acquisition module comprises: a recognition result output determining unit, configured to identify the voice feature by using a neural network based emotion recognition model, and determine whether an accurate recognition result can be output; a first recognition result processing unit, configured to use the accurate recognition result as the emotion recognition result when the accurate recognition result can be output; a second recognition result processing unit, configured to identify the voice feature by using a support vector machine based emotion recognition model when the accurate recognition result cannot be output, and obtain the emotion recognition result; Or the voice data to be tested carries a data source ID; The emotion recognition result acquisition module includes: a target model obtaining unit, configured to acquire a target emotion recognition model associated with the data source ID based on the data source ID of the voice data to be tested; And a recognition result obtaining unit, configured to identify the voice feature by using the target emotion recognition model, and obtain the emotion recognition result. The emotion recognition-based voice quality inspection apparatus according to claim 7, wherein the emotion recognition-based voice quality inspection apparatus further comprises: a target model training module, configured to pre-train associated with the data source ID Target emotion recognition model; The target model training module includes: a training voice data acquiring unit, configured to acquire, according to the data source ID, training voice data associated with the data source ID in a database; a quantity determining unit, configured to determine whether the quantity of the training voice data reaches the emotional model training threshold; a target model training unit, configured to perform emotion recognition model training based on the training voice data associated with the data source ID to obtain the target emotion recognition model when the training voice data reaches the emotion model training threshold; The voice data acquisition module to be tested is further configured to acquire the voice data to be tested collected by the associated terminal in real time; The emotion recognition result feedback module is further configured to send the emotion recognition result to the associated terminal in real time, so that the associated terminal displays the emotion recognition result. A terminal device comprising a memory, a processor, and computer readable instructions stored in the memory and operable on the processor, wherein the processor executes the computer readable instructions as follows step: Obtaining voice data to be tested; Performing feature extraction on the voice data to be tested to obtain a voice feature; Identifying the voice feature by using an emotion recognition model to obtain an emotion recognition result; Transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result. The terminal device according to claim 10, wherein the using the emotion recognition model to identify the voice feature and obtaining the emotion recognition result comprises: The speech recognition feature is identified by using a neural network based emotion recognition model to determine whether an accurate recognition result can be output; If the accurate recognition result can be output, the accurate recognition result is used as the emotion recognition result; If the accurate recognition result cannot be output, the voice feature is identified by using a support vector machine based emotion recognition model, and the emotion recognition result is obtained. The terminal device according to claim 11, wherein said processor executes said computer readable instructions before said step of identifying said voice features using said neural network based emotion recognition model The following steps: Training the neural network based emotion recognition model; The training the neural network based emotion recognition model comprises: Obtaining training voice data, and performing emotional labeling on the training voice data, so that the training voice data carries an emotional label; Feature extraction of training speech data carrying emotional tags, and acquiring training speech features carrying emotional tags; The training speech features are iteratively calculated using a logistic regression algorithm in a neural network model to obtain a neural network based emotion recognition model. The terminal device according to claim 10, wherein the voice data to be tested carries a data source ID; The using the emotion recognition model to identify the voice feature and obtaining the emotion recognition result includes: Obtaining a target emotion recognition model associated with the data source ID based on the data source ID of the voice data to be tested; The speech feature is identified by the target emotion recognition model, and the emotion recognition result is obtained. The terminal device according to claim 13, wherein said processor further implements the following steps when said computer readable instructions are executed: Pre-training a target emotion recognition model associated with the data source ID; The pre-training the target emotion recognition model associated with the data source ID includes: And acquiring training voice data associated with the data source ID in the database based on the data source ID; Determining whether the number of the training voice data reaches the emotional model training threshold; And if the training voice data reaches the emotion model training threshold, performing emotion recognition model training based on the training voice data associated with the data source ID, and acquiring the target emotion recognition model. The terminal device according to claim 10, wherein the acquiring the voice data to be tested comprises: acquiring the voice data to be tested collected by the calling terminal in real time; Transmitting the sentiment recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result, including: transmitting the emotion recognition result to the associated terminal in real time, so that the associated terminal displays the location Describe the results of emotional recognition. A computer readable storage medium storing computer readable instructions, wherein the computer readable instructions, when executed by a processor, implement the following steps: Obtaining voice data to be tested; Performing feature extraction on the voice data to be tested to obtain a voice feature; Identifying the voice feature by using an emotion recognition model to obtain an emotion recognition result; Transmitting the emotion recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result. The computer readable storage medium according to claim 16, wherein the recognizing the speech feature by using an emotion recognition model to obtain an emotion recognition result comprises: The speech recognition feature is identified by using a neural network based emotion recognition model to determine whether an accurate recognition result can be output; If the accurate recognition result can be output, the accurate recognition result is used as the emotion recognition result; If the accurate recognition result cannot be output, the voice feature is identified by using a support vector machine based emotion recognition model, and the emotion recognition result is obtained. The computer readable storage medium according to claim 16, wherein the voice data to be tested carries a data source ID; The using the emotion recognition model to identify the voice feature and obtaining the emotion recognition result includes: Obtaining a target emotion recognition model associated with the data source ID based on the data source ID of the voice data to be tested; The speech feature is identified by the target emotion recognition model, and the emotion recognition result is obtained. The computer readable storage medium of claim 18, wherein the computer readable instructions when executed by the processor further implement the following steps: Pre-training a target emotion recognition model associated with the data source ID; The pre-training the target emotion recognition model associated with the data source ID includes: And acquiring training voice data associated with the data source ID in the database based on the data source ID; Determining whether the number of the training voice data reaches the emotional model training threshold; And if the training voice data reaches the emotion model training threshold, performing emotion recognition model training based on the training voice data associated with the data source ID, and acquiring the target emotion recognition model. The computer readable storage medium according to claim 16, wherein the obtaining the voice data to be tested comprises: acquiring the voice data to be tested collected by the calling terminal in real time; Transmitting the sentiment recognition result to the associated terminal, so that the associated terminal displays the emotion recognition result, including: transmitting the emotion recognition result to the associated terminal in real time, so that the associated terminal displays the location Describe the results of emotional recognition.

Images