CN118197303A - Intelligent speech recognition and sentiment analysis system and method - Google Patents

Abstract

The invention discloses an intelligent voice recognition and emotion analysis system and method, which relate to the technical field of voice recognition analysis, wherein an analysis system records voice of a user, acquires environment data and equipment data in the voice input process of the user, judges whether emotion analysis is required to be performed on the voice data input by the user after the environment data and the equipment data are analyzed through an optimization judgment model after voice interaction is completed, and when emotion analysis is required to be performed, identifies emotion types of the user based on emotion analysis technology, acquires feedback data of a voice recognition assistant on the user, and optimizes the voice recognition assistant based on identification results and feedback data of the emotion types. The analysis method records the voice of the user in the process of recognizing the voice of the user by the voice recognition assistant, judges whether emotion analysis is needed after voice interaction is completed, effectively reduces the data processing burden of an analysis system, and improves the voice recognition efficiency of the voice recognition assistant.

Description

Intelligent speech recognition and sentiment analysis system and method

Technical Field

The present invention relates to the technical field of speech recognition and analysis, and in particular to an intelligent speech recognition and sentiment analysis system and method.

Background technique

Intelligent speech recognition (ASR) is an important part of the field of speech processing. After years of research and development and technological progress, the system can accurately convert spoken language into text. The rise of deep learning technology, especially end-to-end model design, has significantly improved the performance of ASR. ASR is widely used in voice assistants, voice search, voice recognition notes and other fields. It plays a key role in improving user experience and enhancing the human-computer interaction capabilities of devices. Sentiment analysis, also known as sentiment computing and opinion mining, is a method based on natural language processing and machine learning technology to identify and understand the sentiment tendency in text. The development of sentiment computing benefits from the progress in natural language processing, text mining and machine learning.

Existing analysis systems usually perform sentiment analysis when the speech recognition assistant recognizes the user's speech, and then optimize the speech recognition assistant based on the results of the sentiment analysis after the current interaction with the user is completed. This processing method has the following defects:

If sentiment analysis is performed every time a user inputs speech into the speech recognition assistant, the data processing burden of the analysis system will increase, causing the analysis system to have to both recognize the speech and perform sentiment analysis, thereby reducing the speech recognition efficiency of the speech recognition assistant.

Summary of the invention

The purpose of the present invention is to provide an intelligent speech recognition and sentiment analysis system and method to address the deficiencies in the background technology.

In order to achieve the above object, the present invention provides the following technical solution: an intelligent speech recognition and sentiment analysis method, the analysis method comprising the following steps:

The user wakes up the voice recognition assistant through a physical button or voice keyword, and the user inputs voice into the voice recognition assistant. After the voice recognition assistant recognizes the voice, it provides corresponding feedback based on the user's voice recognition content;

During the user's voice input process, the analysis system records the user's voice and obtains the environmental data and device data during the user's voice input process. After the voice interaction is completed, the environmental data and device data are analyzed by optimizing the judgment model to determine whether sentiment analysis of the user's voice data is required.

When it is determined that sentiment analysis is needed, the analysis system obtains the user's recorded data, extracts feature data from the recorded data, identifies the user's sentiment category after analyzing the feature data based on sentiment analysis technology, and obtains the voice recognition assistant's feedback data on the user. After analyzing the feedback data, the voice recognition assistant is optimized based on the recognition results of the sentiment category and the feedback data analysis results.

In a preferred embodiment, after the voice interaction is completed, the environmental data and the device data are analyzed by optimizing the judgment model, the environmental data includes the background noise index, and the device data includes the sampling rate and the voice correct recognition index.

In a preferred embodiment, after the voice interaction is completed, after analyzing the environment data and the device data by optimizing the judgment model, determining whether it is necessary to perform sentiment analysis on the voice data input by the user includes the following steps:

After the voice interaction is completed, the background noise index, sampling rate, and voice recognition accuracy index are obtained and substituted into the optimization judgment model for analysis, and the optimization coefficient is output;

If the optimization coefficient is greater than or equal to the optimization threshold, it is determined that sentiment analysis needs to be performed on the voice data input by the user;

If the optimization coefficient is less than the optimization threshold, it is determined that there is no need to perform sentiment analysis on the voice data input by the user.

In a preferred embodiment, the establishment of the optimization judgment model includes the following steps:

After normalizing the background noise index, sampling rate, and speech recognition accuracy index, the optimization coefficient is obtained through comprehensive calculation;

After obtaining the optimization coefficient yh _x , the optimization coefficient yh _x is compared with the preset optimization threshold to complete the establishment of the optimization judgment model. The optimization threshold is used to analyze the impact of the environment and equipment on speech emotion analysis.

In a preferred embodiment, the logic for obtaining the background noise index is: during the process of the user inputting voice, if other voices appear and the period when the decibel of other voices exceeds the first decibel threshold is the period of human voice decibel warning;

During the process of the user inputting voice, if non-human voice noise occurs and the decibel of the non-human voice noise exceeds the second decibel threshold, it is the period of non-human voice decibel warning;

The background noise index is obtained by integrating the period of human voice decibel warning and the period of non-human voice decibel warning. The calculation expression is: Where Z(t) is the speech recognition error rate, [ _tx , _ty ] is the period of human voice decibel warning, and [ _ti , _tj ] is the period of non-human voice decibel warning.

In a preferred embodiment, the calculation logic of the speech correct recognition index is: obtain the number of speech segments input by the user during the entire interaction process, and calculate the correct recognition rate of each speech segment, the expression is: In the formula, F represents the correct recognition rate, ZQS represents the number of correctly recognized words in speech, and ZSL represents the number of words in all speech during the interaction process;

The correct recognition rate of the speech segment with the maximum correct recognition rate in the whole interaction process is taken as the speech correct recognition index, and the expression is: YZL = max(F ₁ , F ₂ , ..., F _n ), where n represents the number of speech segments in the whole interaction process, and n is an integer, and max() represents the maximum correct recognition rate.

In a preferred embodiment, obtaining the feedback data of the speech recognition assistant to the user and analyzing the feedback data comprises the following steps:

Obtain the feedback data of the voice recognition assistant on users, including recognition response speed, freeze frequency, and voice command coverage;

The recognition response speed, freeze frequency and voice command coverage are calculated comprehensively to obtain the feedback coefficient;

If the feedback coefficient is ≥ the preset feedback threshold, the speech recognition performance of the speech recognition assistant is evaluated to be good;

If the feedback coefficient is less than the preset feedback threshold, the speech recognition performance of the speech recognition assistant is evaluated to be poor.

The present invention also provides an intelligent speech recognition and emotion analysis system, including a wake-up module, a speech recognition module, a recording module, a data acquisition module, a judgment module, a feature extraction module, an emotion analysis module, a recognition analysis module, and an optimization module;

Wake-up module: The user wakes up the speech recognition module through the wake-up module;

Speech recognition module: The user inputs speech into the speech recognition module, and after the speech recognition module recognizes the speech, it provides corresponding feedback based on the user's speech recognition content;

Recording module: Record the user's voice during the user input process;

Data collection module: obtains environmental data and device data during user voice input;

Judgment module: After the voice interaction is completed, the environment data and device data are analyzed by optimizing the judgment model to determine whether sentiment analysis is needed on the voice data input by the user;

Feature extraction module: when it is determined that sentiment analysis is required, the user's recording data is obtained and feature data is extracted from the recording data;

Sentiment analysis module: identifies the user's sentiment category after analyzing feature data based on sentiment analysis technology;

Recognition and analysis module: obtains the feedback data of the speech recognition module to the user and analyzes the feedback data;

Optimization module: Optimize the speech recognition module based on the recognition results of emotion categories and feedback data analysis results.

In the above technical solution, the technical effects and advantages provided by the present invention are:

1. In the process of user inputting voice, the analysis system of the present invention records the user's voice and obtains the environmental data and device data during the user's voice input process. After the voice interaction is completed, after analyzing the environmental data and device data through the optimization judgment model, it is determined whether it is necessary to perform sentiment analysis on the voice data input by the user. When it is determined that sentiment analysis is necessary, the sentiment category of the user is identified based on the sentiment analysis technology, and the feedback data of the voice recognition assistant to the user is obtained. The voice recognition assistant is optimized based on the recognition result of the sentiment category and the feedback data. This analysis method records the voice during the process of the voice recognition assistant recognizing the user's voice, and determines whether sentiment analysis is needed after the voice interaction is completed, which effectively reduces the data processing burden of the analysis system and improves the voice recognition efficiency of the voice recognition assistant;

2. The present invention obtains environmental data and device data during the user's voice input process. After the voice interaction is completed, the present invention analyzes the environmental data and device data through an optimized judgment model to determine whether it is necessary to perform sentiment analysis on the voice data input by the user. When the environment and the device have too much impact on the user's input voice, continuing the sentiment analysis will increase the workload of the analysis system and may cause a decrease in analysis accuracy, affecting the analysis results. After analyzing the impact of the environment and the device on the recording, we determine whether to carry out sentiment analysis based on the actual situation, so that we can make appropriate decisions when there is no need to perform sentiment analysis on the recording.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

FIG. 1 is a flow chart of the method of the present invention.

FIG. 2 is a system module diagram of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Embodiment 1: Please refer to FIG. 1 , an intelligent speech recognition and sentiment analysis method described in this embodiment, the analysis method comprises the following steps:

The user wakes up the voice recognition assistant by pressing a physical button or using a voice keyword, and the user inputs voice to the voice recognition assistant, including the following steps:

Physical button wake-up: The user activates the voice recognition assistant by pressing a specific physical button on the device, such as the voice assistant button on a mobile phone;

Voice keyword wake-up: When the device is in standby mode, the system continuously monitors the ambient sound and activates the voice recognition assistant once it detects that the user speaks a predefined voice keyword;

Voice input: After waking up, the user can start speaking voice commands, questions or requests. Voice input may include conversations, queries, commands and other forms.

Voice signal collection: The device uses a built-in or external microphone to collect the user's voice signal. Voice signal collection may involve pre-processing steps such as noise reduction and voice endpoint detection to improve voice quality and accuracy.

User feedback and interaction: After the system provides a response to the user, the user can continue to interact with the voice assistant, asking further questions or instructions. The feedback mechanism can be used to improve the performance of the system and ensure user satisfaction.

After the speech recognition assistant recognizes the speech, it provides corresponding feedback based on the user's speech recognition content. The corresponding feedback includes text output, speech output or corresponding actions, including the following steps:

Speech recognition result generation: Convert user speech into text to form speech recognition results. Use intelligent speech recognition technology, such as deep learning models, to convert speech signals into corresponding text.

Intent recognition and command parsing: Analyze speech recognition results, determine the user's intent and request, and use natural language processing techniques, such as named entity recognition and keyword extraction, to understand the user's intent and parse the specific command or request;

Generate system response: Generate text or voice response of the system based on the user's intent, using dialogue generation models, predefined response templates or other natural language processing techniques to form the system's response;

Text output or speech synthesis: The generated text response is displayed to the user, for example, on a screen. If the user wishes to receive a spoken response, the system uses speech synthesis technology to convert the text into speech, using a text-to-speech (TTS) engine to generate natural speech based on the generated text;

Voice output: When the user expects to receive a response in an auditory manner, the system provides voice output to the user through a speaker or headphones, using the audio output device on the device to play the generated voice in a way that is audible to the human ear;

Perform corresponding actions: Based on the user's request, the system may need to perform specific operations or actions and call related services or functions, such as controlling smart home devices, sending messages, searching, etc.

User feedback and interaction: Provide opportunities for users to provide feedback, ask more questions, or issue new commands, and design conversational interactions to maintain real-time communication with users;

Record user context: The system records and maintains the user's conversation context to better understand the user's needs and provide coherent conversations. Context management techniques are used to ensure that the system can remember previous conversation history.

During the user's voice input process, the analysis system records the user's voice and obtains the environmental data and device data during the user's voice input process, including the following steps:

Voice signal collection: Use the microphone and other hardware devices on the device to collect the user's voice signal, and use audio collection technology to convert the sound waveform into an electrical signal through the microphone;

Speech signal preprocessing: preprocess the collected speech signals to improve the accuracy of speech recognition. The preprocessing steps may include noise reduction, speech endpoint detection, echo removal, etc.

Recording start mark: mark the start time of voice input, record timestamp or other marks, so as to determine the time range of voice input in subsequent analysis;

Environmental data collection: Obtain data related to the voice input environment, such as ambient noise level, temperature, humidity, etc., and use sensors or devices to record relevant information about the surrounding environment;

Device data collection: Obtain relevant information about the voice input device, such as device model, battery status, network connection status, etc., and use the device's sensors and system interfaces to obtain the device's status and attribute information;

Recording end mark: mark the end time of voice input, record timestamp or other marks, so as to determine the time range of voice input in subsequent analysis;

Voice signal storage: The collected voice signals are stored for subsequent analysis and processing, and the voice signals are saved in the system in the form of audio files, usually in .wav or .mp3 formats;

Data transmission and processing: The collected voice signals, environmental data and equipment data are transmitted to the corresponding analysis system, and the data is sent to the background system for further processing via network connection or local data transmission.

After the voice interaction is completed, after analyzing the environment data and device data through the optimization judgment model, it is determined whether it is necessary to perform sentiment analysis on the voice data input by the user. When it is determined that sentiment analysis is required, the analysis system obtains the user's recording data and extracts feature data from the recording data. The feature data includes sentiment vocabulary, intonation, speech speed, etc., including the following steps:

Voice data preprocessing: preprocess the recorded data to reduce noise and improve the accuracy of analysis, including noise reduction, voice endpoint detection, echo removal and other processing steps;

Speech signal feature extraction: extract useful features from speech data for subsequent sentiment analysis;

Spectral feature extraction: including Mel-frequency cepstral coefficients (MFCC), sound energy, etc.;

Features based on time domain: such as short-time energy, short-time zero-crossing rate, etc.

Features based on frequency domain: such as spectrum average, spectrum bandwidth, etc.;

Emotional vocabulary extraction: Extract words containing emotional information from recording data, use the emotional vocabulary library, and extract keywords related to emotions through text analysis or speech recognition results;

Intonation analysis: Analyze the intonation patterns in speech to understand the speaker's emotional state, and use techniques such as fundamental frequency extraction and intonation curve analysis to identify the pitch fluctuations of speech;

Speech rate analysis: Analyze the speech rate, that is, the speed at which the speaker speaks, and estimate the speech rate by calculating the duration of syllables in the speech, the change in speech rate, and other information;

Feature normalization: Normalize the extracted features to ensure that they are on the same scale, which is beneficial for subsequent model training and analysis. Standardization or other normalization methods can be used to make the value ranges of different features similar;

Feature storage and analysis: Store the extracted features and perform subsequent sentiment analysis and speech recognition. Use a database or file system to store feature data and use the corresponding algorithm for sentiment analysis.

After analyzing the feature data based on sentiment analysis technology, the user's emotion category is identified. The emotion categories include happiness, sadness, anger, etc., including the following steps:

Obtain the user's recording data, extract the feature data in the recording data, including tone, speaking speed, energy, and frequency, and use the trained KNN model to perform sentiment classification prediction on the feature data. For each test sample, find its nearest K neighbors and vote based on their sentiment categories, and select the category with the most votes as the prediction result;

The establishment of the KNN model includes the following steps:

Collect a dataset containing user voice samples and corresponding emotion category labels, annotate the voice samples, mark emotion categories such as happiness, sadness, and anger, perform feature extraction on the voice data, convert the voice into numerical features that can be used by the machine learning model, extract acoustic features related to emotions, and possible text features such as emotion vocabulary, preprocess the feature data to ensure data consistency and availability, including steps such as data cleaning, normalization, and standardization to reduce the difference between features and ensure that the distribution of training and test data sets is consistent, divide the data set into training and test sets for use in training and evaluating the model, select the best performing K neighbors through cross-validation, use the training set to train the KNN model, provide the feature data of the training set and the corresponding emotion category labels to the KNN algorithm, train the model to establish the relationship between features and emotion categories, and complete the establishment of the KNN model;

Selecting the best performing number of K neighbors through cross validation involves the following steps:

Select a series of K values, usually starting with a smaller value and gradually increasing. For example, you can choose K = 1, 3, 5, 7, 9, etc. For each K value, perform a cross-validation cycle. For each cycle: divide the training set into a sub-training set and a validation set, use the sub-training set to train the KNN model, and use the validation set to evaluate the model performance. For each K value, calculate the performance indicators on the validation set, such as accuracy, precision, recall, etc. You can choose one or more indicators to evaluate the model performance. Based on the results of the performance indicators, select the K value that performs best on the validation set. This may be the K value with the highest accuracy or other excellent performance. Use the entire training set (including the validation set) and the selected optimal K value to retrain the KNN model.

For example, the voice data collected in this application is shown in Table 1:

Voice Sample Sound Energy tone Emotional categories Sample1 10 3 Happy Sample2 8 2 Sad Sample3 5 1 Angry Sample4 12 4 Happy Sample5 6 2 Sad

Table 1

We select sound energy and tone as features and divide the dataset into training and test sets. Here we use the holdout method, with 80% of the data as the training set and 20% of the data as the test set. Assuming we choose K=3, we use the feature data of the training set and the corresponding emotion category labels for model training. For each sample, the KNN algorithm will find the three nearest neighbors and use the trained KNN model to perform emotion classification prediction on new samples in the test set. For each test sample, find its three nearest neighbors.

Obtain feedback data from the voice recognition assistant on users, analyze the feedback data, and optimize the voice recognition assistant based on the recognition results of the emotion categories and the feedback data analysis results.

In the process of user voice input, the analysis system of this application records the user's voice and obtains the environmental data and device data during the user's voice input process. After the voice interaction is completed, after analyzing the environmental data and device data through the optimization judgment model, it is determined whether it is necessary to perform sentiment analysis on the voice data input by the user. When it is determined that sentiment analysis is necessary, the user's sentiment category is identified based on sentiment analysis technology, and the feedback data of the voice recognition assistant to the user is obtained. The voice recognition assistant is optimized based on the recognition results of the sentiment category and the feedback data. This analysis method records the voice during the process of the voice recognition assistant recognizing the user's voice, and determines whether sentiment analysis is needed after the voice interaction is completed, which effectively reduces the data processing burden of the analysis system and improves the voice recognition efficiency of the voice recognition assistant;

This application obtains environmental data and device data during the user's voice input process. After the voice interaction is completed, it analyzes the environmental data and device data through an optimized judgment model to determine whether it is necessary to perform sentiment analysis on the voice data input by the user. When the environment and equipment have too much impact on the user's input voice, continuing the sentiment analysis will increase the workload of the analysis system and may cause a decrease in analysis accuracy and affect the analysis results. After analyzing the impact of the environment and equipment on the recording, we determine whether to conduct sentiment analysis based on the actual situation, so that we can make appropriate decisions when there is no need to perform sentiment analysis on the recording.

Embodiment 2: After the voice interaction is completed, after analyzing the environment data and the device data by optimizing the judgment model, it is determined whether it is necessary to perform sentiment analysis on the voice data input by the user, including the following steps:

After the voice interaction is completed, the environmental data and device data are analyzed by optimizing the judgment model. The environmental data includes the background noise index, and the device data includes the sampling rate and the voice recognition accuracy index.

After the voice interaction is completed, the background noise index BZS, sampling rate CYL and voice correct recognition index YZL are obtained, and substituted into the optimization judgment model for analysis, and the optimization coefficient is output;

If the optimization coefficient is greater than or equal to the optimization threshold, it is determined that sentiment analysis needs to be performed on the voice data input by the user;

If the optimization coefficient is less than the optimization threshold, it is determined that there is no need to perform sentiment analysis on the voice data input by the user;

The logic for obtaining the background noise index is as follows: during the user inputting voice, if other voices other than the user appear, the error rate of the analysis system in identifying the user's voice emotion will increase. The greater the decibel of other voices, the greater the error rate of the analysis system in identifying the user's voice emotion. Therefore, during the user inputting voice, if other voices appear and the decibel of other voices exceeds the first decibel threshold, it is the period of human voice decibel warning;

If non-human voice noise appears during the user's voice input process, it will not only reduce the accuracy of the analysis system in identifying the user's voice emotion, but also may damage the recording equipment. The greater the non-human voice noise, the greater the error rate of the analysis system in identifying the user's voice emotion. Therefore, during the user's voice input process, if non-human voice noise appears and the decibel of the non-human voice noise exceeds the second decibel threshold, it is the period of non-human voice decibel warning;

The background noise index is obtained by integrating the period of human voice decibel warning and the period of non-human voice decibel warning. The calculation expression is: Where Z(t) is the speech recognition error rate, [t _x , _ty ] is the period of human voice decibel warning, and [t _i , t _j ] is the period of non-human voice decibel warning;

During the user input process, if other voices other than the user appear, the error rate of the analysis system in recognizing the user's voice emotion will increase. Specifically:

Confusion and overlap: The voices of other people may be mixed with the user's voice, making it difficult for the sentiment analysis system to accurately distinguish and identify the user's emotions. Due to confusion and overlap, the system may mistakenly include the emotional influence of others in the analysis, resulting in an increase in the error rate;

Differences in emotional expression: Different individuals may express the same emotion in different ways, and other people’s voices may introduce additional differences in emotional expression. The sentiment analysis system needs to be able to distinguish the emotional expressions of different speakers, otherwise it may mislead the user’s sentiment analysis;

Emotional changes: Users’ emotions may change during voice input, and other people’s voices may also have different emotional tones. The system needs to track and understand possible emotional changes in voice, and other people’s emotions may introduce noise, making analysis more difficult.

Voice command confusion: If other people make similar voice commands when the user expresses emotions, the system may have difficulty determining which sound is the user's emotional expression, which may cause the emotion analysis system to mistakenly identify other people's emotional expressions or commands as the user's, thereby increasing the error rate;

Introduction of background noise: The voices of other people may be introduced as background noise, reducing the quality of the voice signal. Background noise may make it difficult for the system to accurately extract and analyze the emotional features in the user's voice, thereby increasing the error rate.

If non-human voice noise is present, it will not only reduce the accuracy of the analysis system in identifying the user's voice emotion, but may also damage the recording equipment. Specifically:

Impact on the accuracy of sentiment analysis: Non-human noise, such as mechanical noise and electronic noise, may be mixed with the user's voice, making it difficult for the sentiment analysis system to accurately extract and analyze the emotional features in the user's voice. Non-human noise introduces additional signals, which may interfere with the extraction of emotional features, thereby reducing the accuracy of the system's understanding of the user's emotions.

Damage to recording equipment: Strong non-human voice noise, especially high-intensity noise, may have an adverse effect on recording equipment. High-intensity non-human voice noise may damage the microphone or other components of the recording equipment, reduce the life of the equipment, or even cause damage to the equipment;

Decreased signal-to-noise ratio: Non-human voice noise introduces additional noise components, reducing the signal-to-noise ratio of the speech signal. The reduced signal-to-noise ratio makes it more difficult for the speech emotion analysis system to clearly identify and analyze the user's voice, resulting in difficulty in analysis;

Speech signal distortion: Non-human noise may cause distortion of speech signals, making it impossible for the original speech information to be clearly transmitted to the analysis system. Distorted speech signals will reduce the accuracy of the sentiment analysis system in expressing the user's emotions, because key emotional features may have been affected by the distortion.

The sampling rate is obtained as follows:

Operating system settings: In Windows operating system, you can open the "Sound" settings, select the recording device, and view its properties. In the "Advanced" or "Advanced Properties" tab, you should be able to find the default sampling rate of the device. In macOS, you can open "Audio MIDI Setup", select "Audio Devices", and view the device configuration information, including the sampling rate. In Linux operating system, you can use command line tools such as arecord or aplay and obtain the device's sampling rate information through parameters;

Device documentation: Consult the device's user manual or technical specification sheet. Usually, the device's specification sheet provides detailed information about the sampling rate.

Audio device management tools: In some cases, device manufacturers provide dedicated audio device management tools that allow you to view and configure detailed settings for your device, including sampling rates.

Application settings: In some applications, you can find configuration information for your audio device, including the sample rate, directly in the settings or preferences.

Since the logic of obtaining the background noise index is to analyze the impact of human voice or non-human voice decibels on the error rate of sentiment analysis, in actual situations, when there are abnormalities in the device itself, the accuracy of speech recognition will decrease;

However, in actual applications, even if existing speech recognition assistants fail to correctly recognize certain words or phrases in a speech input by the user, they will automatically fill in the missing words or phrases by understanding the context.

However, some incorrectly recognized characters or words may cause the analysis system to be unable to accurately analyze the user's voice emotion, increasing the analysis error. Therefore, the present application also analyzes the recognition accuracy of the user's input voice;

The calculation logic of the speech recognition accuracy index is: obtain the number of speech segments input by the user during the entire interaction process, and calculate the correct recognition rate of each speech segment. The expression is: In the formula, F represents the correct recognition rate, ZQS represents the number of correctly recognized words in speech, and ZSL represents the number of words in all speech during the interaction process;

The correct recognition rate of the speech segment with the maximum correct recognition rate in the whole interaction process is taken as the speech correct recognition index, and the expression is: YZL = max(F ₁ , F ₂ , ..., F _n ), where n represents the number of speech segments in the whole interaction process, and n is an integer, and max() represents the maximum correct recognition rate.

The establishment of the optimization judgment model includes the following steps:

After the background noise index BZS, sampling rate CYL and speech recognition accuracy index YZL are standardized, the optimization coefficient yh _x is obtained by comprehensive calculation, and the expression is:

Where BZS is the background noise index, CYL is the sampling rate, YZL is the speech recognition index, α, β, and γ are the proportional coefficients of the sampling rate, speech recognition index, and background noise index, respectively, and α, β, and γ are all greater than 0;

After obtaining the optimization coefficient yh _x , the optimization coefficient yh _x is compared with the preset optimization threshold to complete the establishment of the optimization judgment model. The optimization threshold is used to analyze the impact of the environment and equipment on speech emotion analysis.

Obtaining the feedback data of the speech recognition assistant to the user, and analyzing the feedback data includes the following steps:

Obtain user feedback from the voice recognition assistant, including recognition response speed, freeze frequency, and voice command coverage;

The recognition response speed, freeze frequency and voice command coverage are calculated comprehensively to obtain the feedback coefficient fk _x , and the calculation expression is: Where, XYD, FGL, KDL are recognition response speed, voice command coverage, and jamming frequency, respectively; a ₁ , a ₂ , and a ₃ are proportional coefficients of recognition response speed, voice command coverage, and jamming frequency, respectively; and a ₁ , a ₂ , and a ₃ are all greater than 0;

If the feedback coefficient fk _x ≥ the preset feedback threshold, the speech recognition performance of the speech recognition assistant is evaluated to be good;

If the feedback coefficient fk _x < the preset feedback threshold, the speech recognition performance of the speech recognition assistant is evaluated to be poor;

The method to obtain the recognition response speed is as follows: In the code of the voice recognition assistant, a timer or timestamp can be used to record the time when the user's voice input starts and the time when the system generates a response. By calculating the difference between these two time points, the response time of the recognition assistant can be obtained.

The method for obtaining the frequency of jams is to add detailed log records in the speech recognition assistant, including information such as the timestamp, processing time, and response time of the speech recognition request, and analyze these logs to identify whether the system has jams when responding to voice input, as well as the frequency and duration of the jams.

The voice command coverage is obtained as follows: If the system has been used in actual applications, user logs can be analyzed to understand the voice commands actually used by users. This can be achieved by anonymously collecting and analyzing user voice input data, extracting voice commands from user logs, and then analyzing the success rate of each command to evaluate the system's coverage in actual use.

The speech recognition assistant is optimized based on the recognition results of the emotion categories and the feedback data analysis results, including the following steps:

Associate emotions with command responses: Associate emotion categories with corresponding command responses. For example, when a user expresses anger, the system can take a calmer response or provide additional support to solve the user's problem.

Personalized emotion model: Consider developing a personalized emotion model that can adapt to the emotional expressions of different users. Through the personalized model, the system can improve the accuracy of understanding and recognition of user emotions.

Feedback mechanism: Introduce emotional feedback mechanism in user interaction so that the system can adjust the interaction strategy according to the user's emotional state. For example, when the system detects that the user is happy, it can provide a more friendly and relaxed response.

User surveys and feedback: Conduct user surveys to understand user expectations and feedback on emotion recognition in speech recognition assistants. Collect user opinions on the accuracy of emotion recognition in the system and make adjustments accordingly based on the feedback.

Optimize emotion labels and classification: Regularly review and optimize emotion category labels and classification systems. Ensure that the definition of emotion categories is consistent with user expectations and actual context to improve the accuracy of emotion recognition.

Processing multimodal input: Consider processing multimodal input, such as combining voice and image information for sentiment analysis. This can enhance the understanding of user emotions, especially useful in video calls or voice interactions.

Privacy protection: When optimizing emotion recognition functions, pay attention to protecting user privacy. Clearly inform users of the purpose and use of emotion recognition to ensure compliance with privacy regulations.

Voice data quality optimization: Voice recognition assistants may be affected by voice input quality, such as noise, echo, etc. Use noise suppression technology, echo cancellation algorithms, or provide clear voice input guidance before user input to improve the quality of voice data.

Multimodal input support: Relying solely on voice input may limit the performance of the system, especially in complex interaction scenarios. Support multimodal input, such as combining voice and text input, to improve the system's understanding and accuracy of user intent.

Expanded voice command coverage: Voice recognition assistants may not cover a wide range of user voice commands. Expand the coverage of voice commands, including common commands and commands in specific fields, to meet the needs of more users.

Real-time performance monitoring: Speech recognition assistants may have problems with real-time performance, causing freezes or delays. Introduce a real-time performance monitoring mechanism to track the performance of the system and identify potential problems in a timely manner, so that the system can process voice input more smoothly.

Improved language and accent adaptability: Speech recognition assistants may not perform well when dealing with multiple languages and accents. Improve the language model and introduce accent adaptability technology to improve the system's recognition accuracy in different contexts.

Adjust the speech recognition engine parameters: The default parameters of the speech recognition engine may not be suitable for specific application scenarios. Adjust the parameters of the speech recognition engine, such as the parameters of audio feature extraction and model depth, to optimize the recognition performance.

User feedback mechanism: When speech recognition errors occur, lack of user feedback may result in problems not being discovered in time. Introduce a user feedback mechanism to allow users to report speech recognition errors and collect this feedback to guide system optimization.

Model iteration and update: The speech recognition model used may be outdated or not suitable for the current user's language habits. The speech recognition model should be updated regularly, and the latest speech data and technology should be used to iterate and optimize the model.

Difficulty level adaptation: Speech recognition systems may not be able to handle complex or difficult to understand speech input. Introducing a difficulty level adaptation mechanism enables the system to dynamically adjust the level of processing of speech input to accommodate users with different difficulty levels.

Privacy and security: Inadequate privacy and security protections may reduce user trust in voice recognition assistants. Take measures to ensure the secure storage and processing of voice data, and clearly inform users about privacy policies and data usage.

Embodiment 3: Please refer to FIG. 2 , an intelligent speech recognition and sentiment analysis system described in this embodiment includes a wake-up module, a speech recognition module, a recording module, a data acquisition module, a judgment module, a feature extraction module, a sentiment analysis module, a recognition analysis module, and an optimization module;

Wake-up module: The user wakes up the speech recognition module through the wake-up module;

Speech recognition module: The user inputs speech into the speech recognition module. After the speech recognition module recognizes the speech, it provides corresponding feedback based on the user's speech recognition content. The corresponding feedback includes text output, voice output or corresponding actions. The feedback data is sent to the recognition analysis module;

Recording module: When the user inputs voice, the user's voice is recorded and processed, and the recording data is sent to the feature extraction module;

Data collection module: obtains environmental data and device data during the user's voice input process, and sends the environmental data and device data to the judgment module;

Judgment module: After the voice interaction is completed, the environment data and device data are analyzed by optimizing the judgment model to determine whether it is necessary to perform sentiment analysis on the voice data input by the user, and the judgment result is sent to the feature extraction module;

Feature extraction module: when it is determined that sentiment analysis is needed, the user's recording data is obtained, and feature data is extracted from the recording data. The feature data includes sentiment vocabulary, intonation, speech speed, etc. The feature data is sent to the sentiment analysis module;

Sentiment analysis module: After analyzing feature data based on sentiment analysis technology, identify the user's emotion category, which includes happiness, sadness, anger, etc., or perform sentiment polarity analysis, such as positive, negative, neutral, etc. The identification results of the emotion category are sent to the optimization module;

Recognition and analysis module: obtains the feedback data of the speech recognition module on the user, analyzes the feedback data, and sends the feedback data analysis results to the optimization module;

Optimization module: Optimize the speech recognition module based on the recognition results of emotion categories and feedback data analysis results.

The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims (8)

1. An intelligent voice recognition and emotion analysis method is characterized in that: the analysis method comprises the following steps:

the user wakes up the voice recognition assistant through the entity key or the voice keyword, the user inputs voice to the voice recognition assistant, and after the voice recognition assistant recognizes the voice, corresponding feedback is carried out based on the voice recognition content of the user;

In the voice input process of a user, an analysis system carries out recording processing on the voice of the user, acquires environment data and equipment data in the voice input process of the user, analyzes the environment data and the equipment data through an optimization judgment model after voice interaction is completed, and judges whether emotion analysis is needed to be carried out on the voice data input by the user;

When judging that emotion analysis is needed, the analysis system acquires recording data of the user, extracts feature data from the recording data, analyzes the feature data based on emotion analysis technology, then identifies emotion types of the user, acquires feedback data of the voice recognition assistant to the user, and optimizes the voice recognition assistant based on identification results of the emotion types and analysis results of the feedback data after analyzing the feedback data.

2. The intelligent speech recognition and emotion analysis method of claim 1, wherein: after the voice interaction is completed, the environment data and the equipment data are analyzed through the optimization judgment model, wherein the environment data comprise background noise indexes, and the equipment data comprise sampling rates and voice correct recognition indexes.

3. The intelligent speech recognition and emotion analysis method of claim 2, wherein: after the voice interaction is completed, judging whether emotion analysis is needed to be carried out on voice data input by a user after analyzing the environment data and the equipment data through the optimization judgment model comprises the following steps:

After the voice interaction is completed, obtaining a background noise index, a sampling rate and a voice correct recognition index, substituting the background noise index, the sampling rate and the voice correct recognition index into an optimization judgment model for analysis, and outputting an optimization coefficient;

If the optimization coefficient is more than or equal to the optimization threshold, judging that emotion analysis is required to be carried out on voice data input by a user;

If the optimization coefficient is smaller than the optimization threshold value, judging that emotion analysis is not needed to be carried out on voice data input by a user.

4. A method of intelligent speech recognition and emotion analysis according to claim 3, characterized in that: the establishment of the optimization judgment model comprises the following steps:

after the background noise index, the sampling rate and the voice correct recognition index are standardized, comprehensively calculating and obtaining an optimization coefficient;

After the optimization coefficient yh _x is obtained, the optimization coefficient yh _x is compared with a preset optimization threshold value, so that the establishment of an optimization judgment model is completed, and the optimization threshold value is used for analyzing the influence of the environment and equipment on the voice emotion analysis.

5. The intelligent speech recognition and emotion analysis method of claim 4, wherein: the acquisition logic of the background noise index is as follows: in the process of inputting voice by a user, if other voice appears, and the time period when the decibel of the other voice exceeds the first decibel threshold value is the time period of voice decibel early warning;

If non-human sound noise occurs in the process of inputting the voice by the user, and the period of time when the decibel of the non-human sound noise exceeds the second score Bei Yuzhi is the period of non-human sound decibel early warning;

the background noise index is obtained after integral operation is carried out on the time period of the voice decibel early warning and the time period of the non-voice decibel early warning, and the calculation expression is as follows: Wherein Z (t) is a voice recognition error rate, [ t _x,t_y ] is a period of human voice decibel early warning, and [ t _i,t_j ] is a period of non-human voice decibel early warning.

6. The intelligent speech recognition and emotion analysis method of claim 5, wherein: the calculation logic of the voice correct recognition index is as follows: the method comprises the steps of obtaining the number of voice segments input by a user in the whole interaction process, and calculating the correct recognition rate of each segment of voice, wherein the expression is as follows: wherein F represents correct recognition rate, ZQS represents correct recognition word number of voice, ZSL represents word number of all voices in the interaction process;

The correct recognition rate of the voice segment with the largest correct recognition rate in the whole interaction process is used as a voice correct recognition index, and the expression is as follows: yzl=max (F ₁、F₂、...、F_n), where n represents the number of speech segments throughout the interaction, and n is an integer, and max () represents the maximum correct recognition rate.

7. The intelligent speech recognition and emotion analysis method of claim 6, wherein: the method comprises the steps of obtaining feedback data of a voice recognition assistant to a user, and analyzing the feedback data, wherein the steps comprise the following steps:

acquiring feedback data of a voice recognition assistant to a user, wherein the feedback data comprises recognition response speed, a cartoon frequency and voice instruction coverage rate;

Comprehensively calculating the recognition response speed, the clamping frequency and the voice instruction coverage rate to obtain a feedback coefficient;

If the feedback coefficient is more than or equal to a preset feedback threshold value, evaluating that the voice recognition performance of the voice recognition assistant is good;

if the feedback coefficient is smaller than a preset feedback threshold value, the voice recognition performance of the voice recognition assistant is estimated to be poor.

8. An intelligent speech recognition and emotion analysis system for implementing the analysis method of any one of claims 1 to 7, characterized in that: the voice recognition system comprises a wake-up module, a voice recognition module, a recording module, a data acquisition module, a judgment module, a feature extraction module, an emotion analysis module, a recognition analysis module and an optimization module;

And (3) a wake-up module: the user wakes up the voice recognition module through the wake-up module;

And a voice recognition module: the user inputs voice to the voice recognition module, and after the voice recognition module recognizes the voice, corresponding feedback is carried out based on the voice recognition content of the user;

recording module: in the voice input process of a user, recording the voice of the user;

and a data acquisition module: acquiring environment data and equipment data in the voice input process of a user;

And a judging module: after the voice interaction is completed, analyzing the environment data and the equipment data through an optimization judgment model, and judging whether emotion analysis is needed to be carried out on voice data input by a user or not;

and the feature extraction module is used for: when judging that emotion analysis is needed, acquiring recording data of a user, and extracting characteristic data in the recording data;

emotion analysis module: identifying emotion categories of the user after analyzing the characteristic data based on emotion analysis technology;

And the identification and analysis module is used for: acquiring feedback data of a voice recognition module to a user, and analyzing the feedback data;

And an optimization module: and optimizing the voice recognition module based on the recognition result of the emotion type and the feedback data analysis result.