WO2022166219A1 - Voice diarization method and voice recording apparatus thereof - Google Patents

Abstract

A voice diarization method and a voice recording apparatus thereof. The method comprises: acquiring a single-person acoustic feature from multi-channel audio data; acquiring an intermediate state of the single-person acoustic feature by using a preset recurrent-recursive neural network, and storing the intermediate state in a state sequence buffer area; in the state sequence buffer area, operating a clustering algorithm on all the intermediate states in the state sequence buffer area and obtaining at least one cluster; calculating the weighted mean square error between the intermediate state of the single-person acoustic feature and the cluster center of each cluster; and determining a cluster label of a cluster corresponding to the smallest weighted mean square error to be a cluster label of the intermediate state of the single-person acoustic feature. By means of the solution provided in the present application, a clustering process is conveniently optimized, and the diarization accuracy is improved.

Description

A kind of voice distinguishing method and voice recording device thereof technical field

The present invention relates to the technical field of audio, and in particular, to the technical field of voice discrimination.

Background technique

With the rise of deep learning, more and more mobile portable devices have joined the wave of intelligence. In many embedded smart devices, voice is often the key to unlocking the smart world. Whether in home life scenarios or in company meeting scenarios, voice is the input of many smart devices. By analyzing the speaker's voice, smart devices The speaker's instructions can be captured for the next step.

However, in such scenarios, there are often more than one speaker, and how to separate the voices of different speakers has become a problem that needs to be solved in the field of speech. Speaker Diarization is one of the issues worthy of further study. Different from speech recognition and speech separation, speaker distinction does not pay attention to who the speaker is or what the speaker said, but focuses on the question of "who spoke at what time", focusing on the differences between different speakers . After the user obtains the differentiated voices from different speakers, operations such as voice recognition can be performed to improve the corresponding accuracy.

The traditional speaker distinction method is mainly based on clustering. Most of these algorithms are offline mode, that is, to obtain a complete piece of speech, first use a sliding window to segment (or frame) the segment of speech, and then use a sliding window. Fourier transform is performed inside, the Mel cepstral coefficient (MFCC) feature or spectral feature is extracted, and then the feature is mapped to a high-dimensional space. Then move the window, adopt a certain proportion of overlapping window length, and try to ensure that each window contains only the speech of one speaker, and then calculate the embedding of speech features in the next window in the high-dimensional space. By comparing the differences in the speech embedding features of different segments, it is judged whether two speeches belong to the same speaker. Generally, the method to measure this difference is to calculate the cosine similarity of the two or the Euclidean distance in the multi-dimensional space. When the cosine similarity Or when the Euclidean distance is greater than a certain threshold, it is considered that the two are different, that is, the two speeches belong to different speakers; if it is less than the threshold, it is considered that the two speeches belong to the same speaker, and the threshold is often set based on experience or some marked data. Get tested.

However, the speech features used by the clustering algorithm, such as spectrum and amplitude spectrum features, cannot well reflect the differences of different speakers when modeling the speaker's features. Moreover, when the clustering reaches a certain level, no matter how much data training is added to the model, the improvement of the clustering accuracy is very limited. This affects the accuracy of speech discrimination.

SUMMARY OF THE INVENTION

The present application provides a voice distinguishing method and a voice recording device thereof that can improve the distinguishing accuracy.

This application provides the following technical solutions:

In one aspect, a speech analysis method is provided, which includes: obtaining a single-person acoustic feature from multi-channel audio data; obtaining an intermediate state of the single-person acoustic feature using a preset recurrent recurrent neural network, and using the intermediate state Store in the state sequence buffer; in the state sequence buffer, run a clustering algorithm on all intermediate states in the state sequence buffer and obtain at least one cluster; calculate the intermediate states and sums of the single-person acoustic features The weighted mean square error of the cluster center of each of the clusters; the cluster label of the cluster corresponding to the smallest weighted mean square error is determined as the cluster label of the intermediate state of the single-person acoustic feature.

In another aspect, a voice recording device is provided, which includes: an acoustic feature acquisition unit for extracting single-person acoustic features from multi-channel audio data; an intermediate state buffer unit for acquiring the single-person acoustic feature by using a preset recurrent recurrent neural network intermediate states of the feature, and store the intermediate states in the state sequence buffer; in the state sequence buffer, run a clustering algorithm on all the intermediate states in the state sequence buffer and obtain at least one cluster; Calculate the intermediate state of the single-person acoustic feature and the weighted mean square error of the cluster center of each of the clusters; determine the cluster label of the cluster corresponding to the minimum weighted mean square error as the single-person acoustic feature. Cluster labels for intermediate states.

The beneficial effect of the present application is that, through the trained neural network, the intermediate state of the speech data of a single person is predicted, and then the intermediate state is sent to the state buffer for clustering calculation, and the corresponding clustering label is determined. Therefore, the clustering process and the neural network are separated, which facilitates the optimization of the clustering process and improves the accuracy of discrimination.

Description of drawings

FIG. 1 is a flowchart of a method for distinguishing speech according to Embodiment 1 of the present application.

FIG. 2 is a schematic diagram of a specific flow of S110 in Embodiment 1 of the present application.

FIG. 3 is a flowchart of a supervised training process of a recurrent recurrent neural network in Embodiment 1 of the present application.

FIG. 4 is a schematic diagram of a supervised training process of a recurrent recurrent neural network in Embodiment 1 of the present application.

FIG. 5 is a schematic diagram of a testing process of a recurrent recurrent neural network in Embodiment 1 of the present application.

FIG. 6 is a schematic diagram of an update process in the status buffer in Embodiment 1 of the present application.

FIG. 7 is a schematic block diagram of a voice recording apparatus according to Embodiment 2 of the present application.

FIG. 8 is a schematic structural diagram of a voice recording apparatus according to Embodiment 3 of the present application.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to explain the present application, but not to limit the present application. However, the present application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that a thorough and complete understanding of the present disclosure is provided.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the specification of the application are for the purpose of describing particular embodiments only, and are not intended to limit the application.

It should be understood that the terms "system" or "network" are often used interchangeably herein. The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship.

The embodiments of the present application can be applied to various voice recording apparatuses with a voice distinguishing function. For example: voice recorder, audio conference terminal, intelligent conference recording device or intelligent electronic device with recording function, etc. The technical solutions of the present application will be described below through specific embodiments.

Embodiment 1

Please refer to FIG. 1 , which illustrates a method for distinguishing speech according to Embodiment 1 of the present application. The voice discrimination refers to judging the speaker corresponding to the voice information, that is, distinguishing the voice information from different sound sources. The sound source distinction does not need to obtain the complete speech emitted by the sound source, but only needs to obtain a part of it, such as a sentence, or even a word or fragment in a sentence.

The voice distinguishing method 100 includes:

S110, obtain single-person acoustic features from multi-channel audio data; optionally, the single-person acoustic features are high-dimensional vector features;

S120, using a preset cyclic recurrent neural network to obtain the intermediate state of the single-person acoustic feature, and storing the intermediate state in the state sequence buffer;

S130, in the state sequence buffer, run a clustering algorithm on all intermediate states in the state sequence buffer to obtain at least one cluster;

S140, calculating the intermediate state of the single-person acoustic feature and the weighted mean square error of the cluster center of each of the clusters;

S150. Determine the cluster label of the cluster corresponding to the smallest weighted mean square error as the cluster label of the intermediate state of the single-person acoustic feature.

Optionally, please refer to Fig. 2, S110, obtain single-person acoustic features from multi-channel audio data, including the following S111 to S117:

S111, the built-in microphone array of the voice recording device collects and reads the recorded data in real time according to data blocks, and saves the recorded data; wherein, the recorded data is real-time waveform data; optionally, the size of the data block can be set according to real-time requirements The length is generally set to 100 to 500 milliseconds; optionally, the recorded data is stored in the memory in the voice recording device to assist in verifying the speaker distinction result or for other purposes.

S112, the voice detection module detects the recorded data in the data block, and determines the recorded data in the data block.

The voice detection module includes a trained neural network. The input of the voice detection module is the waveform time domain sequence of the recorded data, and the output is a probability value. If the probability value is greater than the preset threshold, the neural network judges the data. The data in the block is the voice data, otherwise the recording data of the data block is discarded and the next data block is waited.

S113, the temporary data buffer receives the recording data.

In order to improve the accuracy of speaker distinction, a temporary data buffer is also set after the speech detection module to receive the output of the speech detection module. The size of the temporary data buffer can also be set by yourself. The larger the buffer, the more voice data streams will be accumulated. After feature extraction, speaker discrimination can improve the discrimination accuracy. However, if the buffer is set too large, you need to wait for a long time. The voice filling buffer will affect the real-time requirements, so in practical applications, the buffer size can be set according to the real-time and accuracy requirements, usually the buffer length does not exceed 3 seconds.

S114, determine whether the number of data blocks in the temporary data buffer reaches the specified capacity; if the specified capacity is reached, execute S115; otherwise, execute S111.

S115, send the data block in the temporary data buffer to the speaker quantity judgment module, and judge whether the number of speakers in the data block is greater than 1; if yes, execute S116, otherwise, execute S117;

Since the real-time voice data is the multi-channel data collected by the microphone array, the microphone array algorithm can be applied to determine how many speakers are included in the multi-channel voice data. However, in actual home life scenarios and conference scenarios, different speakers may speak at the same time, resulting in overlapping voices. Therefore, the number of speakers judgment module needs to determine how many speakers are included in the voice data in the temporary data buffer. This is a two-category task. The speech data is either a single-person speech without overlapping parts; or multiple people speaking with overlapping speech. But in S115, the number of speakers judgment module does not care how many speakers there are in the overlapping part, it only needs to judge whether the number of speakers is greater than 1. If it is determined that there is more than one speaker in the voice data, S116 is executed, otherwise, the voice data is monophonic voice data, and S117 can be executed directly.

S116, sending the voice data with the number of speakers greater than one into the array algorithm module to process the overlapping voices to obtain monophonic voice data.

Optionally, when the array algorithm determines the number of speakers, the scanning method is used, that is, the space plane is divided into different angular regions, and each region is detected separately. If voice data is detected in different angular regions, it means that If there are multiple speakers speaking at the same time in the same time period, the beamforming algorithm is used in each direction area to enhance the speech in that direction, and suppress the sounds in other directions. Through this method, the voices of different speakers are extracted and enhanced to form the monophonic voice data.

S117, extract the acoustic feature of a single speaker according to the monophonic speech data; optionally, the acoustic feature is a high-dimensional vector feature that can distinguish different speakers; optionally, a feature extraction module is used to extract the acoustic feature.

Specifically, the short-time Fourier transform is performed on the speech data, and the waveform signal in the time domain is converted into the frequency domain, and then the amplitude and phase of the spectrum are extracted to form an input vector, which is sent to the trained neural network, and the output A high-dimensional feature vector with a fixed dimension, the high-dimensional feature vector represents the acoustic features of the speaker in the speech data.

Traditional acoustic algorithms often use the Mel cepstral coefficient (MFCC) or i-vector model as the representation of the speaker's acoustic features, but these features are calculated based on mathematical models. Set a series of premise conditions, but these premise conditions are not always met in practical application scenarios. Therefore, traditional acoustic features have a certain bottleneck in representing speaker uniqueness. There are no such problems when using a neural network, and there is no limitation of pre-assumed conditions. After short-time Fourier transform is performed on the speech data, the relevant information is extracted and sent directly to the neural network, and a high-dimensional vector representing the speaker's characteristics is output. , the influence of human factors is avoided in the process, and the accuracy of speaker distinction can be improved.

Optionally, the recurrent recurrent neural network is obtained by using a supervised learning training method. The so-called supervised learning is that when a neural network is used to train a model, a reference label will be provided as a control to tell the model that the training target should be as close as possible to the label.

Optionally, please refer to FIG. 3 and FIG. 4 , FIG. 3 is the supervised training process of the RNN, and FIG. 4 is the testing process of the RNN, that is, the use process thereof. The recurrent recurrent neural network is obtained by supervised learning and training, including:

S121, assign a speaker label to the speech signal, and record the start and end times of the speech signal corresponding to the speaker label; as shown in Figure 4, the speech data of the single person of speaker 1 and the speech data of the single person of speaker 2 , all contain their respective voice data and their corresponding speaker labels; during the training process, the input voice or voice fragment can clarify the speaker identity, therefore, the voice information corresponding to the voice fragment obtained according to this voice, It is also possible to clearly know the speaker;

S122, extract the acoustic features of the voice signal; continue to refer to FIG. 4, the feature extraction module extracts the acoustic features of speaker 1 and the acoustic features of speaker 2 respectively, and at this time, the acoustic features of speaker 1 and speaker 2 are obtained. The speaker label corresponding to the acoustic feature remains unchanged, which is the speaker label assigned to it in S121;

S123, send the acoustic feature and the speaker label into a recurrent recurrent neural network, and use a loss function and an optimizer to optimize the recurrent recurrent neural network.

The above S121 to S123 are the supervised training stages of the RNN. After the training is completed, please refer to Figure 5. In the process of use, that is, in the testing phase, the input voice is not clear who the speaker is, but the trained model needs to be clustered on it, such as As described in S120 to S140, assign a speaker label to it.

Optionally, in S120, a preset cyclic recurrent neural network is used to obtain an intermediate state of the single-person acoustic feature, and the intermediate state is stored in a state sequence buffer, specifically including:

A recurrent recurrent neural network is used to obtain the intermediate states of the single-person acoustic features without speaker labels. At this time, it is not directly given which speaker the feature vector corresponding to the speech belongs to.

Optionally, S130, in the state sequence buffer, run a clustering algorithm on all intermediate states in the state sequence buffer to obtain at least one cluster, including:

Run the clustering algorithm on all the intermediate states maintained in the state buffer; although each piece of speech data contains only one speaker, in the entire audio time series, there may be multiple speakers speaking in turn. The post-class state buffer state may contain at least one class, i.e. at least one cluster, each cluster representing a speaker;

Optionally, S140, calculate the intermediate state of the single-person acoustic feature and the weighted mean square error of the cluster center of each of the clusters;

S141, calculate the cluster center of each cluster; each cluster has a cluster center, which is the mean value of all intermediate states in the cluster;

S142, calculate the weighted mean square error of the intermediate state and each cluster center; wherein, the weighted mean square error may also be called weighted Euclidean distance.

Optionally, S150, determine that the cluster label of the cluster corresponding to the minimum weighted mean square error is the cluster label of the intermediate state of the single-person acoustic feature, including:

Select the cluster label of the cluster center with the smallest weighted mean square error as the cluster label of the intermediate state; wherein, the cluster label is the speaker number; optionally, after confirming the cluster label of the intermediate state, update the cluster label. Class Center.

Optionally, S150, the cluster label of the cluster corresponding to the determined minimum weighted mean square error is the cluster label of the intermediate state of the single-person acoustic feature, including:

S151, if the cluster corresponding to the smallest weighted mean square error has an existing label, determine that the existing label is the cluster label of the single intermediate state;

S152: If the cluster corresponding to the smallest weighted mean square error has no label, assign a new label to the cluster corresponding to the smallest weighted mean square error, and determine that the new label is the one of the intermediate state Cluster labels.

Specifically, the weighted mean square error is the weighted Euclidean distance, which can represent the distance between the intermediate state to be measured and the Possibility convention of clustering; among the weighted Euclidean distances between the intermediate state to be measured and all cluster centers, the smallest one indicates that the intermediate state to be measured is closest to the cluster, so it can be determined that the intermediate state to be measured is closest to the cluster The measured intermediate state belongs to the cluster, and classifies the cluster label of the cluster for the intermediate state to be measured. However, if the intermediate state belongs to a speaker that has appeared before, then the cluster label (that is, the speaker label) already exists in the previous sequence, and the cluster label can be directly assigned to the intermediate state; if The intermediate state is not of the speaker that has appeared before, so there is no cluster label (that is, the speaker label) in the previous sequence, then a new cluster label needs to be set for the current speaker, and then the new cluster label needs to be set for the current speaker. The cluster label of is assigned to this intermediate state.

Among them, the state sequence buffer is specially used to store the intermediate state of the neural network output. However, since the voice data is time series data, as time goes by, the voice data will increase, and the intermediate state corresponding to the output of the neural network will also increase, so the calculation overhead will also increase when updating and maintaining the buffer, and the recording time is long enough. In the case of , it will take longer and longer to run the clustering algorithm, calculate the weighted mean square error, and update the cluster center, and the real-time requirements will be greatly reduced. This is the time delay accumulation problem, that is, when using the clustering algorithm to classify the speaker's speech frames, since most of the clustering methods are obtained through traversal operations, with the extension of the recording time, the speaker's speech frames continue to Increase, the delay generated during the traversal operation will be higher and higher. In the process of use, it may happen that at the beginning of the recording, the system can quickly give the speaker distinction result, but as the recording time goes on, the response of the system to give the distinction result becomes slower and slower, so Affects real-time effects.

Therefore, referring to FIG. 6 , optionally, a preset capacity value can be set for the size of the state buffer according to requirements. When there is still space in the state buffer, the intermediate state output by the neural network is stored in the buffer. If the state buffer is full, the buffer is updated according to a certain strategy to keep the buffer size unchanged. The circle or ellipse in Figure 6 represents a cluster, the solid circular figure represents the cluster center of each cluster, the solid triangular figure is the currently predicted intermediate state, and the solid diamond is the intermediate state to be discarded.

At this time, the voice distinguishing method 100 further updates the strategy of the state buffer, which specifically includes:

S161, if the space size of the state sequence buffer reaches the preset capacity value, in the state sequence buffer for storing intermediate states, calculate all intermediate states in at least one of the clusters and the cluster The Euclidean distance between the cluster centers;

S162, remove the intermediate state corresponding to the smallest Euclidean distance.

Optionally, after S161 and S162, the method further includes:

S163, adding a new intermediate state;

S164, recalculate the cluster centers of the clusters in the state sequence buffer.

The update strategy of the above state buffer can be summarized as the "most recent first out" strategy. That is, when a new intermediate state has been assigned a cluster label, the Euclidean distances of all intermediate states and cluster centers within the class are calculated and sorted. Discard the intermediate state with the smallest Euclidean distance, then add the new state to the cluster, and recalculate the cluster center. Through this strategy, the size of the buffer can be maintained unchanged, so no matter how long the recording time is, when the buffer is full, the response time of the system to distinguish the speaker can remain unchanged as a whole, reducing the response delay caused by the increase in the amount of calculation. cumulative problems. Adopting the strategy of "recent, first out" can ensure that there will be no major difference in the discrimination accuracy, because the closer the distance to the cluster center, the more likely it is that the intermediate state belongs to this category, and the lower the uncertainty, the more likely it is to remain in this category. The value of making further judgments in the state buffer is lower, so you can choose to discard it; on the contrary, the farther away from the cluster center, the greater the uncertainty, and the greater the value of making further judgments in the state buffer. high, therefore, needs to be reserved.

In the embodiment of the present application, by training the neural network through supervised learning, a large amount of labeled data can be used to improve the accuracy of speaker discrimination. The more labeled training data, the higher the algorithm discrimination accuracy. Then, use the recurrent recurrent neural network trained by supervised learning to predict the intermediate state of the speech data of a single person, send the intermediate state to the state buffer for clustering calculation, and determine the corresponding clustering label. Thus, the clustering process and the neural network are separated, which facilitates the optimization of the clustering process. On the other hand, the solution provided by the embodiments of the present application also maintains and updates the state buffer according to the "last first out" strategy, so as to ensure that the state transition area will not cause the system to run more and more due to the accumulation of delay. Therefore, the problem of time delay accumulation in the running process of the real-time clustering algorithm is solved, so as to achieve the effect of real-time speaker discrimination, and improve the real-time performance of the device or system running the speech discrimination method. The so-called real-time speaker discrimination does not require the acquisition of a complete voice file, and provides the judgment result of the speaker's identity at the previous moment in the form of low-latency while the speaker is speaking.

Embodiment 2

Please refer to FIG. 7 , which shows a voice recording apparatus 200 according to Embodiment 2 of the present application. The voice recording device 200 includes, but is not limited to, any one of a voice recorder, an audio conference terminal, or an intelligent electronic device with a recording function, etc. It may also not include a voice pickup function, but only include a voice discrimination function, which can realize this function. A computer or other intelligent electronic device is not limited in the second embodiment.

The voice recording device 200 includes:

Acoustic feature acquisition unit 210, extracting single-person acoustic features from multi-channel audio data;

The intermediate state buffer unit 220 adopts a preset cyclic recursive neural network to acquire the intermediate state of the single-person acoustic feature, and stores the intermediate state in the state sequence buffer; in the state sequence buffer, the All intermediate states in the state sequence buffer run the clustering algorithm and obtain at least one cluster; calculate the intermediate state of the single-person acoustic feature and the weighted mean square error of the cluster center of each of the clusters; determine the smallest all The cluster label of the cluster corresponding to the weighted mean square error is the cluster label of the intermediate state of the single-person acoustic feature.

Optionally, the recurrent recurrent neural network is obtained by using a supervised learning training method. Optionally, the voice recording device further includes a cyclic recurrent neural network obtaining unit 230, configured to assign a speaker tag to the voice signal, and record the start and end times of the voice signal corresponding to the speaker tag; extract the voice signal. The acoustic feature of the signal; the acoustic feature and the speaker label are sent into a recurrent recurrent neural network, and the recurrent recurrent neural network is optimized by using a loss function and an optimizer.

Optionally, the space size of the state sequence buffer is a preset capacity value; then the intermediate state buffer unit 220 is further configured to, if the space size of the state sequence buffer reaches the preset capacity value, In the state sequence buffer storing the intermediate states, calculate the Euclidean distance between all intermediate states in at least one of the clusters and the cluster center of the cluster; remove the minimum corresponding to the Euclidean distance Intermediate state.

Optionally, the intermediate state cache unit 220 further includes: adding a new intermediate state; and recalculating the cluster centers of the clusters in the state sequence buffer.

Optionally, the intermediate state cache unit 220 is configured to determine that the cluster label of the cluster corresponding to the minimum weighted mean square error is the cluster label of the intermediate state of the single-person acoustic feature, specifically including:

The intermediate state cache unit 220 is specifically configured to determine that the existing label is the cluster label of the intermediate state if the cluster corresponding to the smallest weighted mean square error has an existing label; If the cluster corresponding to the mean square error has no label, a new label is assigned to the cluster corresponding to the smallest weighted mean square error, and the new label is determined to be the cluster label of the intermediate state.

For details, optimization solutions, or specific examples in Embodiment 2, please refer to the same or corresponding parts in Embodiment 1 above, which will not be repeated here.

Embodiment 3

Please refer to FIG. 8 , which is a schematic structural diagram of a voice recording apparatus 300 according to Embodiment 3 of the present application. The video processing apparatus 300 includes: a processor 310 and a memory 320 . The communication connection between the processor 310 and the memory 320 is realized through a bus system. The processor 310 invokes the program in the memory 320 to execute any one of the speech analysis methods provided in the first embodiment above.

The processor 310 may be an independent component, or may be a collective term for multiple processing components. For example, it may be a CPU, an ASIC, or one or more integrated circuits configured to implement the above method, such as at least one microprocessor DSP, or at least one programmable gate FPGA, etc. The memory 320 is a computer-readable storage medium on which programs executable on the processor 310 are stored.

Optionally, the voice processing device 300 further includes: a sound pickup device 330 for acquiring voice information. The processor 310, the memory 320, and the sound pickup device 330 are connected to each other through a bus system for communication. The processor 310 invokes the program in the memory 320, executes any one of the speech analysis methods provided in the first embodiment, and processes the multi-channel speech information acquired by the sound pickup device 330.

For details that are not detailed in the third embodiment, please refer to the same or corresponding parts in the above-mentioned first embodiment, which will not be repeated here.

Those skilled in the art should realize that, in one or more of the above examples, the functions described in the specific embodiments of the present application may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it may be implemented by a processor executing software instructions. The software instructions may consist of corresponding software modules. The software modules may be stored in a computer-readable storage medium, which may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that includes an integration of one or more available media. The available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, Digital Video Disc (DVD)), or semiconductor media (eg, Solid State Disk (SSD)) )Wait. The computer-readable storage medium includes but is not limited to random access memory (Random Access Memory, RAM), flash memory, read only memory (Read Only Memory, ROM), Erasable Programmable Read Only Memory (Erasable Programmable ROM, EPROM) ), Electrically Erasable Programmable Read-Only Memory (Electrically EPROM, EEPROM), registers, hard disks, removable hard disks, compact disks (CD-ROMs), or any other form of storage medium known in the art. An exemplary computer-readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the computer-readable storage medium. Of course, the computer-readable storage medium can also be an integral part of the processor. The processor and computer-readable storage medium may reside in an ASIC. Additionally, the ASIC may reside in access network equipment, target network equipment or core network equipment. Of course, the processor and the computer-readable storage medium may also exist as discrete components in the access network device, the target network device or the core network device. When implemented in software, it can also be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer or a chip, which may include a processor, all or part of the processes or functions described in the specific embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer program instructions may be stored on or transmitted from one computer readable storage medium to another computer readable storage medium as described above, for example, the computer instructions may be downloaded from a website, computer, server or The data center transmits to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, Digital Subscriber Line, DSL) or wireless (such as infrared, wireless, microwave, etc.).

The above-mentioned embodiments illustrate but do not limit the present invention, and those skilled in the art can devise various alternative examples within the scope of the claims. It should be appreciated by those skilled in the art that the application is not limited to the precise structures described above and illustrated in the accompanying drawings, but may be applied without departing from the scope of the invention as defined by the appended claims. Appropriate adjustments, modifications, equivalent substitutions, improvements, etc. are made to the specific implementation scheme. Therefore, any modifications and changes made in accordance with the concept and principles of the present invention are within the scope of the present invention as defined by the appended claims.

Claims (14)

A method for distinguishing speech, characterized in that the method comprises: Obtain single-person acoustic features from multi-channel audio data; Use a preset RNN to obtain the intermediate state of the single-person acoustic feature, and store the intermediate state in the state sequence buffer; In the state sequence buffer, run a clustering algorithm on all intermediate states in the state sequence buffer and obtain at least one cluster; Calculate the intermediate state of the single-person acoustic feature and the weighted mean square error of the cluster centers of each of the clusters; The cluster label of the cluster corresponding to the smallest weighted mean square error is determined as the cluster label of the intermediate state of the single-person acoustic feature. The method of claim 1, wherein the recurrent recurrent neural network is obtained by a supervised learning training method. The method of claim 2, wherein the recurrent recurrent neural network is obtained by a supervised learning training method, comprising: Allocate a speaker tag to the voice signal, and record the start and end time of the voice signal corresponding to the speaker tag; extracting acoustic features of the speech signal; The acoustic features and the speaker labels are sent into a recurrent recurrent neural network, and a loss function and an optimizer are used to optimize the recurrent recurrent neural network. The method according to any one of claims 1 to 3, wherein the space size of the state sequence buffer is a preset capacity value, and the method further comprises: If the space size of the state sequence buffer reaches the preset capacity value, in the state sequence buffer for storing intermediate states, calculate all intermediate states in at least one of the clusters and the clustering of the cluster. Euclidean distance between class centers; The intermediate state corresponding to the smallest said Euclidean distance is removed. The method of claim 4, wherein the method further comprises: Add a new intermediate state; Recompute the cluster centers of the clusters in the state sequence buffer. The method according to claim 1, characterized in that, determining that the cluster label of the cluster corresponding to the minimum weighted mean square error is the cluster label of the intermediate state of the single-person acoustic feature, comprising: If the cluster corresponding to the smallest weighted mean square error has an existing label, then determine that the existing label is the cluster label of the intermediate state; If the cluster corresponding to the smallest weighted mean square error has no label, assign a new label to the cluster corresponding to the smallest weighted mean square error, and determine that the new label is the cluster of the intermediate state Label. A voice recording device, wherein the voice recording comprises: Acoustic feature acquisition unit, extracting single-person acoustic features from multi-channel audio data; The intermediate state cache unit adopts a preset cyclic recursive neural network to obtain the intermediate state of the single-person acoustic feature, and stores the intermediate state in the state sequence buffer; in the state sequence buffer, the state All intermediate states in the sequence buffer run the clustering algorithm and obtain at least one cluster; calculate the intermediate state of the single-person acoustic feature and the weighted mean square error of the cluster center of each of the clusters; determine the smallest The cluster label of the cluster corresponding to the weighted mean square error is the cluster label of the intermediate state of the single-person acoustic feature. The voice recording device according to claim 7, wherein the recurrent recurrent neural network is obtained by a supervised learning training method. The voice recording device according to claim 8, wherein the voice recording device further comprises a recurrent recurrent neural network obtaining unit, configured to assign a speaker label to the speech signal, and record the corresponding speaker label. The start and end time of the speech signal; extract the acoustic features of the speech signal; send the acoustic features and the speaker label into the recurrent recurrent neural network, and use the loss function and the optimizer to carry out the recurrent neural network. optimization. The voice recording device according to any one of claims 7 to 9, wherein the space size of the state sequence buffer is a preset capacity value; Then, the intermediate state buffer unit is further configured to, if the space size of the state sequence buffer reaches the preset capacity value, in the state sequence buffer storing the intermediate state, calculate all the data in at least one of the clusters. Euclidean distance between the intermediate state and the cluster center of the cluster; remove the intermediate state corresponding to the smallest Euclidean distance. The voice recording device according to claim 10, wherein the intermediate state buffer unit further comprises: adding a new intermediate state; and recalculating the cluster centers of the clusters in the state sequence buffer. The voice recording device according to claim 7, wherein the intermediate state cache unit is configured to determine that the cluster label of the cluster corresponding to the smallest weighted mean square error is the intermediate state of the single-person acoustic feature The clustering labels of , specifically include: The intermediate state cache unit is specifically configured to determine that the existing label is the cluster label of the intermediate state if the cluster corresponding to the smallest weighted mean square error has an existing label; If the cluster corresponding to the variance has no label, a new label is assigned to the cluster corresponding to the smallest weighted mean square error, and the new label is determined as the cluster label of the intermediate state. A voice recording device, characterized in that, the voice recording device comprises: a processor and a memory; the processor invokes a program in the memory to perform the voice distinction described in any one of the above claims 1 to 6 method. A computer-readable storage medium, characterized in that a program of a speech analysis method is stored on the computer-readable storage medium, and when the program of the speech analysis method is executed by a processor, any one of the above claims 1 to 6 is implemented The voice discrimination method described in item.

Images