CN116564329A - Real-time conversation voiceprint noise reduction method, electronic equipment and storage medium - Google Patents

Abstract

A real-time call voiceprint noise reduction method, electronic equipment, and storage medium, wherein the real-time call voiceprint noise reduction method includes: obtaining real-time call audio, the first voiceprint feature vector of the current speaker, and the registration audio of the current speaker; The registration audio is input to the voiceprint feature extraction network that is trained synchronously with the pre-training noise reduction network to obtain the second voiceprint feature vector; the real-time call audio is input to the encoding part of the pre-training noise reduction network to obtain the third A voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts; after splicing the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector Input to other parts of the pretrained denoising network. In this way, the output audio can better retain the audio of the speaker, and effectively suppress the interference of other human voices.

Description

Real-time call voiceprint noise reduction method, electronic equipment and storage medium

technical field

The embodiments of the present application relate to the technical field of speech processing, and in particular, to a real-time call voiceprint noise reduction method, electronic equipment, and a storage medium.

Background technique

Voiceprint call noise reduction, as the name suggests, is to add voiceprint information to the call noise reduction technology, even in a noisy environment, or in a complex scene where there are multiple interfering people talking, you can extract clear speaker voice information , to filter out the speech of other speakers and background noise. This technology has a very wide range of applications in actual production and life.

In the prior art, the voiceprint noise reduction solution includes a registration phase and a testing phase.

Among them, the registration stage: the speaker first registers a 20-30s audio in a quiet scene according to the prompt on the user interface (UI, User Interface), which is used to extract the voiceprint information of the speaker. In the process of real use, the user may not speak clearly, the environmental background noise is relatively large, and the registered speaker speaks too fast, and the recording time is too short, etc., which will affect the speaker's information collection, which will affect the subsequent algorithm. normal use. For this reason, there are restrictions on the quality of the audio during the registration phase. Specifically, voice quality: Use voice activity detection (Voice Activity Detection, VAD) to detect the registration audio, and calculate the signal-to-noise ratio of the registration audio according to the audio and background; The correct text is verified; finally, the audio length after vad must meet a certain length. Only when these conditions are met can the voiceprint information be registered correctly.

Testing phase: Since most PC products currently on the market often have more than one microphone for sound collection, a microphone array composed of multiple microphones can better personalize voice signals. Voiceprint noise reduction generally requires the speaker in front of the notebook to be enhanced, the voice signals in other directions should be suppressed, and other voices in the same direction that interfere with others should also be suppressed, so the following technologies will be used. Echo cancellation, microphone array technology, voiceprint noise reduction technology and automatic gain control technology.

At present, the equipment that can be used to experience is less in variety except notebooks and mobile phones, and the performance is not good. The main defect is the suppression of the voice of the registered speaker. It is the model that confuses the voice of the speaker with the interfering person, and suppresses them; and the suppression of the interfering human voice is not complete, there will be audio residues, and even the interfering human voice cannot be eliminated. However, according to the audio effect after our offline processing, we found that our algorithm can well preserve the audio of the registered speaker and effectively suppress the interference of other speakers.

Voiceprint noise reduction and other noise reduction will also face the problem of unclean or excessive elimination of the speaker, but in the process of actual use, we will try our best to ensure that the voice of the speaker can be preserved as much as possible, even if the interfering person will Some residue is present.

Contents of the invention

Embodiments of the present invention provide a real-time call voiceprint noise reduction method and device, which are used to solve at least one of the above technical problems.

In the first aspect, an embodiment of the present invention provides a real-time call voiceprint noise reduction method, including: acquiring real-time call audio, the first voiceprint feature vector of the current speaker, and the registration audio of the current speaker; inputting the registration audio to The second voiceprint feature vector is obtained in the voiceprint feature extraction network synchronously trained with the pre-training noise reduction network; the real-time call audio is input to the encoding part of the pre-training noise reduction network to obtain a third voiceprint feature vector, wherein , the pre-trained noise reduction network includes an encoding part and other parts; the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector are concatenated and input to the pre-training Other parts of the denoising network.

In a second aspect, an embodiment of the present invention provides an electronic device, which includes: at least one processor, and a memory connected to the at least one processor in communication, wherein the memory stores information that can be used by the at least one processor Executable instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute any one of the above-mentioned real-time call voiceprint noise reduction methods of the present invention.

In the third aspect, the embodiment of the present invention provides a storage medium, and one or more programs including execution instructions are stored in the storage medium, and the execution instructions can be read by electronic devices (including but not limited to computers, servers, or network equipment, etc.) to read and execute any one of the above-mentioned real-time call voiceprint noise reduction methods of the present invention.

In a fourth aspect, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the The computer executes any one of the above real-time call voiceprint noise reduction methods.

The method of the present application obtains the real-time call audio, the first voiceprint feature vector of the current speaker and the registration audio of the current speaker, and then inputs the registration audio to his synchronously trained voiceprint feature extraction network to obtain the second voiceprint feature vector, and then The real-time call audio is input to the encoder part of the pre-training noise reduction network to obtain the third audio feature vector, and finally the first voiceprint feature vector, the second voiceprint feature vector and the third audio feature vector are spliced and output to the pre-training denoising network. Other parts of the noise network are processed, so that the output audio can better retain the speaker's audio, and effectively suppress the interference of other human voices.

Description of drawings

Fig. 1 is a flow chart of a real-time call voiceprint noise reduction method provided by an embodiment of the present invention;

Fig. 2 is a flow chart of another real-time call voiceprint noise reduction method provided by an embodiment of the present invention;

Fig. 3 is a schematic diagram of overspeaker elimination of a specific example of the prior art provided by an implementation of the present invention;

Fig. 4 is a schematic diagram of a specific example of the prior art provided by an implementation of the present invention, where the disturber is not completely eliminated;

Fig. 5 is a real-time call voiceprint noise reduction network framework diagram of a specific example of a real-time call voiceprint noise reduction method provided by an implementation of the present invention;

6 is a flow chart of the real-time call voiceprint noise reduction registration stage of a specific example of a real-time call voiceprint noise reduction method provided by an implementation of the present invention;

Fig. 7 is a flow frame diagram of a real-time call voiceprint noise reduction test stage flow chart of a specific example of a real-time call voiceprint noise reduction method provided by an implementation of the present invention;

FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Please refer to FIG. 1 , which shows a flow chart of a real-time call voiceprint noise reduction method provided by an embodiment of the present invention.

As shown in Figure 1, in step 101, obtain real-time call audio, the first voiceprint feature vector of the current speaker and the registration audio of the current speaker;

In step 102, input the registration audio into the voiceprint feature extraction network that is trained synchronously with the pre-trained noise reduction network to obtain a second voiceprint feature vector;

In step 103, the real-time call audio is input to the encoding part of the pre-trained noise reduction network to obtain a third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts;

In step 104, the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector are concatenated and input to other parts of the pre-trained denoising network.

In this embodiment, for step 101, the real-time call audio, the first voiceprint feature vector of the current speaker and the registration audio of the current speaker are obtained, for example, the real-time call audio is obtained through the device during the call, and the preparation in advance A good first voiceprint feature vector and registration audio, and the obtained first voiceprint feature vector and registration audio belong to the same speaker.

Then, for step 102, the registration audio is input into the voiceprint feature extraction network that is trained synchronously with the pre-trained denoising network to obtain a second voiceprint feature vector, for example, the voiceprint features that are synchronously trained through the pre-trained denoising network The extraction network extracts the voiceprint information of the registration audio to obtain the second voiceprint feature vector.

Then, for step 103, the real-time call audio is input to the encoding part of the pre-trained noise reduction network to obtain a third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts, for example, The real-time call audio obtained by the device is input to the pre-trained noise reduction network to obtain the third voiceprint feature vector through the encoder, and the pre-training noise reduction network includes the encoding part and other parts.

Finally, for step 104, the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector are concatenated and input to other parts of the pre-trained denoising network, for example, The first, second, and third voiceprint feature vectors are spliced, and then the spliced voiceprint feature vectors are input to other parts of the pre-trained denoising network for processing.

The method of this embodiment is to obtain the real-time call audio, the first voiceprint feature vector of the current speaker and the registration audio of the current speaker, and then input the registration audio to his synchronously trained voiceprint feature extraction network to obtain the second voiceprint feature vector, Then input the real-time call audio to the encoder part of the pre-training noise reduction network to obtain the third audio feature vector, and finally splice the first voiceprint feature vector, the second voiceprint feature vector and the third audio feature vector and output it to the pre-training Other parts of the noise reduction network are processed so that the output audio can better retain the speaker's audio and effectively suppress the interference of other human voices.

In some optional embodiments, the first voiceprint feature vector of the current speaker is obtained by inputting the registration audio of the current speaker to a fixed speaker feature extractor for feature extraction operation, thereby obtaining the first voiceprint feature Vector, so that the first voiceprint feature vector is obtained in advance to facilitate subsequent operations.

Further please refer to FIG. 2 , which shows another flow chart of real-time call voiceprint noise reduction according to an embodiment of the present invention. A further defined step-by-step diagram of the "synchronous training of texture feature extraction network" training step.

As shown in Figure 2, in step 201, input the noisy audio to the encoding part of the pre-trained denoising network to obtain an encoded result, wherein the noisy audio has a corresponding clean audio and the The noise audio belongs to the registration audio of the same speaker;

In step 202, at least splicing the voiceprint extraction result obtained by passing the registration audio through the voiceprint feature extraction network with the coded result to obtain a spliced result;

In step 203, continue to input the spliced result to other parts of the pre-trained denoising network for processing to obtain the output of the pre-trained denoising network;

In step 204, the output of the pre-trained denoising network and the loss of the clean audio are calculated, and the pre-trained denoising network and the voiceprint extraction model are trained based on the losses.

In this embodiment, for step 201, the noisy audio is input to the encoding part of the pre-trained denoising network to obtain an encoded result, wherein the noisy audio has a corresponding clean audio and is the same as the noisy audio The registered audio frequency belongs to the same speaker. For example, in the training phase, the pre-prepared noisy audio input is input into the pre-trained denoising network for processing through its encoder to obtain the encoded result, and the noisy audio has its corresponding The clean audio of , and the registration audio belonging to the same speaker as the noisy audio.

Then, for step 202, at least the voiceprint extraction result obtained by passing the registration audio through the voiceprint feature extraction network is spliced with the encoded result to obtain the spliced result. For example, the registration audio needs to pass through the voiceprint at least The network obtains the result of voiceprint extraction, and splices the result of voiceprint extraction with the encoded result to obtain the spliced result.

Then, for step 203, continue to input the spliced result to other parts of the pre-training noise reduction network for processing to obtain the output of the pre-training noise reduction network, for example, after combining the voiceprint extraction result with the encoding The final results are spliced and input to other parts of the pre-trained noise reduction network for processing and output to obtain the output audio.

Finally, for step 204, calculate the output of the pre-training noise reduction network and the loss of the clean audio, and train the pre-training noise reduction network and the voiceprint extraction model based on the loss, for example, reduce the pre-training The audio output by the noise network is compared with the clean audio to obtain the loss of the processed audio, and the pre-trained noise reduction network and voiceprint extraction model are trained through the audio loss.

The method of this embodiment inputs the corresponding clean audio and the noisy audio of the registration audio of the same speaker to the encoder part of the pre-trained voiceprint feature extraction network to obtain the encoded result, and then inputs the registration audio to the voiceprint extraction The device obtains the extracted result, obtains the voiceprint extracted result, splices it with the encoded result, and inputs it to other parts of the pre-trained noise reduction network to process and output the output audio. Finally, the pre-trained noise reduction network The output audio is compared and calculated with the clean audio, the loss of the processed audio is obtained, and the pre-trained noise reduction network and voiceprint extraction model are trained through the audio loss.

In some possible embodiments, the registration audio can be input into a fixed speaker feature vector extractor to obtain a fixed extraction result, then the registration audio can be input into the voiceprint extraction result obtained by the voiceprint feature extraction network, and finally the fixed The extraction results, the voiceprint feature extraction network extraction results, and the encoded results are spliced to obtain the spliced result, so that the spliced audio is more accurate.

In some optional embodiments, the output of the pre-trained denoising network and the loss of clean audio are calculated by a scale-invariant signal-to-noise ratio (sisnr loss, scale-invariant source-to-noise ratio) loss function, the reason The reason for using the scale-invariant signal-to-noise ratio loss function calculation is that no speaker-related loss function is added in the training stage of the model, so that the pre-trained noise reduction network can only have the function of eliminating interfering human voices, and then through the scale-invariant signal-to-noise ratio loss function. The noise ratio makes full use of the low-dimensional voiceprint information, thereby achieving the function of eliminating irrelevant voiceprint information and retaining only the voiceprint information of the registered speaker.

In some optional embodiments, the noisy audio is obtained by mixing the clean audio of the current speaker with at least one interfering audio, which can be directly mixed. It is also possible to superimpose the clean audio with different room impulse responses first, and then add different signal-to-noise ratio noises for the speaker and the interferer, so that the noisy audio can better simulate the real call scene.

In some optional embodiments, before the real-time call audio is input to the encoding part of the pre-trained noise reduction network, it is necessary to perform echo cancellation (AEC, Acoustic Echo Cancellation) and beamforming (BF, Beam Forming) processing on the real-time call audio , the processed audio is input to the pre-trained noise reduction network for processing, and the processed audio is automatically gained using AGC (Automatic Gain Control), so that the final output audio is more comfortable and stable.

In some optional embodiments, the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector are concatenated and processed by other parts of the pre-trained noise reduction network, so that the processed audio can be sent to For the remote person in the real-time conversation, it can also be input into the speech recognition engine.

Please refer to FIG. 3 , which shows a schematic diagram of overspeaker elimination in a specific example of the prior art provided by an embodiment of the present invention.

As shown in Figure 3, the Mix audio is the noisy audio, and the Ref audio is the clean audio corresponding to the noisy audio. From the noise-reduced audio, it can be seen that the noisy audio of the registered speaker is over-eliminated after being processed by the noise reduction network. Audio, which is caused by the noise reduction network confusing the registered speaker's voice with that of the interferer.

Please refer to FIG. 4 , which shows a schematic diagram of an incomplete elimination of disturbers according to a specific example of the prior art provided by an implementation of the present invention.

As shown in Figure 4, the Mix audio is the noisy audio, and the Ref audio is the clean audio corresponding to the noisy audio. From the noise-reduced audio, it can be seen that the noisy audio of the registered speaker is processed by the noise reduction network and appears to interfere with the interfering person. Incomplete audio suppression leads to residual noise.

The inventor has tried the following technical solutions in the process of implementing this application: one of the solutions uses a fixed speaker vector, and the module is not updated during the training process of the model. The result of this solution is that the vocals of other people are not eliminated enough thorough. Another solution is to jointly train the speaker model and the speech enhancement model. This method is more complicated in the process of model training, and fails to consider the real-time problem in real-world scenarios.

The technical scheme of this application is mainly designed and optimized from the following aspects:

The voiceprint extraction of the application scheme includes two parts, one extracts the embedding feature (embedding) module for text-independent voiceprint recognition, this module is trained by the model, and does not need to be retrained during the training process of the voiceprint denoising model .

The other part of the voiceprint feature extraction changes with the weight of the voiceprint noise reduction model training network, so that as much information as possible can be extracted from the speaker to ensure that the speaker does not lose words in the actual use process.

In the data preparation stage, the registration audio of each speaker is prepared first, with a length of 30s, and the test audio of each speaker is spliced into a long audio. During the model training phase, the data will be read in real time, and each time a speaker will be selected for 10 seconds of quiet audio, and the disturber can choose 0-3 disturbers each time. And add noise with different signal-to-noise ratios for the speaker and the interferer. In order to better simulate the real scene, we can superimpose the clean audio with the impact response of different rooms.

Please refer to FIG. 5 , which shows a real-time call voiceprint noise reduction network framework diagram of a specific example of a real-time call voiceprint noise reduction method provided by an implementation of the present invention.

As shown in Figure 5, we combine the speech enhancement and speaker feature vector extraction model, step 1: extract emb, where emb refers to the embedding feature mentioned earlier. Convert the prepared data, registered audio, clean audio, and mixed audio into the frequency domain. Here we use the Hanning window, the frame length is 512, the frame shift is 256, and the length of fft is 512;

Step 2: Operate the noisy audio according to the process of the noise reduction network on the right in the figure, and calculate the sisnr loss between the output of the network and the clean audio;

Step 3: After feature extraction, the registered audio is input into the network and then further processed in the time dimension, and then the same operation is performed with the noise audio, and the output of the registered audio through the network is spliced after the noise audio input network, where, I- Feature refers to converting speech from the frequency domain to the time domain.

Please refer to FIG. 6 , which shows a flow chart of the real-time call voiceprint noise reduction registration stage of a specific example of a real-time call voiceprint noise reduction method provided by an implementation of the present invention.

As shown in Figure 6, step 1: the user registers about 20-30s of registration audio in advance; step 2 conducts quality inspection on the registration audio; step 3: extracts the speaker’s voiceprint feature 1; step 4: extracts the speaker’s Voiceprint feature 2. Wherein, voiceprint feature 1 is a fixed feature extracted by a fixed speaker feature extractor, and voiceprint feature 2 is extracted by a voiceprint feature extraction network synchronously trained by a pre-trained noise reduction network.

Please refer to FIG. 7 , which shows a flow chart of a real-time call voiceprint noise reduction test phase flow chart of a specific example of a real-time call voiceprint noise reduction method provided by an implementation of the present invention.

As shown in Figure 7, step 1: the user selects the human voice separation mode. Step 2: The microphone records the mixed audio; Step 3: Obtains the mixed human voice to extract its voiceprint features; Step 4: Processes the audio using AEC (Acoustic Echo Cancellation); Step 5: Processes it using Beam Forming (BF, Beam Forming) Audio; Step 6: Input the audio into the voiceprint noise reduction network for processing;

Step 7: The processed audio is input to the automatic gain control (AGC, Automatic Gain Control) to gain the audio and then output the audio.

In some other embodiments, the embodiments of the present invention also provide a non-volatile computer storage medium, the computer storage medium stores computer-executable instructions, and the computer-executable instructions can execute any of the above method embodiments for Lead tag identification method for sales calls;

As an implementation mode, the non-volatile computer storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are set to:

Obtain real-time call audio, the first voiceprint feature vector of the current speaker and the registration audio of the current speaker;

Inputting the registration audio into a voiceprint feature extraction network synchronously trained with a pre-trained noise reduction network to obtain a second voiceprint feature vector;

Inputting the real-time call audio to the encoding part of the pre-trained noise reduction network to obtain a third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts;

The first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector are concatenated and input to other parts of the pre-trained denoising network.

The non-volatile computer-readable storage medium can include a storage program area and a storage data area, wherein the storage program area can store an operating system, an application program required by at least one function; Data created by the use of tag identification devices, etc. In addition, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the non-transitory computer-readable storage medium optionally includes memory located remotely from the processor, and the remote memory may be connected to the lead tag identification device for sales calls via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

An embodiment of the present invention also provides a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by the computer, the computer executes the above-mentioned Any one is used for the noise reduction method of real-time call voiceprint.

FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As shown in FIG. 8 , the device includes: one or more processors 810 and memory 820 , and one processor 810 is taken as an example in FIG. 8 . The device for the lead tag identification method for sales calls may further include: an input device 830 and an output device 840 . The processor 810, the memory 820, the input device 830, and the output device 840 may be connected through a bus or in other ways. In FIG. 8, connection through a bus is taken as an example. The memory 820 is the above-mentioned non-volatile computer-readable storage medium. The processor 810 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 820 , that is, realizes the lead label identification method for sales calls in the above method embodiment. The input device 830 can receive input numbers or character information, and generate key signal input related to user settings and function control of the lead label recognition device for sales calls of the embodiment. The output device 840 may include a display device such as a display screen.

The above-mentioned products can execute the methods provided by the embodiments of the present invention, and have corresponding functional modules and beneficial effects for executing the methods. For technical details not described in detail in this embodiment, refer to the method provided in the embodiment of the present invention.

As an implementation, the above-mentioned electronic device is applied to a lead label identification device for sales calls, and includes: at least one processor; and a memory connected to at least one processor; Processor-executable instructions, the instructions being executed by at least one processor to enable at least one processor to:

Obtain real-time call audio, the first voiceprint feature vector of the current speaker and the registration audio of the current speaker;

Inputting the registration audio into a voiceprint feature extraction network synchronously trained with a pre-trained noise reduction network to obtain a second voiceprint feature vector;

Inputting the real-time call audio to the encoding part of the pre-trained noise reduction network to obtain a third voiceprint feature vector, wherein the pre-trained noise reduction network includes an encoding part and other parts;

The first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector are concatenated and input to other parts of the pre-trained denoising network.

Electronic devices in the embodiments of this application exist in various forms, including but not limited to:

(1) Mobile communication equipment: This type of equipment is characterized by mobile communication functions, and its main goal is to provide voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, feature phones, and low-end phones.

(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, with computing and processing functions, and generally also has the characteristics of mobile Internet access. Such terminals include: PDA, MID and UMPC equipment, such as iPad.

(3) Portable entertainment equipment: This type of equipment can display and play multimedia content. Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

(4) Server: A device that provides computing services. The composition of a server includes a processor, hard disk, memory, system bus, etc. The server is similar to a general-purpose computer architecture, but due to the need to provide high-reliability services, it is important in terms of processing power and stability. , Reliability, security, scalability, manageability and other aspects have high requirements.

(5) Other electronic devices with data interaction function.

The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place , or can also be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.

Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic Discs, optical discs, etc., include several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods of various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A real-time conversation voiceprint noise reduction method comprises the following steps:

acquiring real-time conversation audio, a first voiceprint feature vector of a current speaker and registration audio of the current speaker;

inputting the registered audio to a voiceprint feature extraction network which is synchronously trained with a pre-training noise reduction network to obtain a second voiceprint feature vector;

inputting the real-time call audio to a coding part of the pre-training noise reduction network to obtain a third voiceprint feature vector, wherein the pre-training noise reduction network comprises the coding part and other parts;

and splicing the first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector, and inputting the spliced first voiceprint feature vector, the second voiceprint feature vector and the third voiceprint feature vector to other parts of the pre-training noise reduction network.

2. The method of claim 1, wherein the first voiceprint feature vector of the current speaker is obtained by:

and performing feature extraction on the registered audio of the current speaker by using a fixed speaker feature extractor to obtain the first voiceprint feature vector.

3. The method of claim 1, wherein the method of synchronous training of the pre-training noise reduction network and the voiceprint feature extraction network comprises:

inputting a noisy frequency to an encoding part of the pre-training noise reduction network to obtain an encoded result, wherein the noisy frequency has corresponding clean audio and registered audio belonging to the same speaker as the noisy frequency;

at least splicing a voiceprint extraction result obtained by the registered audio through the voiceprint feature extraction network with the encoded result to obtain a spliced result;

continuously inputting the spliced result to other parts of the pre-training noise reduction network for processing to obtain the output of the pre-training noise reduction network;

calculating the output of the pre-training noise reduction network and the loss of the clean audio, and training the pre-training noise reduction network and the voiceprint extraction model based on the loss.

4. A method according to claim 3, wherein the splicing at least the voiceprint extraction result of the registered audio through the voiceprint feature extraction network with the encoded result to obtain a spliced result comprises:

and splicing a fixed extraction result obtained by the fixed speaker feature extractor of the registered audio, a voiceprint extraction result obtained by the voiceprint feature extraction network of the registered audio and the encoded result to obtain a spliced result.

5. The method of claim 3 or 4, wherein the calculating the output of the pre-training noise reduction network and the loss of clean audio comprises:

and calculating the output of the pre-training noise reduction network and the scale-invariant signal-to-noise ratio loss of the clean audio.

6. A method according to claim 3, wherein the noisy frequency is obtained by:

mixing at least the clean audio of the current speaker with the interference audio of at least one interference person to obtain the noisy audio.

7. The method of claim 1, wherein prior to inputting the real-time talk audio into the encoded portion of the pre-trained noise reduction network, the method further comprises:

performing echo cancellation processing and beam forming processing on the real-time call audio;

after the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector are spliced and input to other portions of the pre-trained noise reduction network, the method further comprises:

and performing automatic gain control processing on the output of the pre-training noise reduction network.

8. The method of claim 1 or 7, wherein after concatenating the first voiceprint feature vector, the second voiceprint feature vector, and the third voiceprint feature vector, inputting to the other portion of the pre-trained noise reduction network, the method further comprises:

the output of the pre-training noise reduction network is sent to a far-end person who communicates in real time; or (b)

And sending the output of the pre-training noise reduction network to a voice recognition engine.

9. An electronic device, comprising: at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any one of claims 1 to 7.

10. A storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the method according to any of claims 1 to 7.