CN116959475A - A speech denoising method based on improved spectral subtraction - Google Patents

Abstract

The invention discloses a speech denoising method based on improved spectral subtraction, which includes the following steps: inputting noisy speech, preprocessing the signal of the noisy speech, converting the signal from the time domain dimension to the frame dimension; using Fourier transform Transform to obtain the full spectrum of the signal; calculate the proportion of spectrum energy of the sub-band where the human voice is located frame by frame; generate a speech shielding mask to estimate the noise; divide voiced speech, unvoiced speech and noise according to the adjustment factor; classify speech voiced, unvoiced speech, The noise is weighted after spectral subtraction. The present invention is a speech denoising method based on improved spectral subtraction, which can effectively suppress stationary and non-stationary noise in real time, significantly improve the speech signal-to-noise ratio, and at the same time ensure that the effective part of the speech signal is not subject to distortion caused by front-end denoising. Impact, suitable for front-end denoising of the system under various speech tasks, improving system processing speed and performance.

Description

A speech denoising method based on improved spectral subtraction

Technical field

The invention belongs to the technical field of speech enhancement, and specifically relates to a speech denoising method based on improved spectral subtraction.

Background technique

Speech denoising is deployed at the front end of the speech system to suppress the degradation distortion of speech signals under noise and improve the auditory quality and understandability of speech signals to solve certain tasks such as speech recognition, voiceprint recognition, scene recording and listening assistance. In extreme scenarios, common problems include serious noise spectrum pollution and low voice quality, which in turn affects the performance of back-end tasks. Therefore, the noise problem is the focus of the field, and many speech denoising methods have been proposed and applied.

Although the existing single-channel and microphone array-based signal processing denoising methods and deep learning denoising methods have solved the noise problem in some scenes, they still have some technical deficiencies, which limit their practical application. These shortcomings can be summarized as follows:

(1) In single-channel denoising methods, for example, Wiener filtering requires that the expected pure speech is known, wavelet transform has a limited scope of application and the denoising effect is generally poor, and denoising methods based on statistics and subspaces are computationally intensive. Not suitable for real-time tasks;

(2) Denoising methods based on microphone arrays have certain practical deployment problems. The cost of deploying multiple microphones is high, and the beam noise suppression formed under the microphone array configuration in some extreme scenarios is average or even unfeasible;

(3) Deep learning denoising methods are mostly data-driven. The denoising effect depends on the quality and scale of the training data, and the number of parameters is huge. There are certain thresholds for actual deployment and limited applications.

Therefore, there is an urgent need for a speech denoising method with strong applicability, small computational load, high real-time performance, and low deployment cost.

Contents of the invention

In view of this, the object of the present invention is to provide a speech denoising method based on improved spectral subtraction. The present invention aims to solve the problems of existing speech denoising methods for tasks such as speech recognition, voiceprint recognition, scene recording, and hearing assistance, which have small scope of application, large amount of calculation, poor real-time performance, and high deployment cost.

In order to achieve the above objectives, the present invention provides a speech denoising method based on improved spectral subtraction, which includes the following steps:

S1. Input the noisy speech, preprocess the noisy speech signal, and convert the signal from the time domain dimension to the frame dimension;

S2. Use Fourier transform to obtain the full spectrum of the signal;

S3. Calculate the spectral energy proportion of the sub-band where the human voice is located frame by frame;

S4. Generate a speech shielding mask and estimate the noise;

S5. Classify voiced speech, unvoiced speech and noise according to the adjustment factor;

S6. Perform weight correction after spectral subtraction for voiced speech, unvoiced speech, and noise.

Further, in step S1, the preprocessing operations include pre-emphasis, framing and windowing.

Furthermore, in step S1, the Hanning window function is used for windowing.

Further, step S3 includes the following sub-steps:

S3.1 divides the sub-bands in the frequency domain where the human voice energy is concentrated. The frequency range of the sub-bands is: 300 to 3400Hz;

S3.2 Statistics of the spectral energy E _sub of the sub-band where the human voice is located and the spectral energy E _all of the full frequency domain of the signal;

S3.3 Calculate the spectrum energy proportion R _en of the sub-band. The calculation expression is as follows:

Further, in step S3.2, the spectral energy E _sub of the sub-band where the human voice is located is calculated as follows:

In the formula, f _start and f _end are both cutoff frequencies, f _start =300Hz, f _end =3400Hz; S is the amplitude corresponding to frequency f.

Further, the step S4 includes the following sub-steps:

S4.1 Set the energy proportion threshold T _en , and compare the energy proportion threshold T _en to generate the frame-dimensional speech segment mask Mask _n . The calculation expression of Mask _n is as follows:

S4.2 The speech mask is multiplied by the spectrum energy to obtain the noise spectrum estimate.

Further, in step S4.1, the energy proportion threshold T _en is 0.6.

Further, the step S5 includes the following sub-steps:

Calculate the two frequency domain characteristics of the average amplitude and zero-crossing rate frame by frame for the entire S5.1 signal, and take the ratio as the adjustment factor of the suppression function, as follows:

In the formula, AA represents the average amplitude of the audio signal; ZCR represents the short-time zero-crossing rate; F represents the adjustment factor of the suppression function;

S5.2 divide the threshold values T ₁ and T ₂ of noise, voiced sound and unvoiced sound according to the adjustment factor, as follows:

T ₁ =α ₁ *max(F)+α ₂ *min(F)

T ₂ =β ₁ *max(F)+β ₂ *min(F)

In the formula, α ₁ and α ₂ represent the regulatory factors corresponding to T ₁ , α ₁ + α ₂ = 1; β ₁ and β ₂ represent the regulatory factors corresponding to T ₂ , β ₁ + β ₂ = 1;

F<T ₁ is classified as noise; T ₁ <F <T ₂ is classified as voiceless sound; T ₂ <F is classified as voiced sound.

Further, the specific steps of step S6 are as follows:

Set the spectrum weight of the signal after spectral subtraction, the correction weight of the noise segment is w ₁ , the correction weight of the unvoiced speech segment is w ₂ , and the correction weight of the voiced segment is w ₃ , and then perform the weight correction of the signal after spectral subtraction to obtain the denoised The signal spectrum is as follows:

In the formula, is the signal spectrum after denoising;/> is the signal spectrum of the noise section;/> is the signal spectrum of the unvoiced segment of speech;/> is the signal spectrum of the voiced segment.

Further, in step S6, the correction weight w ₁ of the noise segment is 0.6, the correction weight w ₂ of the unvoiced speech segment is 0.8, and the correction weight w ₃ of the voiced segment is 1.1

The beneficial effects of the present invention are:

The invention discloses a speech denoising method based on improved spectral subtraction. Aiming at noise problems in tasks such as speech recognition, voiceprint recognition, scene recording and hearing assistance, it can suppress stationary and non-stationary noise in real time and effectively, and significantly It improves the speech signal-to-noise ratio and at the same time ensures that the effective part of the speech signal is not affected by distortion caused by front-end denoising. It is suitable for front-end denoising of the system under various speech tasks and improves system processing speed and performance.

Other advantages, objects, and features of the invention will, to the extent that they are set forth in the description that follows, and, to the extent that they will become apparent, or can be apparent to those skilled in the art upon examination of the following, Teach this invention by practicing it. The objects and other advantages of the invention may be realized and obtained by the following description.

Description of the drawings

Figure 1 is a flow chart of the speech denoising method of the present invention;

Figure 2 is a schematic diagram of the sub-band area where the human voice is located;

Figure 3 is a schematic diagram of the relationship between average amplitude and zero-crossing rate and speech.

Detailed ways

In order to make the technical solutions, advantages and purposes of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of this application.

As shown in Figure 1, the present invention provides a speech denoising method based on improved spectral subtraction, which includes the following steps:

S1. Input the noisy speech, preprocess the noisy speech signal, and convert the signal from the time domain dimension to the frame dimension;

The noisy signal to be processed used in the present invention can be human voice data under various speech tasks such as speech recognition, voiceprint recognition, scene recording, and listening assistance. In the embodiment of the present invention, the selected data set is the corpus CN-Celeb, The data set has 600,000 utterances from 3,000 speakers, and was recorded without genre restrictions. It has real noise in a variety of scenes, simulates speech denoising under the voiceprint recognition task, and is processed in .wav format.

The speech data in the time domain form is sequentially subjected to signal preprocessing by pre-emphasis, framing, and window functions to convert the signal from the time domain dimension to the frame dimension. The frame length win_length and frame offset hop_length are set to 2048 and 512 respectively. This paper The embodiment uses the Hanning window function.

S2. Use Fourier transform to obtain the full spectrum of the signal;

Use Fourier transform and set the Fourier transform size NFFT to 2048 to obtain the full spectrum of the signal.

S3. Calculate the spectral energy proportion of the sub-band where the human voice is located frame by frame;

According to the acoustic experimental conclusion that the energy proportion of the speech segment of the signal in the low-frequency sub-band is higher than that of the non-speech segment, specific sub-bands where the human voice energy is concentrated are divided in the frequency domain, and the spectral energy of each frame of the signal is counted, and the signal The ratio is calculated by summing the spectral energy over the entire frequency domain.

S3.1 divides the sub-bands where vocal energy is concentrated in the frequency domain;

According to the conclusions of acoustic experiments, human voice energy is concentrated in the frequency range from 300 to 3400Hz as shown in Figure 2, so the frequency range of the sub-band is 300 to 3400Hz.

S3.2 Statistics of the spectral energy E _sub of the sub-band where the human voice is located and the spectral energy E _all of the full frequency domain of the signal;

The amplitude of the signal spectrum is squared to obtain the energy of each frame of the original signal, and the original phase of the signal is retained. Call the numpy.fft.fftfreq function of the numpy scientific computing library to adaptively set the center frequencies of multiple frequency intervals at equal intervals for the signal, and count the spectrum energy of the entire signal segment at these frequencies.

The spectral energy E _sub of the sub-band where the human voice is located, the calculation expression is as follows:

In the formula, f _start and f _end are both cutoff frequencies, f _start =300Hz, f _end =3400Hz; S is the amplitude corresponding to frequency f;

S3.3 Calculate the spectrum energy proportion R _en of the sub-band. The calculation expression is as follows:

S4. Generate a speech shielding mask and estimate the noise;

S4.1 Set the energy proportion threshold T _en , T _en =0.6, and compare the energy proportion threshold T _en to generate the frame-dimensional speech segment mask Mask _n . The calculation expression of Mask _n is as follows:

S4.2 The speech mask is multiplied by the spectrum energy to obtain the noise spectrum estimate.

S5. Classify voiced speech, unvoiced speech and noise according to the adjustment factor;

For the case of sudden non-stationary noise, the suppression function under the full range of fixed factor values is not reasonable enough, and a certain attribute of the signal needs to be dynamically calculated to adjust the suppression function. According to the high zero-crossing rate of noise segments and unvoiced speech, the average amplitude of noise with excitation sources is greater than that of unvoiced segments, the energy of voiced speech is large and the vocal fold excitation is obvious, and the ratio is greater than the changing relationship between unvoiced and noise segments, the average amplitude and zero-crossing are calculated frame by frame The ratio of the two frequency domain characteristics is used as the adjustment factor of the suppression function, and the judgment threshold value for dividing noise, voiced sound and unvoiced sound is set according to the ratio.

The frequency domain characteristics of the average amplitude and zero-crossing rate are calculated frame by frame for the entire S5.1 signal, as shown in Figure 3. The ratio is used as the adjustment factor of the suppression function, as follows:

In the formula, AA represents the average amplitude of the audio signal; ZCR represents the short-time zero-crossing rate; F represents the adjustment factor of the suppression function;

S5.2 divide the threshold values T ₁ and T ₂ of noise, voiced sound and unvoiced sound according to the adjustment factor, as follows:

T ₁ =α ₁ *max(F)+α ₂ *min(F)

T ₂ =β ₁ *max(F)+β ₂ *min(F)

In the formula, α ₁ and α ₂ represent the regulatory factors corresponding to T ₁ , α ₁ + α ₂ = 1; β ₁ and β ₂ represent the regulatory factors corresponding to T ₂ , β ₁ + β ₂ = 1;

F<T ₁ is classified as noise; T ₁ <F <T ₂ is classified as voiceless sound; T ₂ <F is classified as voiced sound.

S6. Perform weight correction after spectral subtraction for voiced speech, unvoiced speech, and noise.

Shrink the noise segment as much as possible to suppress and reduce the residual to eliminate music noise. The suppression of the unvoiced segment of speech is relatively weakened, keeping the voiced segment stable when reconstructing the signal. For the voiced segment of speech, it is necessary to increase the gain to further reduce the spectrum, and set the spectrum After subtracting the corrected weights of the signal noise segment, unvoiced segment and voiced segment, the signal spectrum after denoising is obtained.

Set the spectrum weight of the signal after spectrum subtraction. The correction weight of the noise segment is w ₁ , w ₁ =0.6; the correction weight of the unvoiced speech segment is w ₂ , w ₂ =0.8; the correction weight of the voiced segment is w ₃ , w ₃ =1.1 ; Then perform weight correction on the signal after spectrum subtraction to obtain the denoised signal spectrum, as follows:

In the formula, is the signal spectrum after denoising;/> is the signal spectrum of the noise section;/> is the signal spectrum of the unvoiced segment of speech;/> is the signal spectrum of the voiced segment.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified. Modifications or equivalent substitutions without departing from the purpose and scope of the technical solution shall be included in the protection scope of the present invention.

Claims (10)

1. A speech denoising method based on improved spectral subtraction, characterized in that it includes the following steps: S1. Input the noisy speech, preprocess the noisy speech signal, and convert the signal from the time domain dimension to the frame dimension; S2. Use Fourier transform to obtain the full spectrum of the signal; S3. Calculate the spectral energy proportion of the sub-band where the human voice is located frame by frame; S4. Generate a speech shielding mask and estimate the noise; S5. Classify voiced speech, unvoiced speech and noise according to the adjustment factors; S6. Perform weight correction after spectral subtraction for voiced speech, unvoiced speech, and noise. 2. A speech denoising method based on improved spectral subtraction according to claim 1, characterized in that: in step S1, the preprocessing operations include pre-emphasis, framing and windowing. 3. A speech denoising method based on improved spectral subtraction according to claim 2, characterized in that in step S1, a Hanning window function is used for windowing. 4. A speech denoising method based on improved spectral subtraction according to claim 1, characterized in that the step S3 includes the following sub-steps: S3.1 divides the sub-bands in the frequency domain where the human voice energy is concentrated. The frequency range of the sub-bands is: 300 to 3400Hz; S3.2 Statistics of the spectral energy E _sub of the sub-band where the human voice is located and the spectral energy E _all of the full frequency domain of the signal; S3.3 Calculate the spectrum energy proportion R _en of the sub-band. The calculation expression is as follows: 5. A speech denoising method based on improved spectral subtraction according to claim 4, characterized in that: in the step S3.2, the spectral energy E _sub of the sub-band where the human voice is located has the following calculation expression: In the formula, f _start and f _end are both cutoff frequencies, f _start =300Hz, f _end =3400Hz; S is the amplitude corresponding to frequency f. 6. A speech denoising method based on improved spectral subtraction according to claim 5, characterized in that the step S4 includes the following sub-steps: S4.1 Set the energy proportion threshold T _en , and compare the energy proportion threshold T _en to generate the frame-dimensional speech segment mask Mask _n . The calculation expression of Mask _n is as follows: S4.2 The speech mask is multiplied by the spectrum energy to obtain the noise spectrum estimate. 7. A speech denoising method based on improved spectral subtraction according to claim 6, characterized in that: in step S4.1, the energy proportion threshold T _en is 0.6. 8. A speech denoising method based on improved spectral subtraction according to claim 7, characterized in that the step S5 includes the following sub-steps: Calculate the two frequency domain characteristics of the average amplitude and zero-crossing rate frame by frame for the entire S5.1 signal, and take the ratio as the adjustment factor of the suppression function, as follows: In the formula, AA represents the average amplitude of the audio signal; ZCR represents the short-time zero-crossing rate; F represents the adjustment factor of the suppression function; S5.2 divide the threshold values T ₁ and T ₂ of noise, voiced sound and unvoiced sound according to the adjustment factor, as follows: T ₁ =α ₁ *max(F)+α ₂ *min(F) T ₂ =β ₁ *max(F)+β ₂ *min(F) In the formula, α ₁ and α ₂ represent the regulatory factors corresponding to T ₁ , α ₁ + α ₂ = 1; β ₁ and β ₂ represent the regulatory factors corresponding to T ₂ , β ₁ + β ₂ = 1; F<T ₁ is classified as noise; T ₁ <F <T ₂ is classified as voiceless sound; T ₂ <F is classified as voiced sound. 9. A speech denoising method based on improved spectral subtraction according to claim 8, characterized in that the specific steps of step S6 are as follows: Set the spectrum weight of the signal after spectral subtraction, the correction weight of the noise segment is w ₁ , the correction weight of the unvoiced speech segment is w ₂ , and the correction weight of the voiced segment is w ₃ , and then perform the weight correction of the signal after spectral subtraction to obtain the denoised The signal spectrum is as follows: In the formula, is the signal spectrum after denoising;/> is the signal spectrum of the noise section;/> is the signal spectrum of the unvoiced segment of speech;/> is the signal spectrum of the voiced segment. 10. A speech denoising method based on improved spectral subtraction according to claim 9, characterized in that in step S6, the correction weight w ₁ of the noise segment is 0.6, and the correction weight w ₂ of the unvoiced speech segment is 0.8, and the modified weight w ₃ of the voiced segment is 1.1.