CN110600038A - Audio fingerprint dimension reduction method based on discrete kini coefficient - Google Patents

Abstract

The invention relates to an audio fingerprint dimensionality reduction method based on discrete Gini coefficient calculation, which aims to solve the problem of high dimensionality of audio fingerprint features. features for dimensionality reduction. The present invention introduces the discrete Gini coefficient of the fingerprint into each dimension of the audio fingerprint, and the size of the discrete Gini coefficient of each dimension of the audio fingerprint reflects the distinguishability of different audio in this dimension, and deletes the dimension with the large discrete Gini coefficient by retaining the dimension with the small discrete Gini coefficient Dimensionality achieves the purpose of dimensionality reduction. The sample audio fingerprint library constructed by dimensionality-reduced audio fingerprint features has smaller data volume and higher utilization rate.

Description

A Dimensionality Reduction Method for Audio Fingerprint Based on Discrete Gini Coefficient

technical field

The invention belongs to the field of intelligent application-type sound fields, and in particular relates to an audio fingerprint dimensionality reduction method based on discrete Gini coefficient calculation.

Background technique

In recent years, intelligence has been deeply loved by people, and it has been widely studied and discussed by people. The intelligent recognition of audio is an important cornerstone of the development of artificial intelligence. The intelligent recognition of audio is inseparable from the feature extraction of audio. Many features of audio Among them, audio fingerprint is the most popular one in recent years. Audio fingerprint refers to a content-based compact digital signature that can represent important acoustic features of a piece of audio. Its main purpose is to represent a large amount of audio data with a small amount of digital information. Compared with traditional audio features, it has the advantages of small data volume, high anti-noise performance, and relatively simple feature extraction process. It is widely used in music recognition, advertising supervision, copyright protection and other fields. However, one disadvantage of audio fingerprints is that the fingerprints have a high dimension, which slows down the recognition speed in audio recognition and takes up a lot of computer memory. In this regard, if the dimensionality of audio fingerprints can be reduced, the data volume of audio fingerprints can be reduced to a large extent, and at the same time, the speed of audio retrieval can be improved, and the performance of audio recognition can be enhanced.

Contents of the invention

Aiming at the high-dimensional problem of audio fingerprint features, the present invention introduces fingerprint discrete Gini coefficients in each dimension of audio fingerprints, and the size of discrete Gini coefficients in each dimension of audio fingerprints reflects the distinguishability of different audio in this dimension. The larger the discrete Gini coefficient of a certain bit of the audio fingerprint, the greater the difference between different audios in this bit, indicating that the better the discrimination of the bit, otherwise the poorer the discrimination. Therefore, retaining the bits with good discrimination and removing the bits with poor discrimination can convert high-dimensional audio fingerprints into lower dimensions, thereby effectively reducing the amount of fingerprint data.

The technical solution of the present invention is used to solve the problem of excessive data volume of the audio fingerprint feature database. The sample audio fingerprint database constructed by the dimensionality-reduced audio fingerprint feature has a smaller data volume and higher utilization rate. The technology of the present invention is mainly divided into the following Several steps:

Step 1, classify and build the target sound library

This design classifies the audio and builds a database according to the types of audio characteristics or existing data. Since different types of audio have different features, it is easy to find the commonality of audio features by classifying audio. If audio classification is not performed, the dimensionality reduction quality of audio fingerprints will be affected. The existing audio data is classified and stored, and then fingerprint features are extracted for each type of audio. The flowchart of audio fingerprint extraction algorithm is shown in Figure 1.

Step 2, extract the fingerprint features of the sample audio

Select various audio data from the built audio library as the original sample audio. The original sample fingerprint features are extracted and the discrete Gini coefficient is introduced to reduce the dimensionality of the fingerprint features. The specific process is:

Step2.1: Preprocessing the target audio

Before data feature extraction, preprocessing operation is done first. Preprocessing includes: bandpass filtering, pre-emphasis, and framing.

(1) The 8kHz sampled audio signal is selected as the processing object for band-pass filter processing to extract the most important frequency components perceived by the human ear, and a band-pass filter with a pass band range of 20 Hz-4000 Hz is selected to process the signal. In this design, the bandpass filter uses a finite impulse response (Finite Impulse Filter, FIR) filter, and the filtering process is:

Among them, T is the number of sampling points of the processed signal, p is the time domain label, h(l) is the coefficient of the FIR filter, x(p) is the input signal, and y(p) is the signal after bandpass filtering.

(2) Perform pre-emphasis processing on the band-pass filtered signal y(p). This design uses a 6dB/octave digital filter to improve the high-frequency characteristics of the pre-processed signal, making the signal spectrum become It is relatively flat, and at the same time, the speech signal can use the same signal-to-noise ratio to calculate the spectrum in the entire frequency band from low frequency to high frequency. The pre-emphasis processing is shown in the following formula:

s(p)=y(p)-μ*y(p-1)

Among them, μ is the pre-emphasis coefficient, and its value is 0.96, and s(p) is the signal after pre-emphasis processing.

(3) Perform windowing and framing processing on the pre-emphasis signal. Although the method of continuous segmentation can be used for framing, the method of overlapping segments is generally used, which is to make the transition between frames smooth and maintain their continuity. The audio is divided into frames with a frame length of 0.064 seconds, and an overlap rate of 75% is maintained between frames. Each frame is weighted with a Hanning window of the same length. The windowing formula is as follows:

Among them, T is the length of the Hanning window and also the frame length of one frame of audio.

Step2.2: Extract fingerprint features from audio data

A digital audio fingerprint can be regarded as the condensed essence of a piece of audio, which contains the most important part of the audio data, and the dimension of the audio fingerprint is a key factor affecting the amount of fingerprint data and the retrieval rate, so this technology refers to discrete Gini The coefficient performs feature dimensionality reduction on audio fingerprints. Before dimensionality reduction, the audio fingerprint features must be extracted first. The steps are as follows:

(1) Discrete Fourier transform is carried out to framed audio signal, and each frame data of audio signal is carried out discrete Fourier transform, and transformation formula is as follows:

Among them, X(k) is the frequency domain signal, x(p) is the time domain signal, k is the frequency index, and T is the sample length of the discrete Fourier transform.

(2) The frequency domain signal after discrete Fourier transform is divided into spectrum subbands, and 33 non-overlapping frequency bands are selected from the spectrum, distributed in the range of 20-4000Hz (the human ear's identification of audio is mainly concentrated in this range Within), the frequency bands are equally logarithmically spaced (according to the human ear's response to different frequencies is not linear). The start frequency of the mth subband, that is, the stop frequency f(m) of the m-1th subband can be expressed as the following formula:

Among them, Fmin is the lower limit of mapping, which is 20Hz here, Fmax is the upper limit of mapping, which is 4000Hz here, and M is the number of subbands, which is 33 here.

(3) Calculate the energy of each subband of each frame of audio, and calculate the energy of the 33 non-overlapping frequency bands selected above, assuming that the start frequency of the mth subband is f(m), and the end frequency is f(m+1) , the frequency-domain signal after discrete Fourier transform is x(k), then the formula for the energy of the mth subband of the nth frame is as follows:

(4) Generate the sub-fingerprint of each frame of audio, and perform bit difference discrimination on the 33 sub-band energies required for each frame above, generate 32-bit binary codes (sub-fingerprints) of each frame of audio, the mth sub-band energy of the nth frame is E(n,m), and its corresponding binary bit information is F(n,m), then the binary audio fingerprint information discrimination formula of each frame is as follows:

It can be seen from the above formula that a 32-dimensional binary sub-fingerprint information is finally generated for each frame of audio, and the sub-fingerprint contains less information, and an audio fingerprint feature often consists of multiple sub-fingerprints.

Step 3, Dimensionality reduction of audio fingerprint features

After the above-mentioned audio fingerprint extraction process, each frame of audio data finally generates a 32-dimensional binary sub-fingerprint information. For a piece of audio, the audio fingerprint information contained is composed of multiple binary sub-fingerprint information, and the amount of fingerprint information data is still large. In practical applications, it is hoped to further reduce the dimensionality of the audio fingerprint so as to effectively reduce the amount of fingerprint data. . This design proposes an audio fingerprint dimension reduction method based on discrete Gini coefficient calculation. This method calculates the discrete Gini coefficient for each dimension of the audio fingerprint, and achieves the purpose of reducing the fingerprint dimension by the size of the discrete Gini coefficient of each dimension of the fingerprint.

Calculate the discrete Gini coefficients of each dimension of fingerprints by dividing the data of each dimension of the audio fingerprint into a group of 50 frames, because the discrete Gini coefficients of the fingerprints of each dimension reflect the degree of discreteness of the data of a certain dimension of the audio fingerprint, that is, the difference of the data of a certain dimension of the audio fingerprint The greater the discrete Gini coefficient of a certain bit of the audio fingerprint, the greater the difference between different audios in this bit, indicating that the better the discrimination of the bit, otherwise the poorer the discrimination. In this design, the 32-dimensional audio fingerprint is converted to a lower dimension to reduce the amount of fingerprint data by retaining the more distinguishable dimension information in the audio fingerprint and removing the less distinguishable dimension information.

Step3.1: Calculate the discrete Gini coefficient of each dimension of the audio fingerprint

The derivation process and specific steps are as follows:

(1) Calculate the discrete Lorenz curve of the audio fingerprint, the discrete Lorenz curve is the key curve for calculating the discrete Gini coefficient, which is calculated by the proportion vector of the accumulated fingerprint data The components of each element, j represents the dimension number of the audio fingerprint, and the value range j=1,2,...,32 to find the proportion vector of the accumulated fingerprint data The calculation process is as follows:

All kinds of audio fingerprints in the audio fingerprint library are processed by frame, and every 50 frames of audio fingerprint data are divided into N groups, and the j-th dimension cumulative fingerprint data vector is constructed:

in is the group number and has Construct the j-th dimension cumulative fingerprint data proportion vector Each element in the vector is defined as:

………

The curve formed by each element of the proportion vector is a discrete Lorenz curve. Specifically, the process of drawing the discrete Lorenz curve of the j-th dimension of the audio fingerprint is as follows:

The abscissa takes the proportion of the cumulative number of audio fingerprints as the abscissa (the abscissa is ), taking the proportion of accumulated fingerprint data in the j-dimension of the audio fingerprint as the vertical coordinate, and the proportion of the accumulated fingerprint data in the j-dimension of the audio fingerprint Each element of the discrete point is drawn, and the discrete curve linked by the discrete point is the discrete Lorenz curve of the j-th dimension of the audio fingerprint, and its coordinates are

(2) Calculate the discrete Gini coefficient of each dimension of various audio fingerprints, and use the discrete Lorenz curve obtained above as the dividing line, the formula of the Gini coefficient of the jth dimension of the audio fingerprint can be obtained as follows:

Among them, S _a is the closed area enclosed by the coordinate diagonal line segment OA and the discrete Lorenz curve, the coordinates of point O are (0,0) and the coordinates (1,1) of point A, and S _b is the coordinate line segment OB, BA The closed area surrounded by the discrete Lorenz curve, the coordinates of point B is (1,0), G ^j is the Gini coefficient of the j-th dimension of the audio fingerprint, and the auxiliary figure for the calculation of the discrete Gini coefficient of the fingerprint is shown in Figure 4.

It can be seen from the above that the sum of S _a + S _b is the closed area enclosed by the diagonal line segment OA and the line segments OB and BA, that is: S _a + S _b = 1/2, because the audio fingerprint is discrete, so this design Discretize the above formula as:

Thus, the j-th dimension discrete Gini coefficient of the audio fingerprint is obtained where i is the group number, Accumulate the i-th group of fingerprint data proportions for the j-th dimension of the audio fingerprint.

Step3.2: Count the discrete Gini coefficients of different types of audio fingerprints in each dimension to implement fingerprint dimensionality reduction

Combined with the application scenario, the discrete Gini coefficient training of each dimension of the audio fingerprint is carried out. The training set is constructed from the audio data according to the data type or combined with the existing data set, and the discrete Gini coefficient of each dimension of the audio fingerprint is obtained for the audio data of each training set and statistical analysis is performed.

The audio fingerprint discrete Gini coefficient designed by the present invention is suitable for dimensionality reduction of various audio fingerprints. This time, the following types of audio are selected as analysis objects, and the discrete Gini coefficient of each dimension of the audio fingerprint is statistically analyzed:

As shown in Figure 5. The discrete Gini coefficients of each dimension of the audio fingerprint of the abnormal sound. The discrete Gini coefficients of the 32 dimensions of the audio fingerprint of the abnormal sound are calculated from the audio fingerprint database. By setting the threshold of the audio fingerprint discrete Gini coefficient, it can be discarded. Audio fingerprint dimension information lower than the threshold, keep the audio fingerprint dimension information higher than the threshold, the discrete Gini coefficient threshold range of abnormal sound audio fingerprints is 0.36~0.38, as can be seen from the figure, the discrete Gini coefficient of each dimension of the audio fingerprint The sizes are different. The discrete Gini coefficient of fingerprints in some dimensions is small, such as the 2nd, 25th, and 26th dimensions, and the discrete Gini coefficient of fingerprints in some dimensions is relatively large, such as the 11th, 20th, and 28th dimensions. The size represents the difference of the audio fingerprint data in this dimension. At this time, if the threshold value of the discrete Gini coefficient of the fingerprint is set to 0.37, it can be seen from the figure that the dimensions of the fingerprint discrete Gini coefficient less than 0.37 have the 2nd, 5th, 24th, 25th, and 26th dimensions, indicating that each of the five dimensions The frame fingerprint data has small differences and can be discarded. By discarding these dimensions with small differences, the dimensions of each frame of audio fingerprints are reduced, thereby reducing the data volume of the audio fingerprint database.

Figure 6. It is the discrete Gini coefficient of each dimension of the audio fingerprint of the normal speech class. The discrete Gini coefficient of the 32 dimensions of the audio fingerprint of the normal speech class is calculated from the audio fingerprint database. By setting the threshold value of the discrete Gini coefficient of the audio fingerprint, it can be discarded. The audio fingerprint dimension information below the threshold is removed, and the audio fingerprint dimension information above the threshold is retained. The discrete Gini coefficient threshold range of normal speech audio fingerprints is 0.36 to 0.38. It can be seen from the figure that the discrete Gini coefficient of such audio fingerprints is relatively low. Small dimensions include the 2nd, 22nd, and 25th dimensions, and dimensions with larger discrete Gini coefficients include the 18th, 26th, and 28th dimensions, etc. If the threshold for such audio fingerprint discrete Gini coefficients is set to 0.37, it can be seen from the figure The dimensions of the fingerprint discrete Gini coefficient less than 0.37 have the 2nd, 22nd, and 25th dimensions, indicating that the differences in the fingerprint data of each frame in these 3 dimensions are small and can be discarded.

Figure 7. The discrete Gini coefficients of each dimension of the audio fingerprint of the song class. The discrete Gini coefficients of the 32 dimensions of the audio fingerprint of the song class are calculated from the audio fingerprint library. By setting the threshold value of the discrete Gini coefficient of the audio fingerprint, it can be discarded. The threshold audio fingerprint dimension information, keep the audio fingerprint dimension information higher than the threshold, the discrete Gini coefficient threshold range of song audio fingerprint is 0.36~0.38, from the figure we can see the difference of discrete Gini coefficient of each dimension of this type of audio fingerprint Relatively large, the discrete Gini coefficients of each dimension of the fingerprint are relatively scattered. The discrete Gini coefficients of the 2nd, 14th, 15th, and 25th dimensions of the fingerprint are relatively small, and the discrete Gini coefficients of the 4th, 5th, 11th, 18th, and 29th dimensions of the fingerprint are relatively large. The coefficient is large. At this time, if the discrete Gini coefficient threshold of this type of audio fingerprint is set to 0.37, it can be seen from the figure that the discrete Gini coefficient of the fingerprint is less than 0.37. There are 1, 2, 14, 15, 25, and 26 dimensions It shows that the difference of the fingerprint data of each frame in these 6 dimensions is small and can be discarded.

From these figures, it can be seen that although the discrete Gini coefficients of various audio fingerprints are different, and the trend of the coefficients in each dimension is also different, there are always several dimensions of fingerprints with small Gini coefficients, which shows that audio fingerprints are in these dimensions. The discriminativeness of each dimension is small, so the data of these dimensions can be removed, and the 32-dimensional audio fingerprint can be converted into a lower dimension to effectively reduce the amount of fingerprint data.

The advantages of the present invention are:

1. Algorithm complexity is low and flexibility is stronger

2. Smaller than traditional audio feature data

3. Introduce the discrete Gini coefficient as a dimensionality reduction algorithm, which has low complexity and can handle large batches of data

4. The audio fingerprint after dimensionality reduction is more robust

Description of drawings

Figure 1. Flowchart of Audio Fingerprint Extraction Algorithm

Figure 2. 32-dimensional audio fingerprint block diagram of a piece of audio

Figure 3. The Lorenz curve of the first dimension of audio fingerprints of abnormal sounds

Figure 4. Auxiliary diagram for calculating the discrete Gini coefficient of audio fingerprints

Figure 5. Discrete Gini coefficient diagram of each dimension of abnormal sound audio fingerprint

Figure 6. Discrete Gini coefficient map of each dimension of normal speech audio fingerprints

Figure 7. Discrete Gini coefficient map of each dimension of song audio fingerprint

Detailed ways

The invention provides an audio fingerprint dimensionality reduction method based on discrete Gini coefficient calculation. The method includes classifying and constructing a target sound library, extracting fingerprint features of sample audio, and introducing discrete Gini coefficient to reduce dimensionality of audio fingerprint features.

The technical solution of the present invention is used to solve the problem of excessive data volume of the audio fingerprint feature database. The sample audio fingerprint database constructed by the dimensionality-reduced audio fingerprint feature has a smaller data volume and higher utilization rate. The technology of the present invention is mainly divided into the following Several steps:

Step 1, classify and build the target sound library

This design classifies the audio and builds a database according to the types of audio characteristics or existing data. Since different types of audio have different features, it is easy to find the commonality of audio features by classifying audio. If audio classification is not performed, the dimensionality reduction quality of audio fingerprints will be affected.

The discrete Gini coefficient of audio fingerprints designed by the present invention is suitable for dimensionality reduction of various audio fingerprints. Therefore, the existing audio data should be classified and stored, and then the fingerprint features of various audios should be extracted respectively. The flowchart of audio fingerprint extraction algorithm is shown in Figure 1.

Step 2, extract the fingerprint features of the sample audio

Select various audio data from the built audio library as the original sample audio. The original sample fingerprint features are extracted and the discrete Gini coefficient is introduced to reduce the dimensionality of the fingerprint features. The specific process is:

Step2.1: Preprocessing the target audio

Before data feature extraction, preprocessing operation is done first. Preprocessing includes: bandpass filtering, pre-emphasis, and framing.

(1) The 8kHz sampling audio signal is selected as the processing object for band-pass filter processing. In order to extract the most important frequency components perceived by the human ear, a band-pass filter with a pass band range of 20 Hz-4000 Hz is selected to process the signal. In this design, the bandpass filter uses a finite impulse response (Finite Impulse Filter, FIR) filter, and the filtering process is:

Among them, T is the number of sampling points of the processed signal, p is the time domain label, h(l) is the coefficient of the FIR filter, x(p) is the input signal, and y(p) is the signal after bandpass filtering.

(2) Perform pre-emphasis processing on the band-pass filtered signal y(p). This design uses a 6dB/octave digital filter to improve the high-frequency characteristics of the pre-processed signal, making the signal spectrum become It is relatively flat, and at the same time, the speech signal can use the same signal-to-noise ratio to calculate the spectrum in the entire frequency band from low frequency to high frequency. The pre-emphasis processing is shown in the following formula:

s(p)=y(p)-μ*y(p-1)

Among them, μ is the pre-emphasis coefficient, and its value is 0.96, and s(p) is the signal after pre-emphasis processing.

(3) Perform windowing and framing processing on the pre-emphasis signal. Although the method of continuous segmentation can be used for framing, the method of overlapping segments is generally used, which is to make the transition between frames smooth and maintain their continuity. The audio is divided into frames with a frame length of 0.064 seconds, and an overlap rate of 75% is maintained between frames. Each frame is weighted with a Hanning window of the same length. The windowing formula is as follows:

Among them, T is the length of the Hanning window and also the frame length of one frame of audio.

Step2.2: Extract fingerprint features from audio data

A digital audio fingerprint can be regarded as the condensed essence of a piece of audio, which contains the most important part of the audio data, and the dimension of the audio fingerprint is a key factor affecting the amount of fingerprint data and the retrieval rate, so this technology refers to discrete Gini The coefficient performs feature dimensionality reduction on audio fingerprints. Before dimensionality reduction, the audio fingerprint features must be extracted first. The steps are as follows:

(1) Discrete Fourier transform is carried out to framed audio signal, and each frame data of audio signal is carried out discrete Fourier transform, and transformation formula is as follows:

Among them, X(k) is the frequency domain signal, x(p) is the time domain signal, k is the frequency index, and T is the sample length of the discrete Fourier transform.

(2) The frequency domain signal after discrete Fourier transform is divided into spectrum subbands, and 33 non-overlapping frequency bands are selected from the spectrum, distributed in the range of 20-4000Hz (the human ear's identification of audio is mainly concentrated in this range Within), the frequency bands are equally logarithmically spaced (according to the human ear's response to different frequencies is not linear). The start frequency of the mth subband, that is, the stop frequency f(m) of the m-1th subband can be expressed as the following formula:

Among them, Fmin is the lower limit of mapping, which is 20Hz here, Fmax is the upper limit of mapping, which is 4000Hz here, and M is the number of subbands, which is 33 here.

(3) Calculate the energy of each subband of each frame of audio, and calculate the energy of the 33 non-overlapping frequency bands selected above, assuming that the start frequency of the mth subband is f(m), and the end frequency is f(m+1) , the frequency-domain signal after discrete Fourier transform is x(k), then the formula for the energy of the mth subband of the nth frame is as follows:

(4) Generate the sub-fingerprint of each frame of audio, perform bit differential discrimination on the 33 sub-band energies required for each frame above, and generate the 32-bit binary code of each frame of audio, that is, the sub-fingerprint of each frame of audio, the nth frame of the first The m sub-band energy is E(n,m), and its corresponding binary bit information is F(n,m), then the binary audio fingerprint information discrimination formula of each frame is as follows:

It can be seen from the above formula that a 32-dimensional binary sub-fingerprint information is finally generated for each frame of audio, and the sub-fingerprint contains less information, and an audio fingerprint feature often consists of multiple sub-fingerprints.

In practical applications such as audio retrieval and audio recognition using audio fingerprints, the 32-dimensional binary sub-fingerprint generated by a frame of audio signal contains less information, and cannot accurately retrieve or identify the target audio. In practical applications, audio fingerprints are commonly used The block carries out audio retrieval or audio recognition, and the audio fingerprint block is composed of at least 256 audio sub-fingerprints, as shown in Figure 2. A 32-dimensional audio fingerprint block extracted from a section of audio.

Step 3, Dimensionality reduction of audio fingerprint features

After the above-mentioned audio fingerprint extraction process, each frame of audio data finally generates a 32-dimensional binary sub-fingerprint information. For a piece of audio, the audio fingerprint information contained is composed of multiple binary sub-fingerprint information, and the amount of fingerprint information data is still large. In practical applications, it is hoped to further reduce the dimensionality of the audio fingerprint so as to effectively reduce the amount of fingerprint data. . This design proposes an audio fingerprint dimension reduction method based on discrete Gini coefficient calculation. This method calculates the discrete Gini coefficient for each dimension of the audio fingerprint, and achieves the purpose of reducing the fingerprint dimension by the size of the discrete Gini coefficient of each dimension of the fingerprint.

The data of each dimension of the audio fingerprint is divided into groups of 50 frames, including but not limited to 50 frames, and the discrete Gini coefficients of the fingerprints of each dimension are calculated. Since the discrete Gini coefficients of the fingerprints of each dimension reflect the degree of discreteness of the data of a certain dimension of the audio fingerprint, it is also It is the difference in the data of a certain dimension of the audio fingerprint. The larger the discrete Gini coefficient of a certain digit of the audio fingerprint, the greater the difference between different audios in this digit, indicating that the discrimination of the digit is better, otherwise the discrimination is poor. In this design, the 32-dimensional audio fingerprint is converted to a lower dimension to reduce the amount of fingerprint data by retaining the more distinguishable dimension information in the audio fingerprint and removing the less distinguishable dimension information.

Step3.1: Calculate the discrete Gini coefficient of each dimension of the audio fingerprint

Specific steps are as follows:

(1) Calculate the discrete Lorenz curve of the audio fingerprint, the discrete Lorenz curve is the key curve for calculating the discrete Gini coefficient, which is calculated by the proportion vector of the accumulated fingerprint data Composition, j represents the dimension number of the audio fingerprint, and the value range j=1,2,...,32 to obtain the proportion vector of the accumulated fingerprint data The calculation process is as follows:

All kinds of audio fingerprints in the audio fingerprint library are processed by frame, and every 50 frames of audio fingerprint data are divided into N groups, and the j-th dimension cumulative fingerprint data vector is constructed:

in is the group number and has

Construct the j-th dimension cumulative fingerprint data proportion vector Each element in the vector is defined as:

………

The curve formed by each element of the proportion vector is a discrete Lorenz curve. Specifically, the process of drawing the discrete Lorenz curve of the j-th dimension of the audio fingerprint is as follows:

The abscissa takes the proportion of the cumulative number of audio fingerprints as the abscissa (the abscissa is ), taking the proportion of accumulated fingerprint data in the j-dimension of the audio fingerprint as the vertical coordinate, and the proportion of the accumulated fingerprint data in the j-dimension of the audio fingerprint Each element of the discrete point is drawn, and the discrete curve linked by the discrete point is the discrete Lorenz curve of the j-th dimension of the audio fingerprint, and its coordinates are For example, the Lorenz curve of the first dimension of the abnormal sound audio fingerprint is shown in Figure 3.

(2) Calculate the discrete Gini coefficient of each dimension of various audio fingerprints, and use the discrete Lorenz curve obtained above as the dividing line, the formula of the Gini coefficient of the jth dimension of the audio fingerprint can be obtained as follows:

Among them, S _a is the closed area enclosed by the coordinate diagonal line segment OA and the discrete Lorenz curve, the coordinates of point O are (0,0) and the coordinates (1,1) of point A, and S _b is the coordinate line segment OB, BA The closed area surrounded by the discrete Lorenz curve, the coordinates of point B is (1,0), G ^j is the Gini coefficient of the j-th dimension of the audio fingerprint, and the auxiliary figure for the calculation of the discrete Gini coefficient of the fingerprint is shown in Figure 4.

It can be seen from the above that the sum of S _a + S _b is the closed area enclosed by the diagonal line segment OA and the line segments OB and BA, that is: S _a + S _b = 1/2, because the audio fingerprint is discrete, so this design Discretize the above formula as:

Thus, the j-th dimension discrete Gini coefficient of the audio fingerprint is obtained where i is the group number, Accumulate the i-th group of fingerprint data proportions for the j-th dimension of the audio fingerprint.

Step3.2: Count the discrete Gini coefficients of different types of audio fingerprints in each dimension to implement fingerprint dimensionality reduction

Combined with the application scenario, the discrete Gini coefficient training of each dimension of the audio fingerprint is carried out. The training set is constructed from the audio data according to the data type or combined with the existing data set, and the discrete Gini coefficient of each dimension of the audio fingerprint is obtained for the audio data of each training set and statistical analysis is performed.

The discrete Gini coefficient designed by the present invention is suitable for dimensionality reduction of various audio fingerprints. This time, the following types of audio are selected as analysis objects, and the discrete Gini coefficients of each dimension of the audio fingerprint are statistically analyzed:

As shown in Figure 5. The discrete Gini coefficients of each dimension of the audio fingerprint of the abnormal sound. The discrete Gini coefficients of the 32 dimensions of the audio fingerprint of the abnormal sound are calculated from the audio fingerprint database. By setting the threshold of the audio fingerprint discrete Gini coefficient, it can be discarded. Audio fingerprint dimension information lower than the threshold, keep audio fingerprint dimension information higher than the threshold, the discrete Gini coefficient threshold range of abnormal sound audio fingerprints is 0.36-0.38, as can be seen from the figure, the discrete Gini coefficient of each dimension of the audio fingerprint The sizes are different. The discrete Gini coefficient of fingerprints in some dimensions is small, such as the 2nd, 25th, and 26th dimensions, and the discrete Gini coefficient of fingerprints in some dimensions is relatively large, such as the 11th, 20th, and 28th dimensions. The size represents the difference of the audio fingerprint data in this dimension. At this time, if the threshold value of the discrete Gini coefficient of the fingerprint is set to 0.37, it can be seen from the figure that the dimensions of the fingerprint discrete Gini coefficient less than 0.37 have the 2nd, 5th, 24th, 25th, and 26th dimensions, indicating that each of the five dimensions The frame fingerprint data has small differences and can be discarded. By discarding these dimensions with small differences, the dimensions of each frame of audio fingerprints are reduced, thereby reducing the data volume of the audio fingerprint database.

Figure 6. It is the discrete Gini coefficient of each dimension of the audio fingerprint of the normal speech class. The discrete Gini coefficient of the 32 dimensions of the audio fingerprint of the normal speech class is calculated from the audio fingerprint database. By setting the threshold value of the discrete Gini coefficient of the audio fingerprint, it can be discarded. The audio fingerprint dimension information below the threshold is removed, and the audio fingerprint dimension information above the threshold is retained. The discrete Gini coefficient threshold range of normal speech audio fingerprints is 0.36 to 0.38. It can be seen from the figure that the discrete Gini coefficient of such audio fingerprints is relatively low. Small dimensions include the 2nd, 22nd, and 25th dimensions, and dimensions with larger discrete Gini coefficients include the 18th, 26th, and 28th dimensions, etc. If the threshold of the discrete Gini coefficient of such audio fingerprints is set to 0.37, it can be seen from the figure The dimensions of the fingerprint discrete Gini coefficient less than 0.37 have the 2nd, 22nd, and 25th dimensions, indicating that the differences in the fingerprint data of each frame in these 3 dimensions are small and can be discarded.

Figure 7. The discrete Gini coefficients of each dimension of the audio fingerprint of the song class. The discrete Gini coefficients of the 32 dimensions of the audio fingerprint of the song class are calculated from the audio fingerprint library. By setting the threshold value of the discrete Gini coefficient of the audio fingerprint, it can be discarded. The threshold audio fingerprint dimension information, keep the audio fingerprint dimension information higher than the threshold, the discrete Gini coefficient threshold range of song audio fingerprint is 0.36~0.38, from the figure we can see the difference of discrete Gini coefficient of each dimension of this type of audio fingerprint Relatively large, the discrete Gini coefficients of each dimension of the fingerprint are relatively scattered. The discrete Gini coefficients of the 2nd, 14th, 15th, and 25th dimensions of the fingerprint are relatively small, and the discrete Gini coefficients of the 4th, 5th, 11th, 18th, and 29th dimensions of the fingerprint are relatively large. The coefficient is large. At this time, if the discrete Gini coefficient threshold of this type of audio fingerprint is set to 0.37, it can be seen from the figure that the discrete Gini coefficient of the fingerprint is less than 0.37. There are 1, 2, 14, 15, 25, and 26 dimensions It shows that the difference of the fingerprint data of each frame in these 6 dimensions is small and can be discarded.

From these figures, it can be seen that although the discrete Gini coefficients of various audio fingerprints are different, and the trend of the coefficients in each dimension is also different, there are always several dimensions of fingerprints with small Gini coefficients, which shows that audio fingerprints are in these dimensions. The discriminativeness of each dimension is small, so the data of these dimensions can be removed, and the 32-dimensional audio fingerprint can be converted into a lower dimension to effectively reduce the amount of fingerprint data.

Claims (2)

1. An audio fingerprint dimension reduction method based on discrete kini coefficients is characterized by comprising the following steps:

step 1, constructing a target sound library in a classified manner

Classifying the audios to build a library according to the audio characteristic types or the existing data conditions;

step2, extracting fingerprint characteristics of sample audios in classification mode

Selecting various types of audio data from a constructed audio library as original sample audio, extracting fingerprint features of the original sample according to the types and introducing a discrete kini coefficient to reduce the dimension of the fingerprint features, wherein the specific process comprises the following steps:

step2.1: pre-processing the original sample audio, the pre-processing comprising: band-pass filtering, pre-emphasis, windowing and framing;

step2.2: fingerprint feature extraction is carried out on preprocessed audio data

(1) Performing discrete Fourier transform on the framed audio signal, and performing discrete Fourier transform on each frame of data of the audio signal, wherein the transform formula is as follows:

wherein, x (k) is a frequency domain signal, x (p) is a time domain signal, k is a frequency index, and T is a sample length of discrete fourier transform;

(2) dividing frequency spectrum sub-bands of the frequency domain signal after discrete Fourier transform, selecting 33 non-overlapping frequency bands from the frequency spectrum, distributing the frequency bands in the range of 20-4000Hz, wherein the frequency bands are equally logarithmically spaced, and the starting frequency of the mth sub-band, namely the termination frequency f (m) of the mth-1 sub-band, can be expressed as the following formula:

wherein Fmin is a mapping lower limit, Fmax is a mapping upper limit, and M is the number of subbands, here 33;

(3) calculating the energy of each sub-band of each frame of audio, and respectively calculating the energy of the selected 33 non-overlapping frequency bands, so that the formula of the energy of the mth sub-band of the nth frame is as follows:

wherein, f (m) is the starting frequency of the mth sub-band, f (m +1) is the ending frequency of the mth sub-band, and x (k) is the frequency domain signal after the discrete fourier transform of the nth frame;

(4) generating a sub-fingerprint of each frame of audio, specifically: and performing bit difference judgment on the 33 subband energies obtained by each frame to generate 32-bit binary codes of each frame of audio, namely the sub-fingerprints of each frame of audio, wherein F (n, m) is binary bit information corresponding to the 32-bit binary codes, and a specific judgment formula is as follows:

wherein E (n, m) is the mth subband energy of the nth frame, and m is 1,2, …, 32;

step3, reducing the dimension of the audio fingerprint characteristics of various samples

Step3.1: the discrete kini coefficient of each dimensionality of the audio fingerprints of various samples is obtained, and the method specifically comprises the following steps:

(1) and processing the audio fingerprints of various types of samples according to frames, and dividing the audio fingerprints into N groups.

(2) Constructing j-dimension accumulated fingerprint data vectors of the audio fingerprints of various samples, wherein a specific calculation formula is as follows:

whereinAre numbered with group numbers and have F (, x) is the binary bit information in step2.

(3) Constructing j-dimension accumulated fingerprint data proportion vector of audio fingerprints of various types of samples The elements in the vector are defined as:

………

(4) obtaining discrete kini coefficients of all dimensionalities of the audio fingerprints of all types of samples, wherein a specific calculation formula is as follows:

wherein i is a group number,accumulating the proportion of the ith group of fingerprint data for the jth dimension of the audio fingerprint, and specifically obtaining the proportion in the previous step (3);

step3.2: performing fingerprint dimensionality reduction on discrete kini coefficients of all dimensionalities of audio fingerprints of various types of samples

And (3) respectively forming training sets corresponding to the categories by various sample data, performing statistical analysis on the discrete kini coefficient of each dimensionality of the audio fingerprint of each training set to obtain a threshold value of the discrete kini coefficient of the audio fingerprint of each category, then performing dimensionality reduction operation on the audio to be subjected to dimensionality reduction according to the obtained threshold value, namely, retaining the audio fingerprint dimensionality information higher than the threshold value, deleting the audio fingerprint dimensionality information lower than the threshold value, and finishing dimensionality reduction.

2. The discrete kini coefficient-based audio fingerprint dimension reduction method according to claim 1, wherein:

selecting 8kHz sampled audio signals as processing objects in the step2, and selecting a band-pass filter with a passband ranging from 20Hz to 4000Hz to process the sampled audio signals, wherein the band-pass filter is a finite impulse response FIR filter;

the pre-emphasis process is shown as follows:

s(p)＝y(p)-μ*y(p-1)

wherein y (p) is a band-pass filtered signal, μ is a pre-emphasis coefficient, and s (p) is a pre-emphasis processed signal, preferably implemented with a digital filter having 6 dB/octave;

the windowing framing processing specifically refers to framing the pre-emphasized audio in an overlapping and segmenting mode, wherein a 75% overlapping rate is kept between frames, each frame is weighted by a Hanning window with the same length, and a windowing formula is as follows:

wherein, T is the length of the hanning window and is also the frame length of one frame of audio.