WO2008148289A1 - An intelligent audio identifying system and method - Google Patents

Abstract

An intelligent audio identifying system and method are provided. The system includes an audio data set for collecting and storing all kinds of audio data samples (1), a training unit (2) and an identifying unit (3). The training unit (2) is used for extracting feature vectors of the audio data samples, searching and establishing the map relationship of the feature vectors of the audio data samples and corresponding classes. The identifying unit (3) is used for storing the established data presenting the map relationship of the audio data samples and the corresponding classes, and extracting a feature vector of the audio data to be identified, and obtaining the identifying result according to the feature vector of the audio data to be identified.

Description

 Intelligent audio recognition system and identification method

 The present invention relates to a system and method for automatically recognizing audio data. Background technique

 Hearing is one of the important sources of human access to external information. It is also an important channel for humans to distinguish external occurrences. For example, when you hear a barking, you can judge that there may be dogs nearby. When you hear screams, you can It is determined that someone may be hurt nearby. Many important pieces of information of the present invention can be provided by analysis of audio. At present, most of the functions of audio-based analysis systems are to pre-process the original audio collected, such as: denoising, extracting or enhancing the audio of the specified features, but finally the recognition of the audio requires human participation. In many applications in nature, different sounds need to be automatically identified. For example, for wild zoologists engaged in wildlife research in the wild, it takes a lot of time to track some rare wild animals, if there is automatic audio data. Identifying the system to identify the sound of a wild animal, signaling when it recognizes the sound of the animal, can help the wildlifeologist track it. For example, if there is an automatic audio recognition system in the elevator or the home, it can automatically identify abnormal sounds such as screams, screams, percussive sounds, glass breaks, explosions, and gunshots, and send out alarm signals. Give the monitoring personnel, so as to improve the response time of the monitoring personnel to the abnormal situation. Therefore, the realization of automatic recognition of audio will have important and extensive application value. Summary of the invention

 The technical problem to be solved by the present invention is to provide an intelligent audio recognition system and an automatic identification method for automatically identifying audio data.

 The technical solution adopted by the present invention to solve the above technical problems is:

 An intelligent audio recognition method includes the following steps:

 A. Collect various sample audio data, and mark the collected sample audio data;

 B. Extracting feature vectors reflecting the essential features from the sample audio data one by one;

 C. Dividing the category region according to the feature vector, so that each of the divided different category regions includes as many feature vectors as the sample, and establishing a classifier from the feature vector to the mapping relationship between the categories;

D. processing the identified audio data, and extracting its feature vector; E. The feature vector of the audio data to be identified is input to the classifier, and the classifier performs discrimination according to the feature vector to obtain an identification result of the to-be-identified audio data.

 The method, wherein the step B includes the following steps:

 Bl, preprocessing the sample audio data to obtain training data;

 B2, extracting feature components reflecting the essential characteristics of the training data from the training data;

 B3. Combining the feature components to obtain the feature vector.

 The method, wherein the step D includes the following steps:

 Dl, preprocessing the to-be-identified audio data to obtain identification data;

 D2. Extracting characteristic components reflecting the essential characteristics of the identification data from the identification data;

 D3. Combining the feature components to obtain the feature vector.

 The method, wherein: the characteristic component described in the step B2 or D2 comprises: a center frequency of the audio, an energy feature of the audio in some specific frequency segments or an energy distribution feature of the audio in the plurality of time periods.

 The method, wherein: the feature vector described in the step B3 or D3 is a vector sum of a center frequency of the audio and a sum of audio energy spectra in some specific frequency segments.

 The method, wherein: the category region to be described in step C is divided according to the value of the feature vector, and is defined by a curve or a curved surface.

 The method, wherein: the step E includes the following processing:

 El, inputting a feature vector of the audio data to be identified to the classifier, and the classifier performs discrimination according to the feature vector, and obtains a classification result of the audio data to be identified into the belonging category and a rejection index, where the rejection index is used a parameter to measure the credibility of the classification results;

 E2, judging the credibility of the classification result according to the rejection index, when the category index is higher than the preset threshold, determining that the classification result is credible, the classifier gives the category of the audio data to be identified; when the rejection index is lower than the pre- When the threshold is set, the classifier gives the category of the audio data to be identified, and indicates that the classification result is not authentic.

 The method, wherein: the step A comprises identifying the collected sample audio data, determining and indicating what sound the sample audio data is.

An intelligent audio recognition system includes an audio data set for collecting and storing various types of sample audio data, a training unit, and an identification unit; the training unit is configured to extract feature vectors of sample audio data, and find and establish a mapping relationship between the sample audio data feature vector and the belonging category; the identification unit is configured to store data of a mapping relationship between the established audio data feature vector and the belonging category, and extract the feature data of the audio data to be identified, and according to The feature vector of the audio data to be identified is given the identification result. The system, wherein: the training unit includes a first pre-processing module, a first feature extraction module, and a training module, the pre-processing module is configured to perform denoising processing on the sample audio data to obtain training data; and the feature extraction module A feature vector for extracting sample audio data from the training data, the training module for finding and establishing a mapping relationship from the sample audio data feature vector to the belonging class.

 The system, wherein: the identification unit comprises a second pre-processing module, a second feature extraction module, and a classifier, wherein the second pre-processing module is configured to perform denoising processing on the audio data to be recognized, to obtain identification data; The second feature extraction module is configured to extract a feature vector of the audio data to be recognized from the identification data, where the classifier is configured to store data of a mapping relationship between the audio data feature vector output by the training module and the category, and according to the input The feature vector of the audio data to be identified, and the identification result is output.

 The invention has the beneficial effects that: the intelligent audio recognition system and method of the invention can automatically identify the audio data, and the system has good real-time performance and expansion capability. DRAWINGS

 Figure 1 is a block diagram of the system of the present invention;

 Figure 2 is a block diagram of a training unit of the present invention;

 Figure 3 is a block diagram of an identification unit of the present invention;

 FIG. 4 is a schematic diagram of establishing a mapping relationship between feature vectors and categories when the sample audio data is of four types; FIG. 5 is a schematic diagram of establishing a mapping relationship between feature vectors and categories when the sample audio data is of two types. detailed description

 The present invention will be further described in detail below based on the accompanying drawings and embodiments:

 An intelligent audio recognition system as shown in FIG. 1 includes at least one audio data set 1, a training unit 2 and an identification unit 3 for collecting and storing various types of sample audio data. The training unit 2 is configured to extract feature vectors of the sample audio data, and find and establish a mapping relationship from the sample audio data feature vector to the belonging category; the identification unit 3 is configured to store the established audio data feature vector and the category The data of the mapping relationship is extracted, and the feature vector of the audio data to be identified is extracted, and the identification result is given according to the feature vector of the audio data to be identified. The training unit, as shown in FIG. 2, includes a first pre-processing module 21, a first feature extraction module 22, and a training module 23. As shown in FIG. 3, the identification unit includes a second pre-processing module 31, a second feature extraction module 32, and a classifier 33.

The establishment of the audio data set 1 is to provide the necessary training samples for the subsequent training unit 2. The user collects audio data according to the category of audio that needs to be recognized. The data set can be created by using its own recording, collecting audio material from the Internet, and purchasing Buy audio material CDs and other methods to collect learning samples. Generally speaking, each type of audio needs to collect multiple samples, and in the process of sample collection, the collected samples need to be manually labeled, that is, the collected samples are answered by the human ear, and then the sample is determined to be what sound. In order to ensure the identification of the system, samples should be collected as much as possible.

 In the training unit 2, the collected sample audio data needs to be pre-processed first, that is, the pre-processing module 21 removes noise and the like from the sample audio data from the audio data set 1, and the sample audio to be recognized is from a complex audio background. The separated training data is obtained. Then, the feature extraction module 22 extracts components from the training data that reflect the essential characteristics of the sample audio data, such as: the center frequency of the audio, and the energy characteristics of the audio in certain frequency segments. The energy distribution characteristics of the audio in the plurality of time periods are obtained by Fourier transforming the audio signal, and the features are combined to obtain corresponding feature vectors. For example: The center frequency of the sample audio is 33, and the sum of the energy spectra in an audio segment is 1000. The resulting feature vector is the vector sum of the sum of the center frequency and the energy spectrum in an audio segment (33,1000). Then, the training module 23 uses the extracted feature components to train the classifier 33 for recognizing the audio, that is, the training classifier, that is, the training module 23 finds a plurality of classification curves or surfaces according to the feature vectors of the N sample audio data. Separating N classification regions by classification curves or surfaces, so that the feature vectors of each specimen audio data are distributed in different classification regions, and the classification regions are divided according to the values of the feature vectors, that is, a feature vector space is established. The mapping to the category. For example, when the sample audio data is only four different categories of data, for the classification problem of the four categories, when the feature vector is two-dimensional, the training module 23 is equivalent to finding two straight lines so that the feature vectors of the four types of samples are respectively It is distributed in four regions divided by two lines. As shown in Fig. 4, the triangle is the first type of eigenvector obtained during training, and the circle is the second type of eigenvector obtained during training. The five-pointed star is obtained during training. The third type of eigenvectors, the pentagon is the fourth type of eigenvectors obtained during training, and the straight line 1 and the straight line 2 are the classification lines obtained from the four types of eigenvectors (ie, the four regions divided by the two classification lines should be respectively The method includes dividing the feature space into four subspaces, and the training module stores the trained data, that is, the data of the established audio data feature vector and the mapping relationship between the categories, in the classifier 33.

 The division principle of the category region in the method of the invention is: by dividing the feature vector space, the divided different types of regions only contain feature vectors of the same type of samples, or as many feature vectors as possible of the sample, as little as possible Contains feature vectors that are not such samples.

 The function of the identification unit 3 is to use the classifier 33 trained by the training module 23 according to the audio data to be recognized to obtain the identification result. The second pre-processing module 31 and the second feature extraction module 32 in the identification unit respectively function the same as the second pre-processing module 21 and the second feature extraction 22 in the training unit.

After the audio sample to be identified is obtained, it is first pre-processed by the pre-processing module 31 to obtain the processed identification data. Then, the feature extraction method in the feature extraction module 22 is used to identify the audio data. Feature extraction, obtaining a feature vector of the audio data to be recognized; thereafter, the extracted feature vector is input as a classifier 33 (obtained by the training module 23), and the classifier outputs the recognition result according to the input feature vector. For example, when the feature vector to be classified is distributed in the space sandwiched by the upper half of the straight line 1 and the lower half of the straight line 2 (such as the hexagon in FIG. 4), the present invention discriminates the feature vector to be classified as Category 1. If the feature vector to be classified is distributed in the space sandwiched by the upper half of the straight line 1 and the straight line 2 (such as the heptagon in FIG. 4), the present invention discriminates the feature vector to be classified into the first part of the circular feature vector. In the second category, and so on, the octagon in the drawing can be divided into the third category, and the hexagonal star is divided into the fourth category.

 It can be seen that the classifier gives the identification result of the audio class to be identified according to the input feature vector of the to-be-identified audio data, and the more sample audios are collected in the audio data set, the more the classified regions are divided, the more to be identified The finer the audio data classification, the closer the classification result is to the real sound category.

 In the pattern classification system, the more commonly used classifiers are neural networks, support vector machines, Adaboost, and so on. The process of obtaining a linear classification surface by a classifier based on a linear support vector machine is introduced below.

 First, let's distinguish between two types of problems:

Given two types of eigenvectors and their associated categories: ')^'') ^ {±1}. The interface w of the linear support vector machine can be obtained by solving the following optimization problems:

 S.t. ^ [w-X; -^] + ?7, ≥1

≥0, = l,...,l where ^c>0 is a fixed penalty parameter. When we get a new eigenvector χ, if w'x_b ≥ o, then the eigenvector is considered to belong to category 1; if ^w 'x- ^{< 0} , then the eigenvector is considered to belong to category-1. The absolute value of n ^b can be used as a rejection index, which is considered reliable when ^{1 W} ' ^X _ is greater than a certain threshold.

Then use the one-against-one method to extend it to multiple types of problems. For a class classification problem, the one-against-one method constructs ^ _ ^1)/2 classification faces. These classification planes obtain ^^_ ^1)/2 classification planes by taking out the combination of the two types from the class and then using the above-mentioned classification surface construction method for the two types of problems. We use the voting method to determine the category to which the feature vector X belongs. Let: For the class and class, the classification face is ^w. If 'X _ b >= 0, vote for the i-th class; if ¹ ' X _ ^{& k Q} , vote for the j-th class. When the voting is completed according to ^ _ / ² classification faces, the category with the highest number of votes will be used as the final classification result. At the same time, each class needs to accumulate ^{1 W} '^X_&I at the same time as the voting, and the last sum will be used as the rejection index. When the accumulated sum is greater than a certain threshold, the classification is considered to be reliable. The identification result output by the classifier includes a classification result and a rejection index of the category to which the audio to be recognized belongs, and the feature vector of the audio data to be recognized in FIG. 4 is a hexagon as an example, since the hexagon is distributed on the straight line 1. In the first type of space sandwiched between the half and the lower half of the line 2, the category is the first category. The more the position of the hexagon falls into the space of the first type, the more similar the feature vector of the first type of sample is. The higher the credibility of the classification result, the closer the hexagon falls to the classification line when it falls into the space of the first type, indicating that the classification result is less reliable.

 The refusal index is a parameter used to measure the credibility of the classification results. For a classifier based on probability criteria, the output classification result is a probability belonging to a certain class, and the probability can be used as a rejection index. If the probability of belonging to all classes in the output result is less than a certain probability, then the rejection is The sample category is discriminated; for the classifier based on the classification surface, the distance between the feature vector of the sample and the nearest classification surface can be used as the rejection index, if the distance between the feature vector of the sample and the nearest classification surface is smaller than For a certain value, the sample category is rejected.

 The rejection index is used to determine the credibility of the classification result. In practical applications, the threshold can be set according to the experiment (for example: the invention can establish a small-scale test set, and then find a threshold to concentrate the test. For some untrustworthy samples, the value is set to a threshold value. A threshold value is preset. When the rejection index is greater than the preset threshold, the classification result given by the classifier is credible; When the index is less than the preset threshold, the classification result of the classifier is less reliable, and the classifier indicates that the classification result is not credible while giving the category of the audio data to be identified. For example: As shown in Figure 5, for two types of classification problems, when the feature vector is two-dimensional, training the linear classifier is equivalent to finding a straight line so that the eigenvectors of one type of sample are distributed on one side of the line, and the other type The feature vector of the sample is distributed on the other side of the line. The triangle in Fig. 5 is a kind of eigenvector obtained during training. The circle is another type of eigenvector obtained during training. The straight line is the classification line obtained by these two kinds of eigenvectors. When the feature vector of the audio data to be recognized is distributed to the left of the line (such as a square in Fig. 5), the present invention discriminates the feature vector to be classified into a class to which the triangle feature vector belongs. If the feature vector to be classified is distributed to the right of the straight line (such as the five-pointed star in Fig. 4), the present invention discriminates the feature vector to be classified into a class to which the circular feature vector belongs. At this time, the linear classification surface parameter (the linear classification surface parameter can be obtained by obtaining the linear classification surface normal vector) and the symbol (positive or negative) of the dot product of the feature vector to be classified will be used to distinguish the feature vector to be classified, and the point The absolute value of the product is the rejection index, which is used to measure the credibility of the classification. The larger the rejection index (the absolute value of the dot product), the higher the reliability of the classification. When the absolute value of the dot product is greater than the default. At the threshold, the classification is considered reliable.

 In practical applications, the method and system can be used to identify various unused audios in nature, and the system can be used to identify specific audios first, and based on the identification results, to implement subsequent Function, training a fast, scalable classifier to ensure that the system has good real-time and scalability.

The intelligent audio recognition system of the invention can be used for intelligent monitoring in various situations. Such as: can be installed in the elevator The system automatically recognizes abnormal sounds such as screams, screams, and percussive sounds, and sends an alarm signal to the monitoring personnel, thereby improving the reaction time for handling abnormal conditions in the elevator, and at the same time reducing the workload of the elevator monitoring personnel. . The system can also be used for home monitoring. After installing the system indoors, the system can identify the abnormal sounds that may occur in the room such as glass breaking sound, door crashing sound, explosion sound, gunshot sound, etc., and immediately send out an alarm signal after identifying these abnormal sounds, thereby effectively Prevent the occurrence of criminal acts such as theft of doors and windows into the room. The system can also be installed outdoors to automatically identify weather-related sounds such as thunder, wind, and rain, and monitor weather conditions in real time. In addition, the system can help wildlife researchers working in the field to conduct research. Wild zoologists often need to spend weeks or even months to track some rare wild animals. The present invention can identify the sound of a certain wild animal by broadcasting a wireless sensor installed in the designated area. The sound of the animal is signaled to help the wildlifeologist track it. The system can also be used for the diagnosis of mechanical faults. When the machine malfunctions, it will emit a sound different from the normal operation of the machine, and the sound of different faults will be different. The system can learn according to several different fault audios, and then install the real-time sound of the machine work near the machine. When the fault sound is recognized, the alarm will be given and the possible fault categories will be given. The result can help people in time. Found machine failures and provided a basis for machine troubleshooting. The system can also be applied to Internet-based audio retrieval and audio-based scene analysis.

 It is to be understood that those skilled in the art can devise modifications and changes in accordance with the above description, and all such modifications and changes are intended to be included within the scope of the appended claims.

Claims

Rights request 1. An intelligent audio recognition method, comprising the following steps:  A. Collect various sample audio data, and mark the collected sample audio data;  B. Extracting feature vectors reflecting the essential features from the sample audio data one by one;  C. Dividing the category region according to the feature vector, so that each of the divided different category regions includes as many feature vectors as the sample, and establishing a classifier from the feature vector to the mapping relationship between the categories; D. processing the identified audio data, and extracting its feature vector;  E. The feature vector of the audio data to be identified is input to the classifier, and the classifier performs discrimination according to the feature vector to obtain an identification result of the to-be-identified audio data.  2. The method according to claim 1, wherein: the step B comprises the following steps: Bl, preprocessing the sample audio data to obtain training data;  B2, extracting feature components reflecting the essential characteristics of the training data from the training data;  B3. Combining the feature components to obtain the feature vector.  3. The method according to claim 1, wherein: said step D comprises the following steps: Dl, preprocessing the to-be-identified audio data to obtain identification data;  D2. Extracting characteristic components reflecting the essential characteristics of the identification data from the identification data;  D3. Combining the feature components to obtain the feature vector.  The method according to claim 2 or 3, wherein: the characteristic component described in the step B2 or D2 comprises: a center frequency of the audio, an energy characteristic of the audio in some specific frequency segments or an audio in a plurality of time periods Energy distribution characteristics.  5. The method according to claim 4, wherein: the feature vector described in step B3 or D3 is a vector sum of a center frequency of the audio and a sum of audio energy spectra in some specific frequency segments.  6. The method according to claim 5, wherein: the category region to be described in step C is divided according to the value of the feature vector, and is defined by a curve or a curved surface.  7. The method according to claim 6, wherein: said step E comprises the following processing:  El, inputting a feature vector of the audio data to be identified to the classifier, and the classifier performs discrimination according to the feature vector, and obtains a classification result of the audio data to be identified into the belonging category and a rejection index, where the rejection index is used a parameter to measure the credibility of the classification results; E2, determining the credibility of the classification result according to the rejection index, and determining that the category index is higher than a preset threshold The classification result is credible, and the classifier gives the category of the audio data to be identified; when the rejection index is lower than the preset threshold, the classifier gives the category of the audio data to be identified, and indicates that the classification result is not credible.  The method according to claim 7, wherein the step A comprises: identifying the collected sample audio data, determining and indicating what sound the sample audio data is.  9. An intelligent audio recognition system, comprising: an audio data set for collecting and storing various types of sample audio data, a training unit, and an identification unit; wherein the training unit is configured to extract characteristics of sample audio data Vector, and searching and establishing a mapping relationship from the sample audio data feature vector to the belonging category; the identifying unit is configured to store data of the mapping relationship between the established audio data feature vector and the belonging category, and extract the to-be-identified audio The data feature vector, and the identification result is given according to the feature vector of the audio data to be recognized.  The system according to claim 9, wherein: the training unit comprises a first pre-processing module, a first feature extraction module, and a training module, wherein the pre-processing module is configured to perform denoising processing on the sample audio data to obtain Training data; the feature extraction module is configured to extract feature vectors of sample audio data from the training data, and the training module is configured to find and establish a mapping relationship from the sample audio data feature vector to the belonging category.  The system according to claim 9 or 10, wherein: the identification unit comprises a second pre-processing module, a second feature extraction module, and a classifier, wherein the second pre-processing module is configured to identify audio data Performing a denoising process to obtain identification data; the second feature extraction module is configured to extract a feature vector of the to-be-identified audio data from the identification data, where the classifier is configured to store the audio data feature vector and the category of the training module The data of the relationship is mapped, and the identification result is output according to the feature vector of the input audio data to be recognized.