WO2024174183A1 - Lung sound enhancement method and system, and device and storage medium - Google Patents

Abstract

The embodiments of the present application belong to the technical field of sound signal processing. Disclosed are a lung sound enhancement method and system, and a device and a storage medium. The lung sound enhancement method comprises: acquiring lung sound data to be enhanced; and converting said lung sound data by using a trained lung sound enhancement model, so as to obtain enhanced lung sound data, wherein the lung sound enhancement model is constructed on the basis of a first variational auto-encoder, a second variational auto-encoder and a cyclic consistency loss model, and is trained by means of clean lung sound data and noisy lung sound data, so as to obtain the enhanced lung sound data.

Description

Lung sound enhancement method, system, device and storage medium Technical Field

The present application relates to the technical field of sound signal processing, and in particular to a lung sound enhancement method, system, device and storage medium.

Background Art

Lung sounds are the sounds produced by the vibrations caused by airflow passing through the body's airways and alveoli during breathing. This sound is transmitted to the body surface through the lung tissue and chest wall. Lung sound signals have non-Gaussian random characteristics and are unstable sound signals. In the medical field, lung sounds are an important indicator of the physiological and pathological health of the lungs. Lung auscultation has the advantages of rapidity, low cost, and sustainable monitoring, and is widely used by medical personnel.

Since lung sounds are prone to contain a lot of noise, and the frequency bands of these noises overlap with the frequency bands of lung sounds, wavelet transforms, adaptive filters, etc. are usually used in the prior art to enhance lung sound signals. However, these methods require a large number of clean lung sounds as samples for model training, and clean lung sounds are difficult to obtain, which increases the cost of lung sound enhancement.

Summary of the invention

In view of this, the present application provides a lung sound enhancement method, system, device and storage medium, which are used to solve the problem that lung sound data collection is difficult in the prior art, resulting in high cost of lung sound enhancement. In order to achieve one or part or all of the above purposes or other purposes, the present application proposes a lung sound enhancement method, system, device and storage medium, in the first aspect:

A lung sound enhancement method, comprising:

Acquire lung sound data to be enhanced;

Using the trained lung sound enhancement model to transform the lung sound data to be enhanced, to obtain enhanced lung sound data;

The lung sound enhancement model is constructed based on a first variational autoencoder, a second variational autoencoder, and a cycle consistency loss model, and is trained with clean lung sound data and noisy lung sound data to obtain enhanced lung sound data.

In one embodiment, before converting the lung sound data to be enhanced using the trained lung sound enhancement model to obtain enhanced lung sound data, the method further includes:

Training the first variational autoencoder using the clean lung sound data to obtain the trained first variational autoencoder and first feature vector data corresponding to the first variational autoencoder;

The second variational autoencoder is trained by using the noisy lung sound data and the first variational autoencoder that has completed training, to obtain second feature vector data corresponding to the second variational autoencoder, a mapping relationship between the first feature vector data and the second feature vector data, and reconstructed noisy data; the second feature vector data, the mapping relationship, and the reconstructed noisy data are obtained under the constraints of the cycle consistency loss model;

When a preset matching condition is satisfied between the reconstructed noisy data and the noisy lung sound data, it is determined that the training of the lung sound enhancement model is completed.

In one embodiment, the step of training the second variational autoencoder using the noisy lung sound data and the trained first variational autoencoder to obtain second feature vector data corresponding to the second variational autoencoder, a mapping relationship between the first feature vector data and the second feature vector data, and reconstructing the noisy data includes:

Using encoder 2 of the second variational autoencoder to encode the noisy lung sound data to obtain the second feature vector data;

According to the mapping relationship between the first feature vector data and the second feature vector data constructed by the cycle consistency loss model, the second feature vector data is decoded by using the decoder 1 of the first variational autoencoder that has completed training to obtain purified lung sound data;

Encode the purified lung sound data using the first encoder of the first variational autoencoder that has been trained Processing to obtain the first feature vector data;

Based on the mapping relationship, the first feature vector data is decoded using decoder 2 of the second variational autoencoder to obtain the reconstructed noisy data.

In one embodiment, after obtaining the enhanced lung sound data, the method further includes:

Processing the enhanced lung sound data using a preset generative adversarial network model to obtain discriminant value data;

The generative adversarial network model includes a generator and a discriminator; the first variational autoencoder and the second variational autoencoder both correspond to the generative adversarial network model; the generative adversarial network model corresponding to the first variational autoencoder determines the decoder 1 as the generator; the generative adversarial network model corresponding to the second variational autoencoder determines the decoder 2 as the generator; after the loss function of the first variational autoencoder converges, the corresponding generative adversarial network model is trained; after the loss function of the second variational autoencoder converges, the corresponding generative adversarial network model is trained.

In one embodiment, the decoder 1, the decoder 2 and the discriminator all include a two-layer multi-head self-attention model.

In one embodiment, after the enhanced lung sound data is processed using a preset generative adversarial network model to obtain discriminant value data, the method further includes:

Processing the enhanced lung sound data using the trained lung sound phase correction model to obtain phase-corrected lung sound data;

The lung sound phase correction model is trained using the clean lung sound data containing white noise.

In one embodiment, after the enhanced lung sound data is processed using the trained lung sound phase correction model to obtain the phase-corrected lung sound data, the method further includes:

The trained signal-to-noise ratio prediction model is used to process the lung sound data to be enhanced and the enhanced lung sound data to obtain corresponding signal-to-noise ratio prediction data.

Second aspect:

A lung sound enhancement system comprises an acquisition module for acquiring lung sound data to be enhanced;

A lung sound enhancement module, used to convert the lung sound data to be enhanced by using the trained lung sound enhancement model to obtain enhanced lung sound data;

The lung sound enhancement model is constructed based on a first variational autoencoder, a second variational autoencoder, and a cycle consistency loss model, and is trained with clean lung sound data and noisy lung sound data to obtain enhanced lung sound data.

In one embodiment, the system further includes a first training module, which is used to train the first variational autoencoder using the clean lung sound data before converting the lung sound data to be enhanced using the trained lung sound enhancement model to obtain the enhanced lung sound data, so as to obtain the trained first variational autoencoder and first feature vector data corresponding to the first variational autoencoder;

A second training module is used to train the second variational autoencoder through the noisy lung sound data and the first variational autoencoder that has completed the training, to obtain second feature vector data corresponding to the second variational autoencoder, a mapping relationship between the first feature vector data and the second feature vector data, and reconstructed noisy data; the second feature vector data, the mapping relationship, and the reconstructed noisy data are obtained under the constraints of the cycle consistency loss model;

The iteration module is used to determine that the training of the lung sound enhancement model is completed when a preset matching condition is met between the reconstructed noisy data and the noisy lung sound data.

In one embodiment, the second training module includes a first encoding unit, configured to encode the noisy lung sound data using encoder 2 of the second variational autoencoder to obtain the second feature vector data;

A first decoding unit is configured to decode the second feature vector data using a decoder of the first variational autoencoder that has completed training, according to a mapping relationship between the first feature vector data and the second feature vector data constructed by the cycle consistency loss model, to obtain purified lung sound data;

A second encoding unit is used to use the first variational autoencoder encoder pair trained to encode the purification The lung sound data is encoded to obtain the first feature vector data;

The second decoding unit is used to decode the first feature vector data using the second decoder of the second variational autoencoder based on the mapping relationship to obtain the reconstructed noisy data.

In one embodiment, the system further comprises a generative adversarial network module, which is used to process the enhanced lung sound data using a preset generative adversarial network model to obtain discriminant value data after the enhanced lung sound data is obtained;

The generative adversarial network model includes a generator and a discriminator; the first variational autoencoder and the second variational autoencoder both correspond to the generative adversarial network model; the generative adversarial network model corresponding to the first variational autoencoder determines the decoder 1 as the generator; the generative adversarial network model corresponding to the second variational autoencoder determines the decoder 2 as the generator; after the loss function of the first variational autoencoder converges, the corresponding generative adversarial network model is trained; after the loss function of the second variational autoencoder converges, the corresponding generative adversarial network model is trained.

In one embodiment, the decoder 1, the decoder 2 and the discriminator all include a two-layer multi-head self-attention model.

In one embodiment, the system further includes a correction module for processing the enhanced lung sound data using a trained lung sound phase correction model to obtain phase-corrected lung sound data after the enhanced lung sound data is processed using a preset generative adversarial network model to obtain discriminant value data;

The lung sound phase correction model is trained using the clean lung sound data containing white noise.

In one embodiment, the system further includes a signal-to-noise ratio prediction module, which is used to process the enhanced lung sound data using the trained lung sound phase correction model to obtain phase-corrected lung sound data, and then use the trained signal-to-noise ratio prediction model to process the lung sound data to be enhanced and the enhanced lung sound data to obtain corresponding signal-to-noise ratio prediction data.

The third aspect:

A computer device includes a memory and a processor, wherein a lung sound enhancement method is stored in the memory, and the processor is used to adopt the above-mentioned method when executing the lung sound enhancement method.

Fourth aspect:

A storage medium stores a computer program that can be loaded by a processor and execute the above method.

Implementing the embodiments of the present application will have the following beneficial effects:

The cycle consistency loss model is used in the construction process of the lung sound enhancement model; clean lung sound data and noisy lung sound data are used in the training process of the lung sound enhancement model. This makes it unnecessary to use clean lung sound data and noisy lung sound data in equal proportions in the training process of the lung sound enhancement model, thereby reducing the demand for clean lung sound data, reducing the difficulty of training the lung sound enhancement model, and also reducing the training cost of the lung sound enhancement model. Only a small amount of clean lung sound data is needed to complete the training, reducing the cost of lung sound enhancement.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

in:

FIG. 1 is a flow chart of a lung sound enhancement method according to an embodiment.

FIG. 2 is a flow chart of obtaining reconstructed noisy data in a lung sound enhancement method in one embodiment.

FIG. 3 is a schematic diagram of a shared latent space correlation structure of lung sound mapping in one embodiment.

FIG4 is a schematic diagram of the structure of a variational autoencoder combined with a generative adversarial network model in one embodiment.

FIG. 5 is a flow chart of lung sound phase correction in one embodiment.

FIG6 is a structural block diagram of a lung sound enhancement system in one embodiment.

FIG. 7 is a schematic diagram of the structure of a lung sound enhancement device in one embodiment.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

The present application provides a lung sound enhancement method. Since lung sounds are prone to contain a large amount of noise, and the frequency bands of these noises overlap with the frequency bands of lung sounds, wavelet transform, adaptive filter, etc. are usually used in the prior art to enhance lung sound signals. However, these methods require a large number of clean lung sounds as samples for model training, and clean lung sounds are difficult to obtain, which increases the cost of lung sound enhancement.

In order to solve the above defects, an embodiment of the present application provides a lung sound enhancement method, as shown in FIG1 , comprising:

101. Obtain lung sound data to be enhanced.

In one embodiment, the lung sound data to be enhanced refers to the lung sound data that needs to be denoised. The lung sound data to be enhanced can be passively acquired by the current executing subject, or it can be actively acquired by the current executing subject. For example, in one application scenario, the lung sound collection device transmits the lung sound data to be enhanced to the current executing subject, so that the current executing subject passively obtains the lung sound data to be enhanced; in another application scenario, after detecting or receiving the enhancement instruction, the current executing subject obtains the lung sound data to be enhanced at a preset path or interface. It should be noted that the current executing subject can be an intelligent device with data processing function, such as MCU, integrated chip, single-chip microcomputer, computer, etc.

102. Use the trained lung sound enhancement model to convert the lung sound data to be enhanced to obtain enhanced lung sound data.

In one embodiment, the lung sound enhancement model is constructed based on a first variational autoencoder, a second variational autoencoder, and a cycle consistency loss model, and is trained by using clean lung sound data and noisy lung sound data.

For ease of understanding, the lung sound enhancement model is explained. Since it is necessary to remove the noise in the lung sound data to be enhanced to obtain enhanced lung sound data with no or little noise, a model that can convert the lung sound data to be enhanced into enhanced lung sound data is needed. Among them, enhanced lung sound data refers to the enhanced lung sound data after denoising. In the process of training the lung sound enhancement model, two variational autoencoders (VAE) are used, named the first variational autoencoder and the second variational autoencoder respectively. By using clean lung sound data to train the first variational autoencoder, the trained first variational autoencoder can extract the features of the clean lung sound data and complete the reconstruction of the clean lung sound data.

In one embodiment, in order to reduce the amount of clean lung sound data required during the training process, a cycle consistency loss model is added during the training process of the second variational autoencoder to solve the imbalance problem between clean lung sound data and noisy lung sound data. The cycle consistency loss model is used to construct a mapping relationship between the two feature vector spaces of the first variational autoencoder and the second variational autoencoder, so that the noisy lung sound data can be converted into enhanced lung sound data after being processed by the first variational autoencoder and the second variational autoencoder, realizing the transformation of the lung sound data to be enhanced to the enhanced lung sound data, and solving the denoising problem of the lung sound data.

The cycle consistency loss model is used in the construction process of the lung sound enhancement model; clean lung sound data and noisy lung sound data are used in the training process of the lung sound enhancement model. This makes it unnecessary to use clean lung sound data and noisy lung sound data in equal proportions in the training process of the lung sound enhancement model, thereby reducing the demand for clean lung sound data, reducing the difficulty of training the lung sound enhancement model, and also reducing the training cost of the lung sound enhancement model. Only a small amount of clean lung sound data is needed to complete the training, reducing the cost of lung sound enhancement.

Encoder 1 In another embodiment of the present application, before the step of converting the lung sound data to be enhanced using the trained lung sound enhancement model to obtain enhanced lung sound data, the method further includes:

201. Train the first variational autoencoder using the clean lung sound data to obtain the trained first variational autoencoder and first feature vector data corresponding to the first variational autoencoder.

In one embodiment, the first variational autoencoder includes an encoder 1 and a decoder 1, wherein the encoder 1 processes the clean lung sound data to obtain first feature vector data, and the decoder 1 processes the first feature vector data to obtain reconstructed clean lung sound data. By comparing the similarity between the clean lung sound data and the reconstructed clean lung sound data, when the similarity reaches a preset threshold, it is determined that the training of the first variational autoencoder is completed.

202. Train the second variational autoencoder using the noisy lung sound data and the trained first variational autoencoder to obtain second feature vector data corresponding to the second variational autoencoder, a mapping relationship between the first feature vector data and the second feature vector data, and reconstruct the noisy data.

The second feature vector data, the mapping relationship and the reconstructed noisy data are obtained under the constraints of the cycle consistency loss model.

For ease of understanding, the first variational autoencoder and the second variational autoencoder pay more attention to data generation compared with the autoencoder. That is, the first variational autoencoder and the second variational autoencoder do not encode the input data as a single point in the latent space, but encode the input data as a probability distribution in the latent space. That is, in one embodiment, it is assumed that the first variational autoencoder corresponds to a first latent space, and the second variational autoencoder corresponds to a second latent space. When training the second variational autoencoder, the first variational autoencoder that has completed the training is used. According to the constraints of the cycle consistency loss model, the noisy lung sound data is processed using the second variational autoencoder to be trained to obtain the second eigenvector data. Assume that there is a shared space, a clean lung sound data is given to the first variational autoencoder, and then a noisy lung sound data is given to the second variational autoencoder. The first eigenvector data located in the first latent space and the second eigenvector data located in the second latent space are constructed to construct a mapping relationship in the shared space to satisfy the cycle consistency loss algorithm.

That is, the process of constructing the mapping relationship between the first feature vector data and the second feature vector data in the shared space is the training process of the second variational autoencoder. The noisy lung sound data is iteratively processed using the second variational autoencoder to be trained and the first variational autoencoder that has completed training, the mapping relationship is optimized, and each iteration generates reconstructed noisy data under the action of decoder 2.

203. When a preset matching condition is satisfied between the reconstructed noisy data and the noisy lung sound data, it is determined that the training of the lung sound enhancement model is completed.

In one embodiment, the reconstructed noisy data is obtained by using encoder 2, decoder 1, encoder 1, and decoder 2 in sequence with noisy lung sound data as input. Therefore, the reconstructed noisy data should be identical or highly similar to the corresponding noisy lung sound data as input. Therefore, in an application scenario, the matching condition is to determine whether the similarity between the reconstructed noisy data and the corresponding noisy lung sound data is greater than a preset threshold. If the similarity is greater than the threshold, it is determined that the matching condition is met, and then it is determined that the lung sound enhancement model training is completed.

By using noisy lung sound data and a cycle consistency loss model to train the second variational autoencoder, the amount of clean lung sound data used is reduced, which helps to reduce the cost of lung sound enhancement.

In addition, in another embodiment of the present application, when training the first variational autoencoder and the second variational autoencoder, the short time Fourier transform (STFT) is used to extract the amplitude spectra of clean lung sound data and noisy lung sound data, and the amplitude spectra are used as inputs of the first variational autoencoder and the second variational autoencoder.

The amplitude spectrogram is used to train the first variational autoencoder and the second variational autoencoder, thereby completing the training of the lung sound enhancement model, thereby improving the quality and efficiency of the lung sound enhancement model.

In another embodiment of the present application, as shown in FIG2, the step of training the second variational autoencoder by using the noisy lung sound data and the first variational autoencoder that has completed the training, obtains the second feature vector data corresponding to the second variational autoencoder, and the first feature vector data and the second feature vector data. The mapping relationship and reconstruction of noisy data include:

301. Use encoder 2 of the second variational autoencoder to encode the noisy lung sound data to obtain the second feature vector data.

For ease of understanding, as shown in FIG3 , the noisy lung sound data _{yi is} input into encoder 2 to obtain the second feature vector data corresponding to the noisy lung sound data.

302. According to the mapping relationship between the first feature vector data and the second feature vector data constructed by the cycle consistency loss model, the second feature vector data is decoded by using the decoder of the first variational autoencoder that has completed training to obtain purified lung sound data.

Specifically, in one embodiment, since a shared space is set, a mapping relationship is established between the first feature vector data and the second feature vector data in the shared space. After obtaining the second feature vector data, based on the mapping relationship, the second feature vector data is made equal to or close to the first feature vector data. Therefore, after the second feature vector data is used as the input of decoder 1, decoder 1 outputs purified lung sound data. It should be noted that, ideally, the purified lung sound data obtained by processing the second feature vector data using the decoder is The same as the reconstructed clean lung sound data obtained by processing the corresponding first feature vector data using a decoder pair, thus completing the conversion of noisy lung sound data to enhanced lung sound data using the mapping relationship. Based on this feature, in the process of training the second variational autoencoder, the mapping relationship is iteratively optimized so that after processing the second feature vector data using a decoder pair, purified lung sound data close to clean lung sound data can be obtained.

303. Encode the purified lung sound data using encoder 1 of the trained first variational autoencoder to obtain the first feature vector data.

Purify lung sound data As the input of encoder one, encoder one purifies lung sound data After processing, the first feature vector data is obtained. Since there is a mapping relationship between the first feature vector data and the second feature vector data, in an ideal state, the first feature vector data is equal to the second feature vector data.

304. Based on the mapping relationship, use decoder 2 of the second variational autoencoder to decode the first feature vector data to obtain the reconstructed noisy data.

The first feature vector data is used as the input of the decoder 2 to obtain the reconstructed noisy data In the ideal state, the first eigenvector data is equal to the second eigenvector data. is equal to the reconstructed noisy lung sound data obtained after the second eigenvector data is processed by the encoder 2. Based on this feature, the optimization progress of the mapping relationship can be determined by comparing the difference between the reconstructed noisy data and the noisy lung sound data. In one embodiment, when the difference between the reconstructed noisy data and the noisy lung sound data is less than a preset threshold, it is determined that the training of the second variational autoencoder is completed, thereby determining that the training of the lung sound enhancement model is completed.

In one application scenario, the intermediate representation during the training of the second variational autoencoder is:

The loss function is:

Under the constraints of the trained first variational autoencoder and the cycle consistency loss model, the second variational autoencoder is trained, thereby completing the training of the lung sound enhancement model. It is not necessary to use more clean lung sounds, which helps to reduce the cost of lung sound enhancement.

In another embodiment of the present application, after the step of obtaining enhanced lung sound data, the method further includes:

The enhanced lung sound data is processed using a preset generative adversarial network model to obtain discriminant value data.

The generative adversarial network model includes a generator and a discriminator; the first variational autoencoder and the second variational autoencoder both correspond to the generative adversarial network model; the generative adversarial network model corresponding to the first variational autoencoder determines the decoder 1 as the generator; the generative adversarial network model corresponding to the second variational autoencoder The generative adversarial network model determines the decoder 2 as the generator; trains the corresponding generative adversarial network model after the loss function of the first variational autoencoder converges; and trains the corresponding generative adversarial network model after the loss function of the second variational autoencoder converges.

In one embodiment, the generative adversarial network model (GAN) is a generative model, and the discriminator should output a value as close to 1 as possible, while for fake samples, the discriminator should output a value as close to 0 as possible. Previously, we trained a pair of VAEs, namely the first variational autoencoder and the second variational autoencoder, to achieve the conversion between clean lung sounds and noisy lung sounds. Since we simply use the mean square error to calculate the reconstruction loss of the VAE for the sample, the magnitude spectrogram generated by the VAE is always not as clear as that generated by the GAN. As shown in Figure 4, a discriminator is added to both VAEs. The two discriminators are combined with the two VAEs. Here, the decoder of the VAE can be regarded as the generator of the GAN, and combined with the discriminator for adversarial learning. The discriminator will simultaneously input the lung audio spectrogram of the current domain and the lung audio spectrogram generated by the VAE switched from another domain and output the probability that it belongs to the current domain. If the spectrum graph is judged to belong to the current domain, the value output by the discriminator will be close to 1. If the spectrum graph is judged to be switched from another domain, the value output by the discriminator will be close to 0. In order to deceive the current discriminator and let the discriminator classify the lung audio spectrogram generated by VAE as the original lung audio spectrogram in the current domain, VAE needs to generate more realistic data. Since the process of training GAN is unstable, in order to speed up the training and obtain better results, the discriminator is added to fine-tune it after the two VAEs have a certain generation ability. Among them, after the loss function converges, it is considered that the judgment model has the generation ability.

The combination of VAE and GAN helps to determine the accuracy of the enhanced lung sound data, thereby promptly discovering the defects of the lung sound enhancement model and improving the quality of lung sound enhancement.

In another embodiment of the present application, the decoder 1, the decoder 2 and the discriminator all include a two-layer multi-head self-attention model.

Specifically, in one embodiment, according to the Self-Attention GAN structure, two layers of multi-head self-attention models are added to the decoder of VAE and the discriminator of GAN respectively.

By adding a multi-head self-attention model, the long-range interactions between temporal features can be better captured. The multi-head self-attention module can help the model focus on information from different representation subspaces from different positions, thereby improving the expressiveness of the model.

In another embodiment of the present application, after the step of processing the enhanced lung sound data using a preset generative adversarial network model to obtain discriminant value data, the method further includes:

The enhanced lung sound data are processed using the trained lung sound phase correction model to obtain phase-corrected lung sound data.

The lung sound phase correction model is trained using the clean lung sound data containing white noise.

In one embodiment, a lung sound phase correction model with a Unet structure is added after the lung sound enhancement model. As shown in FIG5 , the clean lung sound data X is mixed with white noise at different signal-to-noise ratios to obtain X ^* , and the spectrum of X is combined with the phase of X ^* to generate Will As the training data of the lung sound phase correction model, the lung sound phase correction model is trained with the corresponding lung sounds in X as labels. The training is completed when the function converges.

In another embodiment of the present application, after the step of processing the enhanced lung sound data using the trained lung sound phase correction model to obtain phase-corrected lung sound data, the method further includes:

The trained signal-to-noise ratio prediction model is used to process the lung sound data to be enhanced and the enhanced lung sound data to obtain corresponding signal-to-noise ratio prediction data.

In one embodiment, the signal-to-noise ratio prediction model is a support vector regression model (SVR), which extracts the Mel-frequency cepstral coefficients (MFCC) of lung sounds with different signal-to-noise ratios, uses the MFCC of lung sounds as the input of the SVR model, and uses its original signal-to-noise ratio as a label to train the SVR regression model in a self-supervised manner. After the SVR model converges, the MFCC of the lung sounds is input into the model, and the predicted value of the model is used as the predicted signal-to-noise ratio of the lung sounds.

By predicting the lung sound signal-to-noise ratio, it is easy to know the quality of lung sound enhancement, so that when the quality is poor, defects can be checked and remedied in time to improve the enhancement effect of lung sounds.

In another embodiment of the present application, a lung sound enhancement system is also disclosed. As shown in FIG6 , a lung sound enhancement system includes an acquisition module 1 for acquiring lung sound data to be enhanced;

Lung sound enhancement module 2, used to convert the lung sound data to be enhanced by using the trained lung sound enhancement model to obtain enhanced lung sound data;

The lung sound enhancement model is constructed based on a first variational autoencoder, a second variational autoencoder, and a cycle consistency loss model, and is trained with clean lung sound data and noisy lung sound data to obtain enhanced lung sound data.

In one embodiment, the system further includes a first training module, which is used to train the first variational autoencoder using the clean lung sound data before converting the lung sound data to be enhanced using the trained lung sound enhancement model to obtain the enhanced lung sound data, so as to obtain the trained first variational autoencoder and first feature vector data corresponding to the first variational autoencoder;

A second training module is used to train the second variational autoencoder through the noisy lung sound data and the first variational autoencoder that has completed the training, to obtain second feature vector data corresponding to the second variational autoencoder, a mapping relationship between the first feature vector data and the second feature vector data, and reconstructed noisy data; the second feature vector data, the mapping relationship, and the reconstructed noisy data are obtained under the constraints of the cycle consistency loss model;

The iteration module is used to determine that the training of the lung sound enhancement model is completed when a preset matching condition is met between the reconstructed noisy data and the noisy lung sound data.

In one embodiment, the second training module includes a first encoding unit, configured to encode the noisy lung sound data using encoder 2 of the second variational autoencoder to obtain the second feature vector data;

A first decoding unit is configured to decode the second feature vector data using a decoder of the first variational autoencoder that has completed training, according to a mapping relationship between the first feature vector data and the second feature vector data constructed by the cycle consistency loss model, to obtain purified lung sound data;

a second encoding unit, configured to encode the purified lung sound data using encoder 1 of the first variational autoencoder that has been trained to obtain the first feature vector data;

The second decoding unit is used to decode the first feature vector data using the second decoder of the second variational autoencoder based on the mapping relationship to obtain the reconstructed noisy data.

In one embodiment, the system further comprises a generative adversarial network module, which is used to process the enhanced lung sound data using a preset generative adversarial network model to obtain discriminant value data after the enhanced lung sound data is obtained;

The generative adversarial network model includes a generator and a discriminator; the first variational autoencoder and the second variational autoencoder both correspond to the generative adversarial network model; the generative adversarial network model corresponding to the first variational autoencoder determines the decoder 1 as the generator; the generative adversarial network model corresponding to the second variational autoencoder determines the decoder 2 as the generator; after the loss function of the first variational autoencoder converges, the corresponding generative adversarial network model is trained; after the loss function of the second variational autoencoder converges, the corresponding generative adversarial network model is trained.

In one embodiment, the decoder 1, the decoder 2 and the discriminator all include a two-layer multi-head self-attention model.

In one embodiment, the system further includes a correction module for processing the enhanced lung sound data using a trained lung sound phase correction model to obtain phase-corrected lung sound data after the enhanced lung sound data is processed using a preset generative adversarial network model to obtain discriminant value data;

The lung sound phase correction model is trained using the clean lung sound data containing white noise.

In one embodiment, the system further includes a signal-to-noise ratio prediction module, which is used to process the enhanced lung sound data using the trained lung sound phase correction model to obtain phase-corrected lung sound data, and then use the trained signal-to-noise ratio prediction model to process the lung sound data to be enhanced and the enhanced lung sound data to obtain corresponding signal-to-noise ratio prediction data.

It should be noted that the above description of the embodiment of the lung sound enhancement system is similar to the above method description. For technical details not disclosed in the embodiment of the lung sound enhancement system of the present application, those skilled in the art should refer to the description of the embodiment of the method of the present application for understanding.

It should be noted that in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, an embodiment of the present application also discloses a storage medium storing a computer program that can be loaded by a processor and execute the above method.

The embodiment of the present application also discloses a computer device, as shown in FIG7, including a processor 100, at least one communication bus 200, a user interface 300, at least one external communication interface 400 and a memory 500. The communication bus 200 is configured to realize connection and communication between these components. The user interface 300 may include a display screen, and the external communication interface 400 may include a standard wired interface and a wireless interface. The memory 500 stores a lung sound enhancement method. The processor 100 is used to adopt the above method when executing the lung sound enhancement method stored in the memory 500.

The above description of the computer device and storage medium embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the computer device and storage medium embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

A person skilled in the art will understand that all or part of the steps of the above method embodiment can be implemented by a program The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the aforementioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, magnetic disks or optical disks.

Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims (10)

A lung sound enhancement method, comprising: Acquire lung sound data to be enhanced; Using the trained lung sound enhancement model to transform the lung sound data to be enhanced, to obtain enhanced lung sound data; The lung sound enhancement model is constructed based on a first variational autoencoder, a second variational autoencoder, and a cycle consistency loss model, and is trained by clean lung sound data and noisy lung sound data. The lung sound enhancement method according to claim 1, wherein, before converting the lung sound data to be enhanced using the trained lung sound enhancement model to obtain enhanced lung sound data, the method further comprises: Training the first variational autoencoder using the clean lung sound data to obtain the trained first variational autoencoder and first feature vector data corresponding to the first variational autoencoder; The second variational autoencoder is trained by using the noisy lung sound data and the first variational autoencoder that has completed training, to obtain second feature vector data corresponding to the second variational autoencoder, a mapping relationship between the first feature vector data and the second feature vector data, and reconstructed noisy data; the second feature vector data, the mapping relationship, and the reconstructed noisy data are obtained under the constraints of the cycle consistency loss model; When a preset matching condition is satisfied between the reconstructed noisy data and the noisy lung sound data, it is determined that the training of the lung sound enhancement model is completed. The lung sound enhancement method according to claim 2, wherein the step of training the second variational autoencoder using the noisy lung sound data and the first variational autoencoder that has completed training, obtaining the second feature vector data corresponding to the second variational autoencoder, the mapping relationship between the first feature vector data and the second feature vector data, and reconstructing the noisy data comprises: Using encoder 2 of the second variational autoencoder to encode the noisy lung sound data to obtain the second feature vector data; According to the mapping relationship between the first feature vector data and the second feature vector data constructed by the cycle consistency loss model, the second feature vector data is decoded by using the decoder 1 of the first variational autoencoder that has completed training to obtain purified lung sound data; Encoding the purified lung sound data using encoder 1 of the first variational autoencoder that has been trained to obtain the first feature vector data; Based on the mapping relationship, the first feature vector data is decoded using decoder 2 of the second variational autoencoder to obtain the reconstructed noisy data. The lung sound enhancement method according to claim 3, wherein after obtaining the enhanced lung sound data, the method further comprises: Processing the enhanced lung sound data using a preset generative adversarial network model to obtain discriminant value data; The generative adversarial network model includes a generator and a discriminator; the first variational autoencoder and the second variational autoencoder both correspond to the generative adversarial network model; the generative adversarial network model corresponding to the first variational autoencoder determines the decoder 1 as the generator; the generative adversarial network model corresponding to the second variational autoencoder determines the decoder 2 as the generator; after the loss function of the first variational autoencoder converges, the corresponding generative adversarial network model is trained; after the loss function of the second variational autoencoder converges, the corresponding generative adversarial network model is trained. The lung sound enhancement method as described in claim 4, wherein the decoder 1, the decoder 2 and the discriminator each include a two-layer multi-head self-attention model. The lung sound enhancement method according to claim 4, wherein, after the enhanced lung sound data is processed using a preset generative adversarial network model to obtain discriminant value data, the method further comprises: Processing the enhanced lung sound data using the trained lung sound phase correction model to obtain phase-corrected lung sound data; The lung sound phase correction model is trained using the clean lung sound data containing white noise. The lung sound enhancement method according to claim 6, wherein, after the enhanced lung sound data is processed using the trained lung sound phase correction model to obtain the phase-corrected lung sound data, the method further comprises: The trained signal-to-noise ratio prediction model is used to process the lung sound data to be enhanced and the enhanced lung sound data to obtain corresponding signal-to-noise ratio prediction data. A lung sound enhancement system, comprising an acquisition module for acquiring lung sound data to be enhanced; A lung sound enhancement module, used to convert the lung sound data to be enhanced by using the trained lung sound enhancement model to obtain enhanced lung sound data; The lung sound enhancement model is constructed based on a first variational autoencoder, a second variational autoencoder, and a cycle consistency loss model, and is trained by clean lung sound data and noisy lung sound data. A computer device comprises a memory and a processor, wherein the memory stores a lung sound enhancement method, and the processor is used to adopt the method described in any one of claims 1 to 7 when executing the lung sound enhancement method. A storage medium storing a computer program that can be loaded by a processor and execute the method according to any one of claims 1 to 7.