[go: up one dir, main page]

WO2023200955A1 - Détection et débruitage de sons pulmonaires anormaux et extraction d'un cycle respiratoire à partir d'un signal auditif - Google Patents

Détection et débruitage de sons pulmonaires anormaux et extraction d'un cycle respiratoire à partir d'un signal auditif Download PDF

Info

Publication number
WO2023200955A1
WO2023200955A1 PCT/US2023/018496 US2023018496W WO2023200955A1 WO 2023200955 A1 WO2023200955 A1 WO 2023200955A1 US 2023018496 W US2023018496 W US 2023018496W WO 2023200955 A1 WO2023200955 A1 WO 2023200955A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
computing devices
amplitude
determining
threshold value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2023/018496
Other languages
English (en)
Inventor
Ian Mitra McLane
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sonavi Labs Inc
Original Assignee
Sonavi Labs Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/659,205 external-priority patent/US20230329666A1/en
Priority claimed from US17/659,206 external-priority patent/US20230329643A1/en
Application filed by Sonavi Labs Inc filed Critical Sonavi Labs Inc
Publication of WO2023200955A1 publication Critical patent/WO2023200955A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B7/00—Instruments for auscultation
    • A61B7/003—Detecting lung or respiration noise
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B7/00—Instruments for auscultation
    • A61B7/02—Stethoscopes
    • A61B7/04—Electric stethoscopes
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients

Definitions

  • aspects relate generally to detecting a respiratory abnormality and a method of operation thereof.
  • CT computerized tomography
  • the problem can be partially resolved using digital auscultation to monitor a patient’s lungs.
  • Digital auscultation is a well-known method for assessing lung sounds.
  • a patient’s lung sounds can be monitored by recording the lung sounds, so that these sounds can be assessed to determine if any abnormal lung sounds can be heard.
  • These sounds can be, for example, crackling sounds in the patient’s lungs during inhalation and exhalation that can indicate severity of a respiratory disease or indicate progression of a disease.
  • aspects disclosed herein provide improved systems and methods for detecting and de-noising lung sounds obtained via digital auscultation.
  • the systems and methods also allow health care providers to more precisely detect important noise signals that are correlated to disease progression.
  • abnormal lung sounds such as crackles in a patient’s lungs can be detected, and the precise number of crackles during an inhalation and exhalation can be determined.
  • the systems and methods provide significant advantages over conventional systems because conventional systems do not allow for such precise detection and measurement of lung sounds indicative of abnormalities.
  • the systems and methods allow for improved ways of monitoring respiratory disease progression without the need for patients to obtain frequent CT scans, as is typically the case today for diseases such as cystic fibrosis.
  • the amount of radiation that patients are exposed to is significantly reduced. This reduction in exposure to radiation is a significant improvement in the way patients are treated.
  • a computer-implemented system and method for de-noising an auditory signal can implement a method to partition an auditory spectrogram representing the auditory signal into a plurality of windows of equal length timeframes, where each of the windows indicates a frequency response of the auditory signal within each of the timeframes.
  • the auditory signal can represent a lung sound.
  • Each of the windows can be processed using a neural network trained to remove unwanted noise signals from the auditory signal.
  • the processing can include: (i) identifying an odd number of consecutive windows, (ii) identifying a middle window from the odd number of consecutive windows, where the middle window is a window to have the unwanted noise signals removed, (iii) identifying an even number of windows preceding the middle window, (iv) identifying an even number of windows following the middle window, (v) inputting the middle window, the even number of windows preceding the middle window, and the even number of windows following the middle window into the neural network, and (vi) computing, using the neural network, a vector representing the auditory signal with the unwanted noise signals removed.
  • a computer-implemented system and method for decomposing an auditory signal into sub-components can implement a method to filter the auditory signal by performing a wavelet transform, where the wavelet transform utilizes a wavelet representing a sound indicating a respiratory abnormality.
  • the wavelet transform extracts a signal from the auditory signal indicating the respiratory abnormality.
  • the system and method can determine whether a signal amplitude for the extracted signal is above a predetermined threshold value. Based on determining the signal amplitude is above the predetermined threshold value, the extracted signal can be stored as an instance of the respiratory abnormality.
  • the extracted signal can be stored as an instance indicating no respiratory abnormality.
  • the amplitude or width of the wavelet can be adjusted and the aforementioned processes can be performed using the amplitude adjusted or width adjusted wavelet. The purpose of doing this is to capture any variations of the sound indicating the respiratory abnormality.
  • a system and method for extracting a respiratory cycle can implement a method to receive an auditory signal representing a vesicular sound.
  • the vesicular sound refers to a patients breathing with sub-components representing a respiratory abnormality removed.
  • the removal of the sub-components indicating a respiratory abnormality can be done using the systems and methods described with respect to the second aspect described above.
  • the method can further partition the auditory signal into segments.
  • a transformation to each of the segments can be applied to determine a signal envelope.
  • a moving average window to the signal envelope can be applied to obtain an averaged signal envelope.
  • a transformation can be applied to each of the segments to obtain a frequency response of the auditory signal within each of the segments.
  • the frequency response can be summed across the segments to obtain a summed frequency response.
  • An inverse transformation can be applied to the summed frequency response to obtain an averaged signal envelope.
  • a point where the averaged signal envelope initially has an amplitude greater than a threshold value can be identified.
  • a mean value for the amplitude of the averaged signal envelope for a period of time after the point can be determined.
  • the method can further determine whether the mean value is greater than twice the threshold value. Based on determining that the mean value is greater than twice the threshold value, the point can be identified as a start of a respiratory cycle. In aspects, a further point where the averaged signal envelope is less than the threshold value can be identified. A further mean value for the amplitude of the averaged signal envelope for a further period of time prior to the further point can be determined. The method can determine whether the further mean value is greater than twice the threshold value.
  • the further point can be identified as an end of the respiratory cycle.
  • a minimum point for the amplitude of the averaged signal envelope between the start of the respiratory cycle and the end of the respiratory cycle can be determined.
  • the minimum point can be identified as a start of an expiration event.
  • the respiratory cycle can be used to determine where in the respiratory cycle respiratory abnormalities such as crackles occur.
  • a system and method for counting respiratory abnormalities can implement a method to receive an auditory signal representing respiratory abnormality sounds.
  • the auditory signal representing the respiratory abnormality sounds can be extracted using the systems and methods described with respect to the second aspect above.
  • the auditory signal can, for example, be an audio signal with only the crackle sub -components.
  • the method can determine whether an amplitude for the auditory signal is above an inspiration threshold.
  • the inspiration threshold refers to a threshold value above which the system can determine that the signal represents a crackle during inhalation. Based on determining the amplitude is above the inspiration threshold, an instance of a respiratory abnormality can be identified.
  • the method can further determine whether an amplitude for the auditory signal is above an expiration threshold.
  • the expiration threshold refers to a threshold value above which the system can determine that the signal represents a crackle during an exhalation. Based on determining the amplitude is above the expiration threshold, a further instance of the respiratory abnormality can be identified.
  • FIG. 1 is an exemplary control flow for a system for detecting and de-noising an auditory signal according to aspects.
  • FIG. 2A is an exemplary control flow for a deep learning de-noising network used to de-noise the auditory signal according to aspects.
  • FIGS. 2B-2C are exemplary methods for performing the de-noising the auditory signal using the deep learning and de-noising network according to aspects.
  • FIG. 3 A is an exemplary control flow for how the auditory signal is decomposed to its components using a wavelet packet decomposition process according to aspects.
  • FIGS. 3B-3D are exemplary methods for decomposing an auditory signal into its components using a wavelet packet decomposition process according to aspects.
  • FIG. 4A is an exemplary control flow for extracting a respiratory cycle according to aspects.
  • FIGS. 4B-4E are exemplary methods for extracting a respiratory cycle according to aspects.
  • FIG. 4F shows an exemplary method for counting respiratory abnormalities according to aspects.
  • FIG. 5 shows an example digital stethoscope and base station that can be used to implement the functions of the system according to aspects.
  • FIG. 6 shows an exemplary architecture of the digital stethoscope according to aspects
  • FIG. 7 shows an exemplary architecture of the base station according to aspects.
  • FIG. 8 shows exemplary components of the digital stethoscope according to aspects.
  • FIG. 9 shows exemplary components of the base station according to aspects.
  • FIG. 1 shows a control flow for a system 100 for detecting and de-noising an auditory signal 102 according to aspects.
  • the auditory signal 102 can be, for example, a recording of a lung sound.
  • the auditory signal 102 can be obtained via a digital stethoscope and/or other computing device on which the auditory signal 102 can be recorded and/or stored.
  • the computing device can be, for example, a computer, server, etc.
  • the raw signal will likely include unwanted noise signals due to external noise, such as motion artifacts from moving a digital stethoscope, background noise captured by a microphone recording the auditory signal 102, etc.
  • the control flow can proceed initially to de-noise the auditory signal 102.
  • the control flow can proceed to a preprocessing stage 104.
  • the purpose of the preprocessing stage 104 is to put the auditory signal 102 in a form to simplify signal processing at later stages of the control flow, and to remove unnecessary information that is irrelevant to extracting signals indicating respiratory abnormalities and/or is irrelevant to extracting a respiratory cycle.
  • the preprocessing stage 104 can include the auditory signal 102 being low-pass filtered and down-sampled.
  • the filtering can be performed using any known signal processing filter, for example, a Butterworth filter, a Chebyshev filter, an Elliptic filter, a Bessel filter, etc.
  • the auditory signal 102 can be low-pass filtered with a fourth-order Butterworth filter at a 4 kHz cutoff.
  • the filtering can remove any unwanted noise signals above the cutoff because typically normal respiratory sounds signals are found between 50-2500 Hz, and sounds representing respiratory abnormalities such as crackles, wheezes, stridor, squawks, or rhonchi exhibit frequency profiles below 4000 Hz.
  • the down-sampling can be performed to reduce the size of the auditory signal 102 so that it can be stored in bandwidth limited systems and processed quickly.
  • the auditory signal 102 can be down-sampled from 44.1 kHz to 8 kHz.
  • control can pass to a motion artifact detection module 106.
  • the motion artifact detection module 106 enables the detection and removal of unwanted noise signals due to motion artifacts.
  • Motion artifacts refer to noise signals generated due to the movement of a sensor or microphone when recording the auditory signal 102. These motion artifacts can occur, for example, when a digital stethoscope that records the auditory signal 102 is moved around due to patient movements.
  • Motion artifacts are characterized by being broadband signals that occur over a short period of time. Therefore, they can be misclassified as an auditory abnormality such as a crackle.
  • an auditory spectrogram representation of the auditory signal 102 is generated.
  • the auditory spectrogram refers to a two dimensional representation of the auditory signal 102 showing the frequencies of a signal as it varies with time.
  • the auditory signal 102 can be partitioned into windows with an overlap. For example, these windows can be 10 millisecond (ms) windows with 90% overlap with each other. These windows can be converted into a frequency domain representation using a Fast Fourier Transformation (FFT) with 265 sample points (i.e., a 256-point FFT).
  • FFT Fast Fourier Transformation
  • regions of interest are identified that are likely to be signals caused by motion artifacts.
  • these regions of interest are identified as those that contain high spectral content above 1 kHz, with a total span greater than 2 kHz.
  • a threshold can be defined by the total average energy above 1 kHz of the entire signal. The number of consecutive frequency bands above the threshold quantifies a spectral span.
  • consecutive frames of 10 to 100 ms exhibiting similar high-energy content are identified as regions that are likely to be motion artifacts. These identified regions are then removed.
  • the auditory signal 102 and control can pass to a deep learning and de-noising network 108 to further de-noise the auditory signal 102.
  • the further noise to be removed can be noise related to environmental conditions, such as background noise that is recorded as a part of recording the lung sounds.
  • the deep learning and de-noising network 108 can include a neural network 206 (shown in FIG. 2A) that is trained to remove noise related to environmental conditions.
  • the de-noising scheme employed by the deep learning and de- noising network 108 is shown in more detail in FIG. 2A.
  • FIG. 2A shows the control flow 200 for the deep learning and de-noising network 108 used to de-noise the auditory signal 102 according to aspects.
  • control flow 200 can begin by having an auditory spectrogram 202 input into the deep learning and de- noising network 108.
  • the auditory spectrogram 202 can represent the auditory signal 102 resulting from the processing performed by the motion artifact detection module 106.
  • the auditory spectrogram 202 can be the spectrogram output by the motion artifact detection module 106 with the noise due to the motion artifacts removed.
  • the deep learning and de-noising network 108 can partition the auditory spectrogram 202 into a plurality of windows of equal length timeframes.
  • these windows are labeled ⁇ 1,2,..., 17 ⁇ .
  • the 17 windows are merely exemplary and representative of the number of windows the auditory spectrogram 202 can be partitioned into. There can be more or less windows. Each of these windows can indicate a frequency response of the auditory signal 102 within each of the timeframes.
  • the deep learning and de-noising network 108 can begin processing each of the windows to de-noise each by using a neural network 206 trained to remove unwanted noise signals from the auditory signal 102.
  • the neural network 206 can be trained using recordings with varying levels of stationary and non- stationary ambient noise in order to train the neural network 206 to detect background/environmental noise and differentiate that noise from lung sounds.
  • each window can be processed by the neural network 206 sequentially.
  • the processing can remove noise from each of the windows that are processed.
  • the general procedure by which the windows are processed is as follows. First, an odd number of consecutive windows is identified. Second, a middle window from the odd number of consecutive windows can be identified, where the middle window is a window to have the unwanted noise signals removed. Third, an even number of windows preceding the middle window can be identified. Fourth, an even number of windows following the middle window can be identified. Fifth, the middle window, the even number of windows preceding the middle window, and the even number of windows following the middle window can be input into the neural network 206. Sixth, the neural network 206 can then proceed to process the input windows to remove noise from the middle window by analyzing the middle window in the context of the surrounding windows to determine what signals in the middle window are likely to be background/environmental noise.
  • the above described process is the general manner in which the windows are denoised.
  • the neural network 206 processes each of the windows, special cases need to be handled where the window being processed does not have an even number of windows either preceding it or following it.
  • the window labeled “1” does not have an even number of windows preceding it because it is the first window.
  • the window labeled “17” does not have an even number of windows following it because it is the last window.
  • the output of the processing done by the neural network 206 can be a vector 208 representing the window processed but with the unwanted noise signals removed.
  • the neural network 206 can have a full set of windows with their background/environmental noise removed.
  • the architecture of the neural network 206 can have the neural network 206 have three hidden layers of sizes: 1024, 1024, and 256.
  • the odd number of consecutive windows can be varied.
  • the odd number of consecutive windows to be processed can equal nine.
  • the even number of windows preceding and following the middle window can also be varied.
  • the even number of windows can equal four.
  • the odd number of consecutive windows can overlap. This overlap allows the neural network 206 to process the windows by recognizing the continuity between windows and frequency responses therein. By how much the windows overlap can be varied.
  • the windows can overlap with one another by 90%. That is, the preceding window can overlap with the following window, or vice versa, by 90%.
  • FIGS. 2B and 2C show methods 210 and 216 for performing the de-noising of the auditory signal 102 using the deep learning and de-noising network 108 according to aspects.
  • FIG. 2B shows a two-step process by which the auditory signal 102 is de-noised.
  • the auditory spectrogram 202 representing the auditory signal 102 is partitioned into a plurality of windows of equal length timeframes, where each of the windows indicates a frequency response of the auditory signal 102 within each of the timeframes.
  • each of the windows is processed using a neural network 206 trained to remove unwanted noise signals from the auditory signal 102.
  • FIG. 2C is a method 216 for how the neural network 206 processes each of the windows.
  • an odd number of consecutive windows is identified.
  • a middle window from the odd number of consecutive windows can be identified, where the middle window is a window to have the unwanted noise signals removed.
  • an even number of windows preceding the middle window can be identified.
  • an even number of windows following the middle window can be identified.
  • the middle window, the even number of windows preceding the middle window, and the even number of windows following the middle window can be input into the neural network 206.
  • the neural network 206 can then proceed to process the input windows to remove noise from the middle window by analyzing the middle window in the context of the surrounding windows to determine what signals in the middle window are likely to be background/environmental noise. Based on the processing, the neural network 206 can compute a vector 208 representing the auditory signal 102 with the unwanted noise signals removed.
  • the above processes described with respect to FIGS. 2A-2C are in improvement over conventional systems because they apply non-causal techniques to de-noise the auditory signal 102. That is, as opposed to conventional systems, the deep learning and de-noising network 108 uses both information from the past and the future of the window to be de-noised to perform the de-noising. Typically, systems only use past or present information (i.e., are causal). The non-causal nature allows the deep learning and de- noising network 108 to better determine what frequencies are likely to correspond to background/environmental noise.
  • the deep learning and de-noising network 108 can perform the de-noising processing over a set of overlapping windows and can correlate past and future frequencies with one another to better assess which of the frequencies from the past and future are likely to be noise signals based on how they evolve over time.
  • FIG. 3 A shows a control flow 300 for how the auditory signal 102 is decomposed to its components using a wavelet packet decomposition module 110 according to aspects.
  • the purpose of decomposing the auditory signal 102 is to extract portions of the auditory signal 102 that indicate respiratory abnormalities, such as crackles, and those portions that do not. This information can then be used to determine where and how many times in a patient’s respiratory cycle the respiratory abnormality occurs.
  • the decomposition process can begin by having the wavelet packet decomposition module 110 receive the auditory signal 102.
  • the auditory signal 102 is converted back or is otherwise in a time-series format, where the amplitude of the lung sounds recorded is plotted over time (as opposed to spectrogram representation where frequency of the lung sounds is plotted versus time).
  • the auditory signal 102 can have portions related to inhalation/inspiration 302 and an exhalation/expiration 304.
  • the wavelet packet decomposition module 110 will apply a plurality of mother wavelets 306 to the auditory signal 102 to filter for instances of a respiratory abnormality.
  • the mother wavelets 306 refer to archetypal signals that represent a respiratory abnormality.
  • mother wavelets 306 are shown. These mother wavelets 306 can represent, for example, what a typical crackle sound will look like if plotted in time series.
  • the mother wavelets 306 can differ from each other by each having different scaling. For example, amplitudes (represented as “h” in FIG. 3A) and widths (represented as “w” in FIG. 3 A) can be adjusted for each of the mother wavelets 306 so that the archetypal crackle represented by the mother wavelets 306 differs slightly. The purpose of doing this is to capture all the variations of a crackle sound from the auditory signal 102.
  • the wavelet packet decomposition module 110 can then apply each of the mother wavelets 306 to the auditory signal 102 through a wavelet packet transform process.
  • a wavelet packet transform is performed, thus the details of the transform will not be discussed in detail.
  • a further signal 308 can be generated indicating all the potential crackle sounds that occur in the auditory signal 102.
  • FIG. 3 A shows further signal 308 indicating potential crackles at tl and t2.
  • a predetermined threshold 310 can be defined such that if the amplitudes of the potential crackle sounds are above the predetermined threshold 310, the potential crackle sounds can be stored as an instance indicating a respiratory abnormality, and if the amplitudes of the potential crackle sounds are below the predetermined threshold 310 the potential crackle sounds can be stored as an instance indicating no respiratory abnormality.
  • the further signal 308 can be compared to the predetermined threshold 310 to see at what points the amplitudes of the further signal 308 are above or below the predetermined threshold 310. By way of example FIG 3 A, shows the results of this comparison.
  • Plot 312 shows stored instances of when the amplitudes are above the predetermined threshold 310 and a further plot 314 shows stored instances of when the amplitudes are below the predetermined threshold 310.
  • the auditory signal 102 can be decomposed into its sub-components indicating when a crackle occurs within the auditory signal 102 or does not.
  • an inverse wavelet transformation process can be performed on the decomposed signals (e.g., those shown by example plots 312 and further plot 314) to reconstruct the auditory signal 102. This can be done for signals extracted and decomposed for all mother wavelets 306, which can then be reconstructed and combined to reconstruct the auditory signal 102 by using a mean value for the combined signal.
  • the purpose of reconstructing the auditory signal 102 is so that the auditory signal 102 that was received by the wavelet packet decomposition module 110 can be used in further processes of the system 100 to determine a respiratory cycle for the patient. How this is performed will be described further below. A person skilled in the art will know how to perform an inverse wavelet transformation, thus the details of the inverse transform will not be discussed in detail. For the purposes of discussion with respect to FIG. 3 A, it is assumed that the auditory signal 102 is reconstructed.
  • the wavelet packet transform process performed by the wavelet packet decomposition module 110 is unique in several ways.
  • the transform used by the wavelet packet decomposition module 110 can be obtained by iterating the transform on both the detail (wavelet) and approximation (scaling) coefficients of the mother wavelets 306.
  • each of the coefficients w4(m) can be scored based on equation (1) below.
  • Equation (1) k is the standard deviation of the wavelet coefficients in the th subband of the level j.
  • Pi is a multiplicative factor. In a preferred aspect, Pi equals three, and is determined empirically using training data. In aspects, a total score can be quantified for all k subbands of the level j using equation (2) below.
  • the predetermined threshold 310 can be defined using equation (3) below.
  • Pi can be another multiplicative factor.
  • Pi equals 2.5 and is determined empirically using training data.
  • the wavelet packet transform can be applied to the auditory signal 102 by partitioning the auditory signal 102 into a plurality of overlapping windows and applying each wavelet to the windows of length L.
  • the windows can overlap by a percentage, for example, 75%.
  • FIGS. 3B-3D show methods 316, 328, and 340 for decomposing an auditory signal 102 into its components using a wavelet packet decomposition process according to aspects.
  • Methods 316, 328, and 340 can be performed using the wavelet packet decomposition module 110.
  • FIG. 3B shows the steps of method 316.
  • the auditory signal 102 can be filtered using a wavelet transform, where the wavelet transform utilizes a mother wavelet representing a sound indicating a respiratory abnormality. Using the mother wavelet, the wavelet transform extracts a signal from the auditory signal 102 indicating the respiratory abnormality.
  • the signal amplitude can be stored as an instance of the respiratory abnormality.
  • the signal amplitude can be stored as an instance indicating no respiratory abnormality.
  • an inverse wavelet transform can be performed to reconstruct the auditory signal 102.
  • Method 328 is similar to method 316 except it is performed on an amplitude adjusted mother wavelet. That is, it is performed on a mother wavelet similar to the one used in method 316 except with its amplitude adjusted.
  • the mother wavelet can have its amplitude adjusted.
  • the auditory signal 102 can be filtered using a wavelet transform, where the wavelet transform utilizes the amplitude adjusted wavelet representing a sound indicating a respiratory abnormality. Using the amplitude adjusted wavelet, the wavelet transform extracts a signal from the auditory signal 102 indicating the respiratory abnormality.
  • a determination can be made whether a signal amplitude for the extracted signal is above the predetermined threshold 310.
  • the signal amplitude can be stored as an instance of the respiratory abnormality.
  • the signal amplitude can be stored as an instance indicating no respiratory abnormality.
  • Fig. 3D shows steps of method 340.
  • Method 340 is similar to method 316 except it is performed on a width adjusted mother wavelet. That is, it is performed on a mother wavelet similar to the one used in method 316 except with its width adjusted.
  • the mother wavelet can have its width adjusted.
  • the auditory signal 102 can be filtered using a wavelet transform, where the wavelet transform utilizes the width adjusted wavelet representing a sound indicating a respiratory abnormality. Using the width adjusted wavelet, the wavelet transform extracts a signal from the auditory signal 102 indicating the respiratory abnormality.
  • the signal amplitude can be stored as an instance of the respiratory abnormality.
  • the signal amplitude can be stored as an instance indicating no respiratory abnormality.
  • control can pass to a respiratory cycle extraction module 112 shown in FIG. 1.
  • the respiratory cycle extraction module 112 enables the extraction of a respiratory cycle based on the auditory signal 102.
  • the auditory signal 102 can be the signal representing a vesicular sound.
  • the auditory signal 102 can be obtained using the processes described with respect to the wavelet packet decomposition module 110 (e.g., FIGS. 3A-3D).
  • FIG. 4A A more detailed view of the functioning of the respiratory cycle extraction module 112 is shown in FIG. 4A.
  • FIG 4A shows a control flow 400 for extracting a respiratory cycle using the respiratory cycle extraction module 112 according to aspects.
  • control flow 400 can begin by having the respiratory cycle extraction module 112 receive the auditory signal 102.
  • the auditory signal 102 is in time-series format.
  • the auditory signal 102 can be input into a segmentation module 402.
  • the segmentation module 402 can enable the partitioning of the auditory signal 102 into segments.
  • the segments refer to windows over a timeframe.
  • the segments can partition the auditory signal 102 into three-second windows and one-second window steps.
  • the auditory signal 102 can be transmitted to a transformation module 404.
  • the transformation module 404 can apply a transformation to each of the segments to determine a signal envelope for each of the segments.
  • the transformation can be either a Hilbert transformation or a FFT.
  • pn ⁇ n ⁇ represents the Hilbert transformed envelope for each window
  • /?( «») is a segmented window of the auditory signal 102
  • p n ⁇ ) is the instantaneous envelope for each segmented window.
  • control can pass to a moving average module 406.
  • the moving average module 406 can apply a moving average window to the signal envelope to obtain an averaged signal envelope. The purpose of doing this is to smooth out the signal envelope, to have a cleaner signal that produces a more pronounced local minima between breath cycles and less prominent minima between inspiration and expiration of a single cycle.
  • an autocorrelation of the signal envelope can be calculated using equation (5) below.
  • equation (5) x(//) represents the signal, and x(//+/) is a shifted/lagged version of the signal envelope.
  • R PP (T) represents the similarity with respect to the lag. Since the respiratory signals are periodic, the autocorrelation shows significant peaks when the lag is roughly equal to a single respiratory cycle length.
  • equation (5) can be used to estimate the respiratory rate as the average distance between peaks in R PP (T).
  • the estimated respiratory rate (which can be represented as R) can then be used to apply a lagging moving average window from [t - a * (1/R), t] at sample t, with a equal to 0.5, where a is determined empirically. Based on applying the moving average window, the averaged signal envelope can be obtained.
  • control can pass to the detection and extraction module 410.
  • the detection and extraction module 410 can use the averaged signal envelope to identify respiratory pauses indicative of the beginning and end of an inspiration (inhaling) or the beginning of an expiration (exhaling).
  • the detection and extraction module 410 can identify respiratory pauses.
  • a local minimum signal p m in(n) can be extracted from the averaged signal envelope using a moving minimum-value window centered at n with length of P * (1/R), with P equal to 0.5, where P is determined empirically.
  • the points can be identified as respiratory pauses where the averaged signal envelope equals p m in(n).
  • the detection and extraction module 410 can also determine a threshold value.
  • the threshold value can be used when detecting the beginning and end of a respiratory cycle. For example, when amplitudes of the auditory signal 102 are above the threshold value it can indicate a beginning of an inhalation or end of an inhalation, or the beginning of an expiration. How this is determined will be described further below.
  • the threshold value can be determined specifically for each auditory signal 102. For example, a recording-specific threshold value for a patient can be determined as the 75 th percentile of the amplitudes of p m in(ri), which can be empirically set based on training data.
  • a point in the auditory signal 102 can be identified where the averaged signal envelope initially has an amplitude greater than the threshold value.
  • a mean value for the amplitude of the averaged signal envelop for a period of time after that point can be determined.
  • the period of time can equal 0.5 seconds.
  • the beginning of the respiratory cycle i.e., beginning of inhalation/inspiration event
  • the end of the respiratory cycle (i.e., end of an expiration/exhale event) can be determined based on identifying a further point where the averaged signal envelope is less than the threshold value.
  • a further mean value for the amplitude of the averaged signal envelope for a further period of time prior to the further point can be determined.
  • the further period of time can equal 0.5 seconds.
  • the end of the respiratory cycle can be determined as the further point where the further mean value is greater than twice the threshold value.
  • the beginning of an expiration/exhale event can be determined by determining a minimum point for the amplitude of the averaged signal envelope between the start of the respiratory cycle and the end of the respiratory cycle, and identifying the minimum point as a start of the expiration/exhale event.
  • a respiratory cycle can be extracted by applying a Hilbert transformation to the auditory signal 102.
  • the auditory signal 102 can, similar to what was described with respect to the Hilbert transformation, be partitioned into segments.
  • each of the segments can have a FFT applied to it to obtain a frequency response of the auditory signal 102 within each of the segments.
  • the frequency response across all the segments can then be summed to obtain an averaged signal envelope in the frequency domain.
  • an inverse transformation can be applied to the summed frequency response.
  • the inverse transformation can be an inverse FFT.
  • the use of the FFT provides an improved method of extracting respiratory cycle because it provides a faster way to extract the respiratory cycle due to the nature of the FFT algorithm.
  • the aforementioned processes improve computers by providing a novel and fast method that allows computers to extract respiratory cycles from a patient’s lung sounds.
  • the extracted respiratory cycle can be used, along with other extracted data that will be described further below, to determine where in the respiratory cycle respiratory abnormalities occur. By being able to do this, the methods described can provide a fully automated way to analyze a patient’s lung sounds without the need for human interpretation.
  • the processes allow for the precise measurement of where respiratory abnormalities occur within a respiratory cycle, which cannot be done without the aid of these computer-implemented techniques. Additionally, the extraction of the respiratory cycle can be done using these methods on the fly and in real-time. Thus, doctors or care givers can obtain information about a patient’s breathing cycles in real-time from when they record the patient’s lung sounds to determine where and how often a respiratory abnormality happens with the patient’s respiratory cycle.
  • FIGS. 4B-4E are exemplary methods 412, 430, 440, and 446 of extracting a respiratory cycle according to aspects.
  • Methods 412, 430, 440, and 446 can be performed using the respiratory cycle extraction module 112.
  • Method 412 shown in FIG. 4B shows the steps when applying a Hilbert transformation to the auditory signal 102 to extract the respiratory cycle.
  • an auditory signal 102 can be received.
  • the auditory signal 102 can represent a vesicular sound.
  • the auditory signal 102 can be partitioned into segments.
  • a transformation to each of the segments can be applied to determine a signal envelope.
  • a moving average window can be applied to the signal envelope to obtain an averaged signal envelope.
  • the averaged signal envelope represents a smoothed out version of the signal envelope.
  • a point can be identified where the averaged signal envelope initially has an amplitude greater than a threshold value.
  • a mean value for the amplitude of the averaged signal envelope can be determined for a period of time after the point.
  • a determination can be made whether the mean value is greater than twice the threshold value.
  • the point can be identified as a start of the respiratory cycle.
  • Method 430 shown in FIG. 4C shows the steps for determining the end of a respiratory cycle.
  • a further point can be identified where the averaged signal envelope is less than the threshold value.
  • a further mean value for the amplitude of the averaged signal envelope for a further period of time prior to the further point can be determined.
  • a determination can be made whether the further mean value is greater than twice the threshold value.
  • the further point can be identified as an end of the respiratory cycle.
  • Method 440 shown in FIG. 4D shows the steps for determining the beginning on an expiration event within the respiratory cycle.
  • a minimum point for the amplitude of the averaged signal envelope between the start of the respiratory cycle and the end of the respiratory cycle can be determined.
  • the minimum point can be identified as the start of an expiration event.
  • Method 446 shown in FIG. 4E shows the steps when applying a FFT to the auditory signal 102 to extract the respiratory cycle.
  • the auditory signal 102 can be received.
  • the auditory signal 102 can represent a vesicular sound.
  • the auditory signal 102 can be partitioned into segments.
  • a transformation can be applied to each of the segments to obtain a frequency response of the auditory signal 102 within each of the segments.
  • the frequency response can be summed across the segments to obtain a summed frequency response.
  • an inverse transformation can be applied to the summed frequency response to obtain an averaged signal envelope.
  • the inverse transformation can be an inverse FFT.
  • a point can be identified where the averaged signal envelope initially has an amplitude greater than a threshold value.
  • a mean value for the amplitude of the averaged signal envelope for a period of time after the point can be determined.
  • a determination can be made whether the mean value is greater than twice the threshold value.
  • the point can be identified as the start of the respiratory cycle.
  • FIG. 4F shows an exemplary method 466 for counting respiratory abnormalities according to aspects.
  • method 466 can be performed by a crackle peak detection module 114 shown in FIG. 1.
  • the auditory signal 102 can be decomposed into its components indicating instances within the respiratory cycle where crackles occur.
  • the crackle peak detection module 114 can take the decomposed signals that are extracted, and perform the analysis needed to determine how many crackles occur within the respiratory cycle.
  • the crackle peak detection module 114 can receive an auditory signal 102 representing respiratory abnormality sounds.
  • the auditory signal 102 can represent a signal where only the respiratory abnormalities are present.
  • the inspiration threshold can represent a threshold value that can be predetermined empirically and represent a threshold value above which a sound can be classified as a crackle.
  • an instance of the respiratory abnormality can be identified.
  • the expiration threshold similar to the inspiration threshold can represent a threshold value that can be predetermined empirically and represent a threshold value above which a sound can be classified as a crackle.
  • the expiration threshold can relate to the period of time within the respiratory cycle when the patient is exhaling. Because the signal during exhaling has a different power distributions than during inhaling and the amplitudes of the signal during exhaling are smaller, the expiration threshold can be specifically defined such to determine threshold values above which crackles are determined. This is necessary because if the inspiration threshold is used, signals that might be crackles might not be captured because the threshold value would be too high.
  • the thresholds described can be customized for each patient and set based on an average power of the auditory signal 102 for the patient. In aspects, and as shown in step 476, based on determining the amplitude is above the expiration threshold, a further instance of the respiratory abnormality can be identified.
  • the system 100 described above can have its operations performed on hardware components such as a digital stethoscope.
  • a digital stethoscope 510 that can be used to implement the functions of system 100.
  • FIG. 5 shows an example digital stethoscope 510 and base station 518 that can be used to implement the functions of the system 100 according to aspects.
  • the digital stethoscope 510 is an acoustic device for detecting and analyzing noises from a patient’s body.
  • the patient can be, for example, a human or an animal.
  • the noises, from the patient’s body can be for example a cough, a wheeze, a crackle, a breathing pattern, a heartbeat, a chest motion representing a patient’s respiratory cycle, or a combination thereof.
  • the digital stethoscope 510 can include one or more components.
  • the digital stethoscope 510 can include a display unit 502, one or more microphones 506, and a first housing 508.
  • the display unit 502 can be any graphical user interface such as a display, a projector, a video screen, a touch screen, or any combination thereof that can present information detected or generated by the digital stethoscope 510 for visualization by a user of the system 100.
  • the display unit 502 can enable the visual presentation of information detected or generated by the digital stethoscope 510.
  • the display unit 502 can enable the visual presentation of the noises detected, by for example, displaying a plot of the sound frequencies detected over time, displaying a decibel level of the sounds detected, or displaying a value or visual indicator representing the classification of the noises generated, for example “normal” or “abnormal,” or display the number of respiratory abnormalities counted within a respiratory cycle.
  • the display unit 502 can display an indicator, such as a red colored light, or a message indicating that the noise is “abnormal.”
  • the display unit 502 can display an indicator, such as a green colored light, or a message indicating that the noise is “normal.”
  • the display unit 502 can further present other information generated by the digital stethoscope 510, such as a power level indicator indicating how much power the digital stethoscope has, a volume indicator indicating the volume level of output noises being output by the digital stethoscope 510, or a network connectivity indicator indicating whether the digital stethoscope 510 is connected to a device or computer network such as a wireless communication network or wired communication network.
  • a power level indicator indicating how much power the digital stethoscope has
  • a volume indicator indicating the volume level of output noises being output by the digital stethoscope 510
  • a network connectivity indicator indicating whether the digital stethoscope 510 is connected to a device or computer network such as a wireless communication network or wired communication network.
  • the display unit 502 can further include one or more buttons 526 that can be used by the user of the system 100 to enable interaction with the digital stethoscope 510.
  • the buttons 526 can provide functionality such as powering the digital stethoscope 510 on or off or enable the digital stethoscope 510 to start or stop recording the noises.
  • the digital stethoscope 510 can further include one or more microphones 506A and B. The microphones 506A and B enable the digital stethoscope 510 to detect and convert the noises into electrical signals for processing by the digital stethoscope 510, or a further device such as the base station 518.
  • Microphone 506A is mounted on a perimeter side of stethoscope 110 to detect noises external to the patient’s body.
  • the noises originating from external to the patient’s body can be for example background noise, white noise, or a combination thereof.
  • Microphone 506B may be mounted on a side reverse of display 102 and may detect noises originating from the patient’s body.
  • the microphones 506A and B can be standalone devices or can be arranged in an array configuration, where the microphones 506 operate in tandem to detect the noises.
  • each microphone in the array configuration can serve a different purpose.
  • each microphone in the array configuration can be configured to detect and convert into electrical signals the noises at different frequencies or within different frequency ranges such that each of the microphones 506 can be configured to detect specific noises.
  • the noises detected by the microphones 506 can be used to generate the values for classifying the noises as “normal” or “abnormal,” and can be further used to predict the respiratory event or respiratory condition in the future.
  • the digital stethoscope 510 can further have a first housing 508 enclosing the components of the digital stethoscope 510.
  • the first housing 508 can separate components of the digital stethoscope 510 contained within from other components external to the first housing 508.
  • the first housing 508 can be a case, a chassis, a box, or a console.
  • the components of the digital stethoscope 510 can be contained within the first housing 508.
  • some components of the digital stethoscope 510 can be contained within the first housing 508 while other components, such as the display unit 502, the microphones 506, the buttons 526, or a combination thereof, can be accessible external to the first housing 508.
  • the aforementioned are merely examples of components that can be contained in or on the first housing 508 and are not meant to be limiting. Further discussion of other components of the digital stethoscope 510 will be discussed below.
  • a base station 518 can also be included to be used in conjunction with the digital stethoscope 510 .
  • the base station 518 is a special purpose computing device that enables computation and analysis of the noises obtained by the digital stethoscope 510 in order to detect the respiratory abnormality, or to predict the respiratory event or respiratory condition in the future.
  • the base station 518 can provide additional or higher performance processing power compared to the digital stethoscope 510.
  • the base station 518 can work in conjunction with the digital stethoscope 510 to detect, amplify, adjust, and analyze noises from a patient’s body by, for example, providing further processing, storage, or communication capabilities to the digital stethoscope 510.
  • the base station 518 can work as a standalone device to detect, amplify, adjust, and analyze noises to detect the respiratory abnormality, or to predict the respiratory event or respiratory condition in the future.
  • the base station 518 can analyze the noises captured by digital stethoscope 510. For example, in aspects, the base station 518 can generate values classifying the noises detected as “normal” or “abnormal.” The collection, filtering, comparison, and classification of the noises by the base station 518 will be discussed further below.
  • the base station 518 can include one or more components.
  • the base station 518 can include a charging pad 514, one or more air quality sensors 516, a contact sensor 520, and a second housing 512.
  • the charging pad 514 can enable the electric charging of the digital stethoscope 510, through inductive charging where an electromagnetic field is used to transfer energy between the charging pad 514 and a further device, such as the digital stethoscope 510, using electromagnetic induction.
  • the charging pad 514 can enable electric charging of the digital stethoscope 510 upon detecting contact or coupling, via the contact sensor 520, between the digital stethoscope 510 and the charging pad 514.
  • the contact sensor 520 can detect a weight or an electromagnetic signal produced by the digital stethoscope 510 on the charging pad 514, and upon sensing the weight or the electromagnetic signal enable the induction process to transfer energy between the charging pad 514 and the digital stethoscope 510.
  • the contact sensor 520 can detect an electric current or a magnetic field from one or more components of the digital stethoscope 510 and enable the induction process to transfer energy between the charging pad 514 and the digital stethoscope 510.
  • the contact sensor 520 is a device that senses mechanical or electromagnetic contact and gives out signals when it does so.
  • the contact sensor 520 can be, for example, a pressure sensor, a force sensor, strain gauges, piezoresistive/piezoelectric sensors, capacitive sensors, elastoresistive sensors, torque sensors, linear force sensors, an inductor, other tactile sensors, or a combination thereof configured to measure a characteristic associated with contact or coupling between the digital stethoscope 510 and the charging pad 514.
  • the contact sensor 520 can output a contact measure 522 that represents a quantified measure, for example, a measured force, a pressure, an electromagnetic force, or a combination thereof corresponding to the coupling between the digital stethoscope 510 and the charging pad 514.
  • the contact measure 522 can detect one or more force or pressure readings associated with forces applied by the digital stethoscope 510 on the charging pad 514.
  • the contact measure 522 can further detect one or more electric current or magnetic field readings associated with placing the digital stethoscope 510 in proximity of the charging pad 514.
  • the base station 518 can further include one or more air quality sensors 516.
  • the air quality sensors 516 are devices that detect and monitor the presence of air pollution in a surrounding area. Air pollution refers to the presence of or introduction into the air of a substance which has harmful or poisonous effects on the patient’s body.
  • the air quality sensors 516 can detect the presence of particulate matter or gases such as ozone, carbon monoxide, sulfur dioxide, nitrous oxide, or a combination thereof that can be poisonous to the patient’s body, and in particular poisonous to the patient’s respiratory system.
  • the base station 518 can determine whether the amount of air pollution poses a health risk to the patient by, for example, comparing the levels of air pollution to a pollution threshold 524 to determine whether the levels of air pollution in the surrounding area of the base station 518 pose a health risk to the patient.
  • the pollution threshold 524 refers to a pre-determined level for particulate matter or gases measured in micrograms per cubic meter (pg/m 3 ), parts per million (ppm), or parts per billion (ppb), that if exceeded poses a health risk to the patient [0090] For example, in aspects, if the air quality sensors 516 detect the presence of sulfur dioxide above 75 ppb in the air surrounding the base station 518, the base station 518 can determine that the air pollution in the surrounding area poses a health risk to the patient.
  • the detection of air pollution can further be used for detecting the respiratory abnormality or to predict the respiratory event or respiratory condition in the future in the patient by allowing the system 100 to determine what factors are contributing to the “normal” or “abnormal” classification of the noises, or what factors are contributing to the data detected and generated by the system 100 which can be used to predict a respiratory event or respiratory condition in the future.
  • the base station 518 can further have a second housing 512 enclosing the components of the base station 518.
  • the second housing 512 can separate components of the base station 518 contained within, from other components external to the second housing 512.
  • the second housing 512 can be a case, a chassis, a box, or a console.
  • the components of the base station 518 can be contained within the second housing 512.
  • some components of the base station 518 can be contained within the second housing 512 while other components, such as the charging pad 514 or the air quality sensors 516 can be accessible external to the second housing 512.
  • the aforementioned are merely examples of components that can be contained in or on the second housing 512 and are not meant to be limiting. Further discussion of other components of the base station 518 will be discussed below.
  • FIG. 6 shows an exemplary architecture of the digital stethoscope 510 according to aspects.
  • the digital stethoscope 510 can include, alternatively or additionally:
  • a grip ring 640 located around a first upper portion 642 of the first housing 508 which provides a gripping surface for a user of the digital stethoscope 510 to hold the digital stethoscope 510;
  • a glass lens 644 of the display unit 502 which protects the display components, such as for example liquid crystal displays (LCD) of the display unit 502.
  • the glass lens 644 can sit on top of a housing gasket 646, which stabilizes and holds the glass lens 644;
  • a display housing unit 648 on which the housing gasket 646 sits and which contains the components of the display unit 502, such as for example the LCDs; • a flex backing 650 which on which the display housing 648 sits and which provides stability for the display housing 648;
  • a first printed circuit board assembly 664 which can hold the circuitry, including any processors, memory components, active and passive components, or a combination thereof, of the digital stethoscope 510;
  • components such as an O-ring 672 and one or more coils 666 that couple the microphone assembly 670 to the first printed circuit board assembly 664.
  • a diaphragm membrane 682 which forms the bottom surface of the digital stethoscope 510, and which is coupled to the first bottom portion 674 of the first housing 508 with one or more second screws 676 and one or more washers 678;
  • a diaphragm ring 680 coupled to the diaphragm membrane 682, which provides a gripping surface for the first bottom portion 674 of the digital stethoscope 510, such that the digital stethoscope 510 does not slip when placed on a surface.
  • FIG. 7 shows an exemplary architecture of the base station 518 according to aspects.
  • the base station 518 can include, alternatively or additionally:
  • FIG. 8 shows exemplary components of the digital stethoscope 510 according to aspects.
  • FIG. 8 shows an aspect that includes a control unit 810 and a storage unit 818.
  • the control unit 810 can include a processor 814 and a FPGA 816.
  • the storage unit 818 can include a DRAM 854.
  • the processor 814 and the FPGA 816 can be coupled using a control interface 812, which can include bus lines to transfer data.
  • the storage unit 818 can be coupled to the control unit 810 using a storage interface 820, which can include bus bars to transfer data.
  • the DRAM 854 can be coupled, via the storage interface 820 to the processor 814.
  • the processor 814, the FPGA 816, and the DRAM 854 can work in conjunction to process auditory signals detected by the microphones 506.
  • the processor 814 can act as a controller and control the coordination, communications, scheduling, and transfers of data between the FPGA 816, the DRAM 854, or other components of the digital stethoscope 510.
  • the processor 814 can receive the auditory signal 102 from the microphones 506, and transfer the auditory signal 102 to the FPGA 816 for further processing.
  • the FPGA 816 can transfer the output or data generated as a result of its operations back to the processor 814, which can further transfer the output or data to the DRAM 854 for storage.
  • the FPGA 816 can perform the processing of the auditory signal 102.
  • the FPGA 816 can include one or more logic blocks, including one or more reconfigurable logic gates, that can be pre-programmed or configured to perform calculations or computations on the auditory signal 102, and to generate output or data to detect the respiratory abnormality, or to predict a respiratory event or respiratory condition in the future.
  • the FPGA 816 can, for example, have its logic blocks preconfigured with threshold values, stored values, acoustic models, machine learned trained data, machine learning processes, configuration data, or a combination thereof that can be used to perform the processing on the auditory signal 102, the result of which is to detect the respiratory abnormality, to predict the respiratory event or respiratory condition in the future, or otherwise to perform the functions described with respect to the system 100.
  • the FPGA 816 can be preconfigured with a machine learning models, for example a convolutional neural network model, which can have one or more weights 876 associated therewith.
  • the weights 876 refer to values, parameters, thresholds, or a combination thereof that act as filters in the machine learning process and represent particular features of the sounds, noises, and acoustic tones of a respiratory abnormality, respiratory event, respiratory condition, or a combination thereof.
  • the weights 876 can be iteratively adjusted based on training data.
  • the FPGA 816 can, in aspects, use the machine learning models, including the weights 876 to detect whether the auditory signals 262 contain a sound, noise, or acoustic tone indicative of a respiratory abnormality, or whether the auditory signals 262 are indicative of a respiratory event or respiratory condition in the future, or to perform the operations with respect to system 100 and FIGS. 1-4F.
  • FIG. 9 shows exemplary components of the base station 518 according to aspects.
  • FIG. 9 shows an aspect where the base station 518 includes a control unit 936, a sensor unit 902, a communication unit 928, and a wireless charging unit 978.
  • the control unit 936 can include a processor 940 and a FPGA 944.
  • the sensor unit 902 can include the contact sensor 520 and the air quality sensors 516.
  • the communication unit 928 can include an loT modem 932 and a Bluetooth circuit 930.
  • the Bluetooth circuit 930 can further include a real time audio circuit 980 and a data transfer circuit 982.
  • the real time audio circuit 980 and the data transfer circuit 982 can enable the base station 518 to connect to multiple devices simultaneously over a Bluetooth connection.
  • the real time audio circuit 980 can enable a Bluetooth connection to the digital stethoscope 510 to send or receive the auditory signal 102 or a sound file containing the auditory signal 102
  • the data transfer circuit 982 can enable simultaneous Bluetooth connection to a further device, such as a mobile phone 984 to communicate outputs or data generated by the base station 518 as a result of processing the auditory signal 102
  • the loT modem 932 can further be used to communicate outputs or data generated by the base station 518 to a further device, for example a remote server 942.
  • the loT modem 932 can further be used to receive configuration data, such as software updates, including updated acoustic models, machine learned trained data, machine learning processes, firmware, or a combination thereof from the remote server 942.
  • the base station 518 can further communicate the software updates to the digital stethoscope 510 using the Bluetooth circuit 930.
  • the processor 940 and the FPGA 944 can be coupled using a control interface 938, which can include a bus for data transfers.
  • the communication unit 928 can couple to the control unit 936 using a communication interface 934, which can include a bus for data transfers.
  • the sensor unit 902 can couple to the control unit 936 using a sensor unit interface 960, which can include a bus for data transfers.
  • the sensor unit 902 can couple to the wireless charging unit 978 using the sensor unit interface 960.
  • the processor 940 can act as a controller and control the coordination, communications, scheduling, and transfers of data between the FPGA 944 and other components of the base station 518.
  • the processor 940 can receive the auditory signal 102 from the digital stethoscope 510 via the communication unit 928, and transfer the auditory signal 102 to the FPGA 944 for further processing.
  • the FPGA 944 can transfer the output or data generated as a result of its operations back to the processor 940, which can further transfer the output or data to other components of the base station 518.
  • the processor 940 can further transfer the output or data to the communication unit 928 for transfer to the remote server 942, the mobile device 984, the digital stethoscope 510, or a combination thereof.
  • the mobile device 984 can be a device associated with a user of the system 100 that the base station 518 can use to communicate the output or data generated by the base station 518, the digital stethoscope 510, the remote server 942, or a combination thereof to a user of the system 100.
  • the mobile device 984 can be, for example, a mobile phone, a smart phone, a tablet, a laptop computer, or a combination thereof.
  • the FPGA 944 can perform the processing of the auditory signal 102.
  • the FPGA 944 can include one or more logic blocks, including one or more reconfigurable logic gates, that can be pre-programmed or configured to perform calculations or computations on the auditory signal 102, and to generate output or data generated to detect the respiratory abnormality, or to predict a respiratory event or respiratory condition in the future.
  • the FPGA 944 can, for example, have its logic blocks preconfigured with threshold values, stored values, acoustic models, machine learned trained data, machine learning processes, configuration data, or a combination thereof that can be used to perform the processing on the auditory signal 102, the result of which is to detect the respiratory abnormality, or to predict the respiratory event or respiratory condition in the future.
  • the FPGA 944 can be preconfigured with a machine learning model, for example a convolutional neural network model, which can have one or more weights 876 as shown in FIG. 8, associated therewith.
  • the FPGA 944 can be preconfigured with a machine learning model, for example a long short term memory (LSTM) network model, which can have one or more weights 876 associated therewith.
  • the FPGA 944 can work with the remote server 942 to implement the machine learning models, for example the convolutional neural network model, or the LSTM network model, wherein the FPGA 944 and the remote server 942 can divide the processing needed to perform the computations done by the machine learning model.
  • the FPGA 944 can, in aspects, use the machine learning model to detect whether the auditory signal 102 contains a sound, noise, or acoustic tone indicative of a respiratory abnormality. In other aspects, the FPGA 944 can use the machine learning model to predict a respiratory event or respiratory condition in the future using the auditory signals 262, or otherwise perform the functions of the system as described with respect to FIGS. 1-4F.
  • the wireless charging unit 978 can enable the electric charging of the digital stethoscope 510, through inductive charging by, for example, generating the electromagnetic field used to transfer energy between the charging pad 514 of FIG. 5, and a further device, such as the digital stethoscope 510 using electromagnetic induction.
  • the wireless charging unit 978 can include the processors, active and passive components, circuitry, control logic, or a combination thereof to enable the inductive charging.
  • the wireless charging unit 978 can couple to the contact sensor 520 to enable the inductive charging. For example, in aspects, if the contact sensor 520 detects contact or coupling between the digital stethoscope 510 and the charging pad 514, the contact sensor 520 can generate the contact measure 522 of FIG.
  • the wireless charging unit 978 upon receiving the contact measure 522 can determine that a coupling between the digital stethoscope 510 and the charging pad 514 has occurred and can activate the base station’s 518 processors, active and passive components, circuitry, control logic, or a combination thereof to generate the electromagnetic field and begin transferring energy between the charging pad 514 and the digital stethoscope 510.
  • the wireless charging unit 978 can further power off the base station 518 during the time period in which it is charging the digital stethoscope 510 by, for example, generating a signal to the processor 940 that charging is taking place and that the components of the base station 518 should be in an off or idle mode during the time period.
  • the wireless charging unit 978 can further enable the activation of the base station 518 based on determining a termination of the coupling between the digital stethoscope 510 and the charging pad 514.
  • the wireless charging unit 978 can detect a termination of the coupling between the digital stethoscope 510 and the charging pad 514 based on a change in the contact measure 522.
  • the contact sensor 520 can generate a contact measure 522 indicating the removal, and can send the contact measure 522 to the wireless charging unit 978.
  • the wireless charging unit 978 upon receiving the contact measure 522 can determine that the coupling between the digital stethoscope 510 and the charging pad 514 is no longer present and can send a signal to the processor 940 to activate or power up the components of the base station 518, so that the base station 518 can perform computations and processing on auditory signals, or communicate with further devices such as the digital stethoscope 510, the mobile device 984, the remote server 942, or a combination thereof.
  • module can include software, hardware, or a combination thereof in an aspect of the present disclosure in accordance with the context in which the term is used.
  • the software can be machine code, firmware, embedded code, or application software.
  • the hardware can be circuitry, a processor, a microprocessor, a microcontroller, a special purpose computer, an integrated circuit, integrated circuit cores, a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), passive devices, or a combination thereof. Further, if a module or unit is written in the system or apparatus claims section below, the module or unit is deemed to include hardware circuitry for the purpose and the scope of the system or apparatus claims.
  • the modules and units in the following description of the aspects can be coupled to one another as described or as shown.
  • the coupling can be direct or indirect, without or with intervening items between coupled modules or units.
  • the coupling can be by physical contact or by communication between modules or units.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Acoustics & Sound (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Physics & Mathematics (AREA)
  • Pulmonology (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

L'invention concerne, selon des aspects, un système et un procédé pour améliorer des stéthoscopes numériques, leur application et leur fonctionnement. Un premier procédé permet de débruiter un signal auditif. Un deuxième procédé décompose un signal auditif en sous-composants. Un troisième procédé permet d'extraire un cycle respiratoire du signal auditif. Un quatrième procédé compte les anomalies respiratoires sur la base du cycle respiratoire.
PCT/US2023/018496 2022-04-14 2023-04-13 Détection et débruitage de sons pulmonaires anormaux et extraction d'un cycle respiratoire à partir d'un signal auditif Ceased WO2023200955A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US17/659,205 2022-04-14
US17/659,205 US20230329666A1 (en) 2022-04-14 2022-04-14 Detecting and de-noising abnormal lung sounds
US17/659,206 US20230329643A1 (en) 2022-04-14 2022-04-14 Extracting a respiratory cycle from an auditory signal
US17/659,206 2022-04-14

Publications (1)

Publication Number Publication Date
WO2023200955A1 true WO2023200955A1 (fr) 2023-10-19

Family

ID=88330243

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/018496 Ceased WO2023200955A1 (fr) 2022-04-14 2023-04-13 Détection et débruitage de sons pulmonaires anormaux et extraction d'un cycle respiratoire à partir d'un signal auditif

Country Status (1)

Country Link
WO (1) WO2023200955A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102652747B1 (ko) * 2023-10-30 2024-04-01 스마트사운드주식회사 인공 지능 모델 학습 방법 및 이를 위한 전자 장치
CN117982155A (zh) * 2024-04-07 2024-05-07 天津市肿瘤医院（天津医科大学肿瘤医院） 一种胸外科检查数据智能处理方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140336537A1 (en) * 2011-09-15 2014-11-13 University Of Washington Through Its Center For Commercialization Cough detecting methods and devices for detecting coughs
US20160015359A1 (en) * 2014-06-30 2016-01-21 The Johns Hopkins University Lung sound denoising stethoscope, algorithm, and related methods
US20180317876A1 (en) * 2015-10-30 2018-11-08 The Johns Hopkins University Programmable electronic stethoscope devices, algorithms, systems, and methods
US20210145311A1 (en) * 2019-10-21 2021-05-20 Sonavi Labs, Inc. Digital stethoscope for detecting a respiratory abnormality and architectures thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140336537A1 (en) * 2011-09-15 2014-11-13 University Of Washington Through Its Center For Commercialization Cough detecting methods and devices for detecting coughs
US20160015359A1 (en) * 2014-06-30 2016-01-21 The Johns Hopkins University Lung sound denoising stethoscope, algorithm, and related methods
US20180317876A1 (en) * 2015-10-30 2018-11-08 The Johns Hopkins University Programmable electronic stethoscope devices, algorithms, systems, and methods
US20210145311A1 (en) * 2019-10-21 2021-05-20 Sonavi Labs, Inc. Digital stethoscope for detecting a respiratory abnormality and architectures thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MCLANE IAN; LAUWERS ELINE; STAS TOON; BUSCH-VISHNIAC ILENE; IDES KRIS; VERHULST STIJN; STECKEL JAN: "Comprehensive Analysis System for Automated Respiratory Cycle Segmentation and Crackle Peak Detection", IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, IEEE, PISCATAWAY, NJ, USA, vol. 26, no. 4, 27 October 2021 (2021-10-27), Piscataway, NJ, USA , pages 1847 - 1860, XP011905908, ISSN: 2168-2194, DOI: 10.1109/JBHI.2021.3123353 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102652747B1 (ko) * 2023-10-30 2024-04-01 스마트사운드주식회사 인공 지능 모델 학습 방법 및 이를 위한 전자 장치
CN117982155A (zh) * 2024-04-07 2024-05-07 天津市肿瘤医院（天津医科大学肿瘤医院） 一种胸外科检查数据智能处理方法
CN117982155B (zh) * 2024-04-07 2024-06-11 天津市肿瘤医院（天津医科大学肿瘤医院） 一种胸外科检查数据智能处理方法

Similar Documents

Publication Publication Date Title
EP3367908B1 (fr) Dispositifs, algorithmes, systèmes, et procédés de stéthoscope électronique programmable
CN116548953B (zh) 音频与热成像视频协同的睡眠呼吸状态监测方法及装置
US10709353B1 (en) Detecting a respiratory abnormality using a convolution, and applications thereof
CN110731778B (zh) 一种基于可视化的呼吸音信号识别方法及系统
Moussavi Fundamentals of Respiratory System and Sounds Analysis
US9848848B2 (en) Lung sound denoising stethoscope, algorithm, and related methods
US10709414B1 (en) Predicting a respiratory event based on trend information, and applications thereof
US20220007964A1 (en) Apparatus and method for detection of breathing abnormalities
US20140126732A1 (en) Acoustic monitoring system and methods
US11696703B2 (en) Digital stethoscope for detecting a respiratory abnormality and architectures thereof
Lin et al. Automatic Wheezing Detection Based on Signal Processing of Spectrogram and Back‐Propagation Neural Network
WO2023200955A1 (fr) Détection et débruitage de sons pulmonaires anormaux et extraction d'un cycle respiratoire à partir d'un signal auditif
WO2008036911A2 (fr) système et procédé pour la détection acoustique d'une maladie coronarienne
CN109009125A (zh) 基于移动终端音频的驾驶员细粒度呼吸监测方法及系统
US10716534B1 (en) Base station for a digital stethoscope, and applications thereof
US20230329666A1 (en) Detecting and de-noising abnormal lung sounds
US12433506B2 (en) Digital stethoscope for counting coughs, and applications thereof
Rizal et al. Lung sounds classification using spectrogram's first order statistics features
McLane et al. Comprehensive analysis system for automated respiratory cycle segmentation and crackle peak detection
US20230329643A1 (en) Extracting a respiratory cycle from an auditory signal
US10702239B1 (en) Predicting characteristics of a future respiratory event, and applications thereof
Hassan Naqvi et al. Embedded system design for classification of COPD and pneumonia patients by lung sound analysis
Talab et al. Detecting heart anomalies using mobile phones and machine learning
Pal A novel method for automatic separation of pulmonary crackles from normal breath sounds
Aras et al. Endpoint detection of lung sounds for electronic auscultation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23788947

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23788947

Country of ref document: EP

Kind code of ref document: A1