JP7684367B2 - Respiratory sound analysis device, respiratory sound analysis method, and computer program - Google Patents

Description

The present invention relates to the technical fields of a respiratory sound analysis device, a respiratory sound analysis method, and a computer program that analyzes respiratory sounds including multiple sound types.

One known device of this type is one that distinguishes between normal and abnormal breathing sounds of a living body detected by an electronic stethoscope or the like. For example, Patent Document 1 proposes a technology that outputs the analysis results of abnormal sounds on a three-dimensional display device. Patent Document 2 proposes a technology that detects rales as abnormal sounds contained in breathing sounds. Patent Document 3 proposes a technology that divides sound waveforms into multiple sections to distinguish the type of sound.

JP 2002-538921 A JP 2009-106574 A JP 2013-123494 A

However, the techniques described in Patent Documents 1 to 3 above have the technical problem that they cannot arbitrarily output multiple sound types contained in respiratory sounds.

The above is an example of the problem that the present invention aims to solve. The present invention aims to provide a respiratory sound analysis device, a respiratory sound analysis method, and a computer program that are capable of separating and outputting multiple sound types contained in respiratory sounds.

The respiratory sound analysis device for solving the above problem includes a classification means for classifying respiratory sounds into a plurality of sound types, an input means for receiving an input for selecting the type of respiratory sound to be output, an output means for outputting the respiratory sounds of the sound type selected in accordance with the input received by the input means as a spectral image, and a change means for changing the color of the spectral image for each sound type selected in accordance with the input received by the input means.

The respiratory sound analysis method for solving the above problem includes a separation step for separating respiratory sounds into a plurality of sound types, an input step for receiving an input for selecting the type of respiratory sound to be output, an output step for outputting the respiratory sounds of the sound type selected in accordance with the input received by the input means as a spectral image, and a modification step for modifying the color of the spectral image for each sound type selected in accordance with the input received by the input means.

The computer program for solving the above problem causes a computer to execute a separation process for separating respiratory sounds into a plurality of sound types, an input process for receiving an input for selecting the type of respiratory sound to be output, an output process for outputting the respiratory sounds of the sound type selected in accordance with the input received by the input means as a spectral image, and a modification process for modifying the color of the spectral image for each sound type selected in accordance with the input received by the input means.

1 is a block diagram showing the overall configuration of a respiratory sound analysis device according to an embodiment of the present invention. 4 is a flowchart showing the operation of the respiratory sound analysis device according to the present embodiment. FIG. 13 is a spectrogram diagram showing the results of frequency analysis of respiratory sounds including crepitus. FIG. 1 is a spectrogram diagram showing the results of frequency analysis of respiratory sounds including whistling sounds. 1 is a graph showing a spectrum of a breath sound including a crepitus at a predetermined timing. FIG. 1 is a conceptual diagram showing a method for approximating the spectrum of respiratory sounds including crepitus. 1 is a graph showing a spectrum of a breath sound including a whistle sound at a predetermined timing. FIG. 1 is a conceptual diagram showing a method for approximating the spectrum of respiratory sounds including whistling sounds. 13 is a graph showing an example of a frequency analysis method. FIG. 11 is a diagram illustrating an example of a frequency analysis result. FIG. 13 is a conceptual diagram showing a result of spectrum peak detection. 1 is a graph showing normal alveolar breath sound baseline. 13 is a graph showing a crepitus basis. 1 is a graph showing continuous rales. 1 is a graph showing a white noise basis. 1 is a graph showing a frequency-shifted continuous rales. FIG. 13 is a diagram showing the relationship between the spectrum and the basis and coupling coefficient. FIG. 1 shows an example of an observed spectrum and a basis used for approximation. FIG. 13 is a diagram showing each basis and coupling coefficients showing a spectrum. FIG. 1 is a conceptual diagram (part 1) showing a method for separating continuous rales according to a first modified example. FIG. 13 is a conceptual diagram (part 2) showing the method for separating continuous rales according to the first modified example. 13 is a graph showing threshold values used to distinguish between a whistling sound and a snore sound in the second modified example. 13 is a graph showing initial values of threshold values used to distinguish between a whistling sound and a snore sound in the third modified example. 13 is a graph (part 1) showing adjusted values of a threshold value used to distinguish between a whistle sound and a snore sound in the third modified example. 13 is a graph (part 2) showing adjusted values of the threshold used to distinguish between a whistle sound and a snore sound in the third modified example. FIG. 1 is a spectrogram diagram of respiratory sounds including a wheeze. 1 is a graph showing the peak frequency and the number of peaks of a whistle sound. FIG. 1 is a spectrogram diagram of respiratory sounds including snoring sounds. 1 is a graph showing the peak frequency and the number of peaks of snoring sounds. FIG. 4 is a plan view showing a display example on a display unit. 11 is a spectrogram diagram showing the extraction results for each sound type. FIG. FIG. 11 is a conceptual diagram (part 1) showing an example of audio output for each classified sound type. FIG. 11 is a conceptual diagram (part 2) showing an example of audio output for each classified sound type. FIG. 13 is a conceptual diagram showing a method for adjusting the volume of each classified sound type. FIG. 13 is a conceptual diagram showing a method of adjusting the volume for each frequency band. FIG. 11 is a conceptual diagram showing an example of image processing executed for each sound type. FIG. 11 is a plan view showing an example of an image generated by superimposing images that have been subjected to image processing for each sound type. FIG. 13 is a conceptual diagram showing a method for adjusting the display color for each classified sound type.

＜1＞
The respiratory sound analysis device according to this embodiment includes a classification means for classifying respiratory sounds into normal sounds and abnormal sounds, and an output means for outputting any one of the classified respiratory sounds.

According to the respiratory sound analysis device of this embodiment, during operation, the respiratory sounds are first separated into normal sounds and abnormal sounds. The respiratory sounds separated by the separation means do not have to be two sounds, normal sounds and abnormal sounds, and each of the normal sounds and abnormal sounds may be further separated into a plurality of sound types. For example, abnormal sounds may be separated into whistling sounds, snoring sounds, crepitus sounds, etc. Note that the specific separation method is not particularly limited, and it is sufficient if the separation can be performed in a state where any respiratory sound described later can be output.

Once the breathing sounds have been separated, any of the separated breathing sounds is output. In other words, a plurality of separated breathing sounds are selectively output. This makes it possible, for example, to output only the breathing sounds desired by the user. More specifically, for example, it is possible to output only the whistling sounds contained in the breathing sounds, or to output only the whistling sounds and snoring sounds. Note that there are no particular limitations on the output format, and the sounds may be output as audio or images, or in other formats.

As described above, by separating and outputting respiratory sounds, for example, it is possible to easily diagnose health conditions. Specifically, for example, when multiple respiratory sounds are heard mixed together, even experienced doctors have difficulty distinguishing between the different sound types. However, if any type of respiratory sound can be output, it becomes easier to distinguish the type of sound contained in the respiratory sound. In this way, if it becomes easier to hear only specific sound types contained in respiratory sounds, this can also be used in the education and research of doctors.

As described above, the respiratory sound analysis device according to this embodiment can separate respiratory sounds and output any respiratory sound, making it possible to optimally analyze respiratory sounds that contain multiple sound types.

＜2＞
In one aspect of the respiratory sound analysis device according to the present embodiment, the output means simultaneously outputs a plurality of respiratory sounds from among the separated respiratory sounds.

According to this embodiment, multiple breathing sounds can be output simultaneously (in other words, superimposed) as any breathing sound, so that desired breathing sounds can be output in appropriate combination.

＜3＞
In another aspect of the respiratory sound analysis device according to the present embodiment, the output means outputs the arbitrary respiratory sound as a voice or a spectral image.

According to this aspect, any of the separated respiratory sounds is output as audio using, for example, a speaker or headphones. Alternatively, any of the respiratory sounds is output as a spectral image using a display such as an LCD monitor. Thus, any of the output respiratory sounds can be used advantageously.

＜4＞
In another aspect of the respiratory sound analysis device according to the present embodiment, the device further comprises a change means capable of changing the output state of the arbitrary respiratory sound for each sound type.

According to this aspect, the output state of any respiratory sound (e.g., output volume, image display mode, etc.) can be changed for each sound type, so that any output respiratory sound can be used more appropriately. For example, by increasing the volume of only a specific abnormal sound contained in the respiratory sound, it is possible to make the specific abnormal sound easier to hear even when multiple respiratory sounds are mixed together. In addition, even if the volume is changed to an easy-to-hear state once and then returned to normal, it is considered that the sound will be easier to hear thereafter. Therefore, it can be effectively used for training inexperienced doctors, etc.

＜5＞
In an aspect further comprising the above-mentioned change means, the change means may be capable of changing the output volume of the arbitrary breath sound for each sound type.

In this case, the output volume can be changed for each sound type, improving convenience by making it easier to hear only the desired breathing sounds.

＜6＞
In the above-described aspect in which the output volume can be changed for each sound type, the change means may be capable of changing the output volume of the arbitrary breath sound for each predetermined frequency band.

In this case, the output volume for each sound type can be changed in more detail, so that, for example, the output volume for a specific frequency band can be increased to make the desired breathing sounds easier to hear. The specific frequency band may be set according to the characteristics of the sound type to be separated.

＜７＞
Alternatively, in an aspect further comprising a change means, the change means may be capable of executing a predetermined image processing for each sound type on the image showing the arbitrary respiratory sound.

In this case, a predetermined image processing can be applied to an image showing a given respiratory sound (e.g., a spectrogram that extracts only one type of respiratory sound) to make it easier to visually recognize. Examples of predetermined image processing include color change (e.g., RGB adjustment), binarization, edge detection, etc.

＜8＞
In an aspect in which the above-mentioned image processing can be performed for each sound type, the change means may be capable of outputting an image obtained by performing the predetermined image processing for each sound type, with the image being superimposed for a plurality of sound types.

In this case, multiple images for each sound type that have been separately image processed can be displayed on top of each other, so that multiple sound types displayed in an appropriate manner through image processing can be recognized together in a single image, making it easy to compare multiple sound types, for example.

＜9＞
In another aspect of the respiratory sound analysis device according to this embodiment, the classification means includes an acquisition means for acquiring information about frequencies corresponding to predetermined characteristics of the spectrum of the respiratory sounds, a shift means for shifting a plurality of reference spectra serving as a basis for classifying the respiratory sounds in accordance with the information about the frequencies to acquire a frequency-shifted reference spectrum, and a proportion output means for outputting a proportion of the plurality of reference spectra contained in the respiratory sounds based on the respiratory sounds and the frequency-shifted reference spectra.

According to this aspect, the classification means first obtains information about the frequency corresponding to a predetermined feature of the spectrum of the respiratory sound. Note that the "predetermined feature" here means a feature that occurs at a specific frequency depending on the sound type contained in the spectrum of the body sound, such as a peak that appears in a frequency-analyzed signal. Furthermore, the "information about the frequency" is not limited to information that directly indicates the frequency, but is intended to include information that can indirectly derive the frequency.

When frequency-related information is acquired, multiple reference spectra that serve as the basis for classifying respiratory sounds are shifted according to the frequency-related information to acquire frequency-shifted reference spectra. Note that the "reference spectra" here refer to spectra that are preset according to each sound type in order to classify multiple sound types contained in respiratory sounds (e.g., normal respiratory sounds, continuous rales, crepitus, etc.). The reference spectra are frequency-shifted according to, for example, a peak position, which is a predetermined feature acquired from the respiratory sounds, to become frequency-shifted reference spectra.

When the frequency-shifted reference spectrum is acquired, the proportion of the multiple reference spectra contained in the respiratory sound is output based on the respiratory sound and the frequency-shifted reference spectrum. Specifically, the proportion of the sound types corresponding to the multiple reference spectra contained in the respiratory sound to be analyzed is calculated, and the result is output. More specifically, for example, a calculation is performed on the spectrum of the respiratory sound based on the multiple reference spectra, and the proportion of the reference spectrum is calculated as a combination coefficient.

As a result, the separation means according to this embodiment can suitably separate respiratory sounds that include multiple sound types. In particular, this embodiment can suitably separate the proportions of each sound type even when multiple respiratory sounds are mixed on the same frequency axis.

＜10＞
In the aspect using a frequency-shifted reference spectrum described above, the predetermined feature may be a local maximum.

In this case, for example, a frequency analysis using a fast Fourier transform (FFT) or the like is performed on a signal indicating respiratory sounds, and information about the frequency corresponding to the maximum value (i.e., peak) of the analysis result is obtained. Note that the frequency information is obtained as information corresponding to the position of the maximum value, but it may be information about a frequency corresponding to a position near the maximum value, even if the frequency does not completely match the position of the maximum value.

As described above, by using the local maximum value as a predetermined feature of the respiratory sound spectrum, frequency-related information can be obtained more easily and accurately.

<11>
The respiratory sound analysis method according to this embodiment includes a separation step of separating respiratory sounds into normal sounds and abnormal sounds, and an output step of outputting any one of the separated respiratory sounds.

The respiratory sound analysis method according to this embodiment can effectively analyze respiratory sounds including multiple sound types, similar to the respiratory sound analysis device according to this embodiment described above.

The respiratory sound analysis method according to this embodiment can also employ various aspects similar to those of the respiratory sound analysis device according to this embodiment described above.

＜12＞
The computer program according to this embodiment causes a computer to execute a separation step of separating respiratory sounds into normal sounds and abnormal sounds, and an output step of outputting any of the separated respiratory sounds.

The computer program according to this embodiment can cause a computer to execute the same processing as the respiratory sound analysis method according to this embodiment described above, so that respiratory sounds including multiple sound types can be analyzed effectively.

The computer program according to this embodiment can also have various aspects similar to those of the respiratory sound analysis device according to this embodiment described above.

＜13＞
The recording medium according to this embodiment has the above-mentioned computer program recorded thereon.

According to the recording medium of this embodiment, by executing the above-mentioned computer program on a computer, it becomes possible to appropriately analyze respiratory sounds including multiple sound types.

The functions and other advantages of the respiratory sound analysis device and respiratory sound analysis method, as well as the computer program and recording medium according to this embodiment, will be described in more detail in the examples shown below.

Below, we will explain in detail the respiratory sound analysis device, respiratory sound analysis method, computer program, and recording medium examples with reference to the drawings.

<Overall composition>
First, the overall configuration of a respiratory sound analysis device according to the present embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the overall configuration of a respiratory sound analysis device according to the present embodiment.

In FIG. 1, the respiratory sound analysis device according to this embodiment is configured with the following main components: a biological sound sensor 110, a signal storage unit 120, a signal processing unit 125, an audio output unit 130, a base holding unit 140, a display unit 150, an input unit 160, and a processing unit 200.

The body sound sensor 110 is a sensor configured to detect the breathing sounds of a living body. The body sound sensor 110 is configured, for example, with an ECM (Electret Condenser Microphone), a microphone using a piezoelectric element, a vibration sensor, etc.

The signal storage unit 120 is configured as a buffer such as a RAM (Random Access Memory), and temporarily stores a signal indicating the breathing sound detected by the biological sound sensor 110 (hereinafter, appropriately referred to as the "breath sound signal"). The signal storage unit 120 is configured to be able to output the stored signal to the audio output unit 130 and the processing unit 200, respectively.

The signal processing unit 125 processes the sound acquired by the biological sound sensor 110 and outputs it to the audio output unit 130. The signal processing unit 125 functions, for example, as an equalizer or filter, and processes the acquired sound into a state that is easy for people to hear.

The audio output unit 130 is configured as, for example, a speaker or headphones, and outputs the breathing sounds detected by the body sound sensor 110 and processed by the signal processing unit 125.

The basis storage unit 140 is configured as, for example, a ROM (Read Only Memory) and stores a basis corresponding to a predetermined sound type that may be included in a respiratory sound. The basis according to this embodiment is an example of the "reference spectrum" of the present invention.

The display unit 150 is configured as a display such as an LCD monitor, and displays the image data output from the processing unit 200.

The input unit 160 is a device that accepts input from a user, and is configured as, for example, a keyboard, a mouse, a touch panel, various switches, etc. The input unit 160 is configured to allow input operations for at least selecting the breath sounds to be output.

The processing unit 200 is configured to include multiple arithmetic circuits, memories, etc. The processing unit 200 includes a frequency analysis unit 210, a frequency peak detection unit 220, a basis set generation unit 230, a coupling coefficient calculation unit 240, a signal intensity calculation unit 250, an image generation unit 260, and a respiratory sound selection unit 270.

The operation of each part of the processing unit 200 will be described in detail later.

<Operation description>
Next, the operation of the respiratory sound analysis device according to the present embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the operation of the respiratory sound analysis device according to the present embodiment. Here, a brief explanation will be given to understand the overall flow of the processing executed by the respiratory sound analysis device according to the present embodiment. Details of each processing will be described later.

In FIG. 2, during operation of the respiratory sound analysis device according to this embodiment, respiratory sounds are first detected by the body sound sensor 110, and a respiratory sound signal is acquired by the processing unit 200 (step S101).

When the respiratory sound signal is acquired, the frequency analysis unit 210 performs frequency analysis (e.g., fast Fourier transform) (step S102). In addition, the frequency peak detection unit 220 performs peak (maximum value) detection using the frequency analysis results.

Then, a basis set is generated in the basis set generation unit 230 (step S103). Specifically, the basis set generation unit 230 generates a basis set using the basis stored in the basis storage unit 140. At this time, the basis set generation unit 230 shifts the basis based on the peak position (i.e., the corresponding frequency) obtained from the frequency analysis result.

Once the basis set is generated, the coupling coefficient calculation unit 240 calculates the coupling coefficient based on the frequency analysis results and the basis set (step S104).

Once the coupling coefficient is calculated, the signal intensity calculation unit 250 calculates the signal intensity according to the coupling coefficient (step S105). In other words, the proportion of each sound type contained in the respiratory sound signal is calculated.

Once the signal strength is calculated, image data indicating the signal strength is generated in the image generation unit 260. The generated image data is displayed as the analysis result on the display unit 150 (step S106).

After the analysis results are displayed, when the user inputs the type of sound to be output (step S107: YES), the respiratory sound selection unit 270 selects the respiratory sound to be output, and the selected type of sound is output to the audio output unit 130 or the display unit 150 (step S108).

After that, a determination is made as to whether or not to continue the analysis process (step S109). If it is determined that the analysis process should be continued (step S109: YES), the process is executed again from step S101. If it is determined that the analysis process should not be continued (step S109: NO), the process ends.

<Examples of respiratory sound signals>
Next, a specific example of a respiratory sound signal analyzed by the respiratory sound analysis device according to the present embodiment will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a spectrogram showing the result of frequency analysis of a respiratory sound including a crepitus, and Fig. 4 is a spectrogram showing the result of frequency analysis of a respiratory sound including a whistle.

In the example shown in Figure 3, in addition to the spectrogram pattern corresponding to normal breath sounds, a spectrogram pattern corresponding to a type of abnormal breath sound, called a crepitus, is observed. The spectrogram pattern corresponding to a crepitus has a shape close to a diamond, as shown in the enlarged portion of the figure.

In the example shown in Figure 4, in addition to the spectrogram pattern corresponding to normal respiratory sounds, a spectrogram pattern corresponding to a whistle, which is one type of abnormal respiratory sound, is observed. The spectrogram pattern corresponding to the whistle has a shape resembling a swan's neck, as shown in the enlarged portion of the figure.

As such, there are multiple types of abnormal respiratory sounds, and each type is observed as a spectrogram pattern with a different shape. However, as can be seen from the figure, normal and abnormal respiratory sounds are detected in a mixed state. The respiratory sound analysis device of this embodiment performs analysis to separate the multiple sound types mixed in this way.

Approximation of respiratory sound signals
Next, an analysis method using the respiratory sound analysis device according to the present embodiment will be briefly described with reference to Fig. 5 to Fig. 8. Fig. 5 is a graph showing the spectrum of a respiratory sound including a crackle at a predetermined timing, and Fig. 6 is a conceptual diagram showing a method for approximating the spectrum of a respiratory sound including a crackle. Fig. 7 is a graph showing the spectrum of a respiratory sound including a whistle at a predetermined timing, and Fig. 8 is a conceptual diagram showing a method for approximating the spectrum of a respiratory sound including a whistle.

In Figure 5, if a spectrum is extracted from a respiratory sound signal containing crepitus (see Figure 3) at a timing when the spectrogram pattern corresponding to crepitus is strongly present, the result shown in the figure is obtained. This spectrum is considered to contain normal respiratory sounds and crepitus.

In FIG. 6, the spectrum corresponding to normal breathing sounds and the spectrum corresponding to hair crepitus can be estimated in advance by experiments, etc. Therefore, by using a pattern estimated in advance, it is possible to know the ratio of components corresponding to normal breathing sounds and components corresponding to hair crepitus contained in the above-mentioned spectrum.

In Figure 7, if a spectrum is extracted from a respiratory sound signal (see Figure 4) that contains a whistling sound at a timing when a spectrogram pattern corresponding to the whistling sound is strongly present, the result shown in the figure is obtained. This spectrum is considered to contain normal respiratory sounds and the whistling sound.

In FIG. 8, like the normal breathing sounds and hair crepitation described above, the spectrum corresponding to the whistle sound can also be estimated in advance by experiments, etc. Therefore, by using a pattern estimated in advance, it is possible to know the ratio of components corresponding to normal breathing sounds to components corresponding to the whistle sound in the above-mentioned spectrum.

Below, we will explain in more detail each process to achieve this analysis.

<Frequency analysis>
The frequency analysis of a respiratory sound signal and the detection of peaks in the analysis results will be described in detail with reference to Fig. 9 to Fig. 11. Fig. 9 is a graph showing an example of a frequency analysis method, Fig. 10 is a diagram showing an example of the frequency analysis results, and Fig. 11 is a conceptual diagram showing the results of spectrum peak detection.

In FIG. 9, frequency analysis is first performed on the acquired respiratory sound signal. The frequency can be analyzed using existing techniques such as fast Fourier transform. In this embodiment, the amplitude value for each frequency (i.e., the amplitude spectrum) is used as the frequency analysis result. Note that the sampling frequency, window size, and window function (e.g., Hanning window, etc.) when acquiring data can be determined appropriately.

As shown in FIG. 10, the frequency analysis result is obtained as a result consisting of n values. Note that "n" is a value determined by the window size in the frequency analysis, etc.

In FIG. 11, peak detection is performed on the spectrum obtained by frequency analysis. In the example shown in the figure, peaks p1 to p4 are detected at positions of 100 Hz, 130 Hz, 180 Hz, and 320 Hz. Note that the peak detection process can be a simple process since it is only necessary to know at which frequency the peak exists. However, it is preferable to set the peak detection parameters so that even small peaks are not missed.

In this embodiment, the points at which the maximum value is reached are found, and then up to N points (N is a predetermined value) are detected in order starting from the point with the smallest second derivative value (i.e., the point with the largest absolute value). The maximum value is found from the point where the sign of the difference switches from positive to negative. The second derivative value is approximated by the difference of the differences. Up to N points whose values are smaller than a predetermined threshold value (negative value) are selected in order starting from the smallest, and their positions are stored.

<Generating a basis set>
Next, the generation of the basis set will be described in detail with reference to Fig. 12 to Fig. 16. Fig. 12 is a graph showing a normal alveolar breath sound basis, Fig. 13 is a graph showing a crepitus basis, Fig. 14 is a graph showing a continuous rales basis, Fig. 15 is a graph showing a white noise basis, and Fig. 16 is a graph showing a frequency-shifted continuous rales basis.

As shown in Figures 12 to 15, the basis corresponding to each sound type has a unique shape. Each basis is composed of the same n numerical values (i.e., amplitude values for each frequency) as the frequency analysis results. Each basis is normalized so that the area enclosed by the line indicating the amplitude value for each frequency and the frequency axis becomes a predetermined value (for example, 1).

Incidentally, four bases are shown here: normal alveolar breath sound base, crepitus base, continuous rales base, and white noise base. However, analysis can be performed even if there is only one base. Also, a base other than the bases listed here can be used. Note that if a base corresponding to heartbeat sounds or bowel sounds is used instead of the bases corresponding to respiratory sounds listed here, it becomes possible to perform analysis of heartbeat sounds or bowel sounds.

In FIG. 16, the bases corresponding to continuous rales among the above-mentioned bases are frequency-shifted to match the peak positions detected from the results of frequency analysis. Here, an example is shown in which the continuous rales bases are frequency-shifted to match each of the peaks p1 to p4 shown in FIG. 11. Note that a base other than the base corresponding to continuous rales may also be frequency-shifted.

As a result of the above, the basis set is generated as a set of normal alveolar breath sound basis, crepitus basis, continuous rales basis for the number of peaks detected, and white noise basis.

<Calculation of coupling coefficient>
Next, the calculation of the coupling coefficient will be described in detail with reference to Fig. 17 to Fig. 19. Fig. 17 shows the relationship between the spectrum, the basis, and the coupling coefficient, Fig. 18 shows an example of the observed spectrum and the basis used for approximation, and Fig. 19 shows the approximation result by non-negative matrix factorization.

The relationship between the spectrum y, the basis h(f), and the coupling coefficient u, which is the subject of analysis, can be expressed by the following formula (1).

As shown in FIG. 17, the spectrum y and each basis h(f) have n values. On the other hand, the coupling coefficients have m values, where "m" is the number of bases included in the basis set.

In the respiratory sound analysis device according to the present embodiment, the combination coefficients of each basis included in the basis set are calculated using non-negative matrix factorization. Specifically, it is sufficient to find u (where each component value of u is non-negative) that minimizes the optimization criterion function D shown in the following formula (2).

Note that general non-negative matrix factorization is a method for calculating both a basis matrix that represents a set of basis spectra and an activation matrix that represents the coupling coefficients. In this embodiment, however, the basis matrix is fixed and only the coupling coefficients are calculated.

Incidentally, an approximation method other than non-negative matrix factorization may be used as a means for calculating the coupling coefficient. However, even in this case, the condition of non-negative is desired. Below, the reason for using the non-negative approximation method will be explained with a specific example. As shown in FIG. 18, a case is considered in which the coupling coefficient is calculated by approximating the observed spectrum with four bases A to D. Note that the expected coupling coefficient u when the condition is non-negative is 1 for base A, 1 for base B, 0 for base C, and 0 for base D. That is, when the condition is non-negative, the observed spectrum is approximated as a spectrum obtained by adding together the product of base A multiplied by 1 and the product of base B multiplied by 1.

On the other hand, if non-negativeness is not a requirement, the expected coupling coefficient u is 0 for base A, 0 for base B, 1 for base C, and -0.5 for base D. In other words, if non-negativeness is not a requirement, the observed spectrum is approximated as the sum of base C multiplied by 1 and base D multiplied by -0.5.

When comparing the two examples above, it is possible to obtain higher approximation accuracy when the non-negative condition is not met than when the non-negative condition is met. However, since the coupling coefficient u here represents the amount of components for each spectrum, it must be obtained as a non-negative value. In other words, if the coupling coefficient u is obtained as a negative value, it cannot be interpreted as a component amount. In contrast, if an approximation is performed by imposing a non-negative condition, it is possible to calculate the coupling coefficient u that corresponds to the component amount.

In FIG. 19 , in the bioanalysis device according to this embodiment, as described above, the combination coefficient u is calculated using a basis set consisting of a normal alveolar breath sound basis, a crepitus basis, four continuous rales basis, and a white noise basis, and therefore the combination coefficient u is calculated to have seven values, _u1 to _u7 .

Here, the coupling coefficient _u1 corresponding to the normal alveolar breath sound base can be said to be a value indicating the ratio of normal alveolar breath sounds to breath sounds. Similarly, the coupling coefficient _u2 corresponding to the crepitus base, the coupling coefficient _u3 corresponding to the white noise base, the coupling coefficient _u4 corresponding to the continuous rales base shifted to 100 Hz, the coupling coefficient _u5 corresponding to the continuous rales base shifted to 130 Hz, the coupling coefficient _u6 corresponding to the continuous rales base shifted to 180 Hz, and the coupling coefficient _u7 corresponding to the continuous rales base shifted to 320 Hz can be said to be values indicating the ratio of each sound type to breath sounds. Therefore, the signal intensity of each sound type can be calculated from the coupling coefficient u.

As described above, in this embodiment, multiple sound types contained in respiratory sounds are separated using multiple bases corresponding to each sound type. However, the above separation method is merely an example, and multiple sound types may be separated using other separation methods.

<Modification of separation method>
Below, some examples of classification methods other than the classification method using multiple bases already described will be described.

<First Modification>
First, the method of distinguishing the first modified example will be described with reference to Fig. 20 and Fig. 21. Fig. 20 and Fig. 21 are conceptual diagrams showing the method of distinguishing continuous rales according to the first modified example.

In the classification method according to the first modification, respiratory sounds are classified into continuous rales and other sounds. Specifically, when the peak frequency detected from the frequency analysis results of the respiratory sound signal fluctuates within a predetermined range, it is determined that the sound is continuous rales.

As shown in FIG. 20, continuous rales such as whistling sounds and snoring sounds vary so that the positions of consecutive peaks detected on the time axis fall within a predetermined range. In other words, the peak frequency changes so as to have continuity over time. Therefore, when the positions of consecutive peaks are within a predetermined range, the sound can be determined to be continuous rales.

On the other hand, as shown in FIG. 21, sounds other than continuous rales fluctuate such that the positions of the peaks detected consecutively on the time axis do not fall within a predetermined range. In other words, the peak frequency changes discretely without temporal continuity. Therefore, if the positions of consecutive peaks are not within a predetermined range, it can be determined that the sound is not a continuous ral.

The results of multiple determinations may be used to determine whether a sound is a continuous rale. Specifically, if the positions of peaks detected consecutively on the time axis fluctuate to fall within a predetermined range for a predetermined number of times or more, the sound may be determined to be a continuous rale.

<Second Modification>
Next, a classification method according to the second modified example will be described with reference to Fig. 22. Fig. 22 is a graph showing threshold values used for classifying a whistle sound and a snore sound according to the second modified example.

In the classification method according to the second modified example, continuous rales are classified into whistles and snores. Here, whistles are called high-pitched continuous rales and snores are called low-pitched continuous rales, so that whistles and snores can be distinguished by their pitch (i.e., frequency). However, the peak frequencies of whistles and snores change over time. For this reason, if a single threshold value for the peak frequency (i.e., a single threshold value that does not change) is used to classify whistles and snores, the judgment result may change over time. For example, if the peak frequency changes so as to cross the judgment threshold, something that was previously correctly judged as a sound type will be judged as an incorrect sound type. For this reason, in the second modified example, the judgment threshold value is changed according to the peak frequency.

As shown in FIG. 22, in the classification method according to the second modification, the threshold value varies so that the proportion of sounds determined to be whistling sounds and the proportion of sounds determined to be snoring sounds change smoothly according to the peak frequency. For example, when the peak frequency is 200 Hz, it is determined that 7% of the sounds are whistling sounds and 93% are snoring sounds. When the peak frequency is 250 Hz, it is determined that 50% of the sounds are whistling sounds and 50% are snoring sounds. When the peak frequency is 280 Hz, it is determined that 78% of the sounds are whistling sounds and 22% are snoring sounds. Note that the specific numerical values given here are merely examples, and different values may be set. In addition, the variation characteristics may vary depending on the sex, age, height, weight, etc. of the living subject to be measured.

By using the variable threshold described above, it is possible to effectively prevent erroneous determinations due to fluctuations in peak frequency. In other words, in the classification method according to the second modified example, the threshold for determining whistling sounds and snoring sounds varies to an appropriate value depending on the peak frequency, allowing for more accurate classification compared to, for example, the use of a single, fixed threshold.

<Third Modification>
Next, the classification method according to the third modified example will be described with reference to Fig. 23 to Fig. 25. Fig. 23 is a graph showing the initial values of the thresholds used to classify whistling sounds and snoring sounds according to the third modified example. Figs. 24 and 25 are graphs showing the adjusted values of the thresholds used to classify whistling sounds and snoring sounds according to the third modified example.

The classification method according to the third modification, like the second modification already described, is a method for classifying continuous rales into whistles and snores. It is also similar to the second modification in that it uses a threshold value for the peak frequency obtained from the frequency analysis results to make the judgment.

As shown in FIG. 23, in the classification method according to the third modified example, the judgment result is set to change at the threshold of 250 Hz. Specifically, when the peak frequency is 250 Hz or higher, it is judged that the continuous rales contain 100% whistle components and do not contain any snoring sounds. On the other hand, when the peak frequency is less than 250 Hz, it is judged that the continuous rales contain 100% snore components and do not contain any whistling sounds.

As shown in FIG. 24, in the classification method according to the third modified example, if the previous judgment determined that the sound contained 100% whistle components, the threshold is lowered from 250 Hz to 220 Hz. This makes it easier to judge that the sound contains 100% whistle components. Specifically, if we consider a case where the peak frequency is 230 Hz, the initial threshold (see FIG. 23) would result in the sound being judged as a snore-like sound, but the adjusted threshold (see FIG. 24) would result in the sound being judged as a whistle.

As shown in FIG. 25, in the classification method according to the third modified example, if the previous judgment determined that the sound contained 100% snore-like sound components, the threshold is raised from 250 Hz to 280 Hz. This makes it easier to judge that the sound contains 100% snore-like sound components. Specifically, if we consider a case where the peak frequency is 270 Hz, the initial threshold (see FIG. 23) would result in the sound being judged as a whistle sound, but the adjusted threshold (see FIG. 25) would result in the sound being judged as a snore-like sound.

By adjusting the threshold as described above, it is possible to effectively prevent erroneous determinations due to fluctuations in peak frequency. In other words, in the classification method according to the third modified example, the threshold for determining whistling sounds and snoring sounds is adjusted to an appropriate value based on past determination results, allowing for more accurate determinations to be made compared to, for example, the use of a single, unadjusted threshold.

The threshold may be adjusted based on multiple past judgment results, not just the most recent judgment result. When multiple past judgment results are used, each judgment result may be weighted. For example, weighting may be performed so that the older the judgment result, the smaller the influence. The smooth threshold of the second modified example may be used as the initial value of the threshold to be adjusted (see FIG. 22).

<Fourth Modification>
Next, a classification method according to the third modification will be described with reference to Fig. 26 and Fig. 27. Fig. 26 is a spectrogram of respiratory sounds including a whistle sound, Fig. 27 is a graph showing the peak frequency and the number of peaks of the whistle sound, Fig. 28 is a spectrogram of respiratory sounds including a snore sound, and Fig. 29 is a graph showing the peak frequency and the number of peaks of the snore sound.

The classification method according to the fourth modified example is a method for classifying continuous rales into whistles and snores, similar to the second and third modified examples already described.

In FIG. 26, respiratory sounds including whistling sounds are detected as a spectrum waveform with a certain peak. To detect the peak frequency F and the number of peaks N from this, first create a frequency-amplitude graph corresponding to a single time of the spectrum waveform (i.e., the area surrounded by a white frame in the figure).

The peak frequency F1 and the number of peaks N1 of the whistling sound can be detected from the graph shown in Figure 27. It is known that the distribution of the peak frequencies of the whistling sound is approximately 180 to 900 Hz. As can also be seen from the figure, the number of peaks N1 of the whistling sound is one.

In FIG. 28, respiratory sounds, including snoring sounds, are detected as spectrum waveforms having a specific peak that is different from that of wheezing sounds. To detect the peak frequency F and the number of peaks N from this, a frequency-amplitude graph corresponding to a single time of the spectrum waveform is similarly created.

From the graph shown in FIG. 29, the peak frequency F2 and the number of peaks N2 of the snore sound can be detected. It is known that the distribution of the peak frequencies of the snore sound is approximately 100 to 260 Hz. In other words, the peak frequency F2 of the snore sound is distributed in a region lower than the peak frequency F1 of the whistle sound. As can be seen from the figure, the number of peaks N2 of the snore sound is, for example, three. In other words, the number of peaks N2 of the snore sound is multiple, not one like the number of peaks N1 of the whistle sound.

In the classification method according to the fourth modified example, a judgment is made by utilizing the difference in characteristics between the whistle sound and the snoring sound described above. Specifically, the whistle sound and the snoring sound are classified based on both the peak frequency F and the number of peaks N. In this way, a more accurate classification can be performed compared to, for example, a case in which the whistle sound and the snoring sound are classified using only the peak frequency F.

<Display of analysis results>
Next, the display of the analysis results will be described in detail with reference to Fig. 30. Fig. 30 is a plan view showing an example of display on the display unit.

As shown in FIG. 30, the display area 155 of the display unit 150 displays the analysis results as multiple images. Specifically, area 155a displays the waveform of the acquired respiratory sounds. Area 155b displays the spectrum of the acquired respiratory sounds. Area 155c displays a spectrogram of the acquired respiratory sounds. Area 155d displays a graph showing the time series change in the component amount of each separated sound type (here, five sound types: normal respiratory sounds, snoring, whistling, crepitus, and water bubbles). Area 155e displays the proportion of each separated sound type as a radar chart.

Note that this display format of the analysis results is merely an example, and the analysis results may be displayed in other display formats. For example, the proportion of each separated sound type may be displayed as a bar graph or pie chart, or may be displayed as a numerical value.

<Sound type selection and output>
Next, the selection of a sound type by a user and the output of each selected sound type will be described with reference to Fig. 31 to Fig. 33. Fig. 31 is a spectrogram diagram showing the extraction result of each sound type. Fig. 32 and Fig. 33 are each a conceptual diagram showing an example of the voice output of each classified sound type.

As shown in FIG. 31, the spectrogram displayed in area 155c of display unit 150 may be displayed for each sound type selected by the user. That is, instead of the original (originally acquired respiratory sounds) spectrogram shown in FIG. 31(a), a spectrogram of normal respiratory sounds shown in FIG. 31(b), a spectrogram of snoring sounds shown in FIG. 31(c), a spectrogram of whistling sounds shown in FIG. 31(d), a spectrogram of crackles shown in FIG. 31(e), and a spectrogram of bubbling sounds shown in FIG. 31(f) may be displayed. Furthermore, a plurality of spectrograms for each sound type may be displayed side by side.

As shown in Figs. 32 and 33, a graph for each sound type displayed in area 155d of display unit 150 may be selected, and only the selected sound type may be output at a volume. For example, in the example of Fig. 32, only normal breathing sounds are selected, and other sounds such as snoring, whistling, hair crackles, and babbling sounds are not selected. Therefore, only normal breathing sounds are output as audio from audio output unit 130. Also, in the example of Fig. 33, only normal breathing sounds are not selected, and other sounds such as snoring, whistling, hair crackles, and babbling sounds are all selected. Therefore, audio output unit 130 outputs audio that combines snoring, whistling, hair crackles, and babbling sounds.

<Changing the output mode>
Next, a method for changing the output mode for each sound type will be specifically described with reference to Fig. 34 to Fig. 38. Fig. 34 is a conceptual diagram showing a method for adjusting the volume for each separated sound type, and Fig. 35 is a conceptual diagram showing a method for adjusting the volume for each frequency band. Fig. 36 is a conceptual diagram showing an example of image processing executed for each sound type, and Fig. 37 is a plan view showing an example of an image generated by superimposing images that have been image-processed for each sound type. Fig. 38 is a conceptual diagram showing a method for adjusting the display color for each separated sound type.

As shown in FIG. 34, the output volume of each separated sound type may be adjustable for each sound type. In the operation screen shown in the figure, each sound type can be switched ON/OFF using a checkbox, and the volume of each sound type can be adjusted using a slider. In the example shown in the figure, snoring sounds and whistling sounds are each output, with the whistling sounds being output at a louder volume than the snoring sounds.

As shown in FIG. 35, in addition to adjusting the output volume for each sound type, it may be possible to adjust the output volume for each frequency band. In the example shown in the figure, the gain is adjustable for each frequency band of 125 Hz, 250 Hz, 500 Hz, 1 kHz, and 2 kHz.

As shown in FIG. 36, image processing (e.g., binarization, edge detection, etc.) may be performed on the spectrograms extracted for each sound type. In this way, things that are difficult to recognize when simply extracted can be displayed in a more visually understandable state. Note that the image processing may be a combination of multiple processes. Also, different image processing may be performed depending on the sound type.

As shown in FIG. 37, images of each sound type that have been subjected to image processing (see FIG. 36) may be displayed in an overlapping manner. In this way, the spectrograms for each sound type can be recognized together in a single image, which allows for good visual understanding. Note that while an example in which images of normal breathing sounds and wheezing sounds are displayed in an overlapping manner is shown here, the sound types to be displayed in an overlapping manner can be selected, and only the desired sound types can be selected and displayed as appropriate.

As shown in FIG. 38, the color of the image may be adjustable for each sound type. In the example shown, the RGB values can be adjusted for each sound type by adjusting the sliders corresponding to R (red), G (green), and B (blue). In this way, multiple sound types can be displayed in different colors, making it easier to visually grasp the display.

As described above, the respiratory sound analysis device according to this embodiment can separate respiratory sounds, and then select and output them as appropriate. In addition, the output mode can be changed for each sound type, so data for each separated sound type can be used appropriately.

The present invention is not limited to the above-described embodiment, but may be modified as appropriate within the scope of the claims and the entire specification without violating the spirit or concept of the invention. Respiratory sound analysis devices and methods, as well as computer programs and recording media, incorporating such modifications are also included within the technical scope of the present invention.

REFERENCE SIGNS LIST 110 Body sound sensor 120 Signal storage unit 125 Signal processing unit 130 Audio output unit 140 Basis storage unit 150 Display unit 155 Display area 160 Input unit 200 Processing unit 210 Frequency analysis unit 220 Frequency peak detection unit 230 Basis set generation unit 240 Coupling coefficient calculation unit 250 Signal intensity calculation unit 260 Image generation unit 270 Respiratory sound selection unit y Spectrum h(f) Basis u Coupling coefficient

Claims (2)

A classification means for classifying respiratory sounds into a plurality of sound types;
an input means for receiving an input for selecting a type of breath sound to be output;
an output means for outputting, as a spectral image, respiratory sounds of sound types selected in response to the input received by the input means and corresponding to a plurality of reference spectra that are used as a criterion for classifying the respiratory sounds;
a change means for changing a color of a spectral image for each sound type selected in response to an input received by the input means and corresponding to the plurality of reference spectra ;
Equipped with
The separating means comprises:
an acquisition means for acquiring information about frequencies corresponding to predetermined features of a spectrum of respiratory sounds;
a shifting means for shifting the plurality of reference spectra in accordance with information regarding the frequency to obtain a frequency-shifted reference spectrum;
a ratio output unit that outputs a ratio of the plurality of reference spectra contained in the respiratory sound based on the respiratory sound and the frequency-shifted reference spectra. obtaining information about frequencies corresponding to predetermined features of the spectrum of respiratory sounds;
Shifting a plurality of reference spectra serving as a basis for classifying the respiratory sounds in accordance with the information on the frequency to obtain frequency-shifted reference spectra;
a classification step of classifying the respiratory sounds into a plurality of sound types by outputting a ratio of the plurality of reference spectra contained in the respiratory sounds based on the respiratory sounds and the frequency-shifted reference spectra;
an input step of receiving an input for selecting a type of breath sound to be output;
an output step of outputting, as a spectral image, respiratory sounds of sound types selected in accordance with the input received in the input step and corresponding to the plurality of reference spectra ;
a changing step of changing the color of the spectral image for each sound type selected in accordance with the input received in the input step and corresponding to the plurality of reference spectra ;
A computer program characterized by causing a computer to execute the above.