JP6575365B2 - Sound source detection apparatus, sound source detection method, and program - Google Patents

Description

  The present invention relates to a sound source detection device, a sound source detection method, and a program.

  As a method of detecting a sound source existing around a certain position, a phase difference for each frequency component is calculated from sound signals picked up by a plurality of microphones (hereinafter also referred to as “microphones”), and the direction of the sound source is calculated based on these phase differences. There are known methods for detecting.

  With respect to this type of detection method, by using a linear Hough transform or the like, by detecting a graphic reflecting the proportional relationship between the frequency and the phase difference derived from the same sound source from the phase difference of each frequency component, a plurality of A method for detecting a sound source is known (see, for example, Patent Document 1).

  In addition, a reference sound vector whose magnitude is the amplitude of the observation sound observed by the reference microphone among the plurality of microphones is set, and a plurality of sound sources are set based on the sum of the plurality of sound vectors representing the reference sound vector. A detection method is known (see, for example, Patent Document 2).

  Further, as a related technique, directions of a plurality of other vehicles existing in the traveling direction of the own vehicle based on the frequency of the sound source direction specified based on the phase difference for each frequency component and the threshold value included in the vehicle specifying information. Is known (see, for example, Patent Document 3). In this type of technology, after specifying the direction of the other vehicle based on the first threshold value included in the vehicle specifying information and the frequency of the sound source direction, the frequency of the direction different from the direction of the specified other vehicle, The direction of another other vehicle is specified based on the second threshold value smaller than the second threshold value.

JP 2006-254226 A JP 2007-096418 A International Publication No. 2012/098836

  The above sound source detection method can detect a plurality of sound sources when the directions of the plurality of sound sources viewed from the microphone are different, but the directions of the plurality of sound sources viewed from the microphone are substantially the same. It is very difficult to detect a plurality of sound sources. In other words, with the detection method described above, it is very difficult to determine whether the sound arriving at the microphone from a certain direction is a sound from one sound source or a sound in which sounds from a plurality of sound sources overlap. For this reason, in the above detection method, the second sound source is generated in the same direction as the first sound source when viewed from the microphone during the period in which the first sound source is detected, or substantially the same as the first sound source. It is very difficult to detect that the second sound source existing in the direction of

  In one aspect, an object of the present invention is to detect a change in the number of sound sources when sound sources are detected from one direction.

  A sound source detection apparatus according to one aspect includes a phase difference calculation unit and a sound source detection unit. The phase difference calculation unit calculates a phase difference for each frequency component in the audio signal based on the frequency spectrum of the plurality of audio signals acquired from the plurality of microphones of the microphone array device. The sound source detection unit detects the direction in which the sound source exists and the number of sound sources based on the calculated phase difference for each frequency component. The sound source detection unit includes a frequency component number calculation unit, a phase difference component ratio calculation unit, and a transition information calculation unit. The frequency component number calculating unit calculates the number of frequency components for each phase difference region based on the plurality of phase difference regions divided based on the arrival direction of the sound arriving at the microphone and the phase difference for each calculated frequency component. calculate. The phase difference component ratio calculation unit calculates the ratio of the number of frequency components in each phase difference region in the audio signal based on the calculated number of frequency components for each phase difference region. The transition information calculation unit calculates component ratio transition information based on a temporal change in the ratio of the number of frequency components. The sound source detection unit determines a phase difference region in which a sound source exists based on the number of frequency components for each phase difference region, and a sound source in the phase difference region in which the sound source exists based on transition information of the ratio of the number of frequency components Determine the number.

  According to the above-described aspect, it is possible to detect a change in the number of sound sources when sound sources are detected from one direction.

It is a figure which shows the mechanical structure of the sound source detection apparatus which concerns on 1st Embodiment. It is a flowchart (the 1) explaining the sound source detection process which concerns on 1st Embodiment. It is a flowchart (the 2) explaining the sound source detection process which concerns on 1st Embodiment. It is a flowchart (the 1) explaining the content of the sound source area | region / sound source number determination process which concerns on 1st Embodiment. It is a flowchart (the 2) explaining the content of the sound source area | region / sound source number determination process which concerns on 1st Embodiment. It is a figure which shows the example of a setting of a phase difference area | region. It is a figure which shows the example of the dispersion | variation in a phase difference. It is a figure explaining the determination method of a sound source area. It is a figure which shows the example of the time change of a sound source area | region. It is a graph which shows the example of the time change of a phase difference component ratio. It is a figure explaining the determination method of the change of the number of sound sources based on the time change of the area of a component ratio. It is a figure which shows the example of a detection result. It is a figure which shows the functional structure of the sound source detection apparatus which concerns on 2nd Embodiment. It is a flowchart explaining the content of the process which calculates the phase spectrum difference which concerns on 2nd Embodiment. It is a figure which shows the functional structure of the sound source detection apparatus which concerns on 3rd Embodiment. It is a flowchart explaining the content of the process which outputs the detection result which concerns on 3rd Embodiment. It is a flowchart (the 1) explaining the modification of the process which outputs a detection result. It is a flowchart (the 2) explaining the modification of the process which outputs a detection result. It is a figure which shows the functional structure of the sound source detection apparatus which concerns on 4th Embodiment. It is a flowchart (the 1) explaining the content of the process which outputs the detection result which concerns on 4th Embodiment. It is a flowchart (the 2) explaining the content of the process which outputs the detection result which concerns on 4th Embodiment. It is a flowchart (the 3) explaining the content of the process which outputs the detection result which concerns on 4th Embodiment. It is a flowchart (the 4) explaining the content of the process which outputs the detection result which concerns on 4th Embodiment. It is a figure which shows the hardware constitutions of a computer.

[First Embodiment]
FIG. 1 is a diagram illustrating a mechanical configuration of the sound source detection device according to the first embodiment.

  As shown in FIG. 1, the sound source detection apparatus 1 according to the present embodiment includes an audio signal reception unit 101, a conversion unit 102, a phase difference calculation unit 103, a sound source region / sound source number detection unit 104, and a detection result output. Unit 105.

  The audio signal receiving unit 101 receives an input of an audio signal from the microphone array 2 including the first microphone 201 and the second microphone 202. Hereinafter, the microphone is simply referred to as a microphone. The audio signal receiving unit 101 divides the audio signal input from the first microphone 201 and the audio signal in the time domain input from the second microphone 202 into processing units (frames) for sound source detection processing, respectively. To do.

  The conversion unit 102 converts the time-domain audio signal input from the first microphone 201 and the second microphone 202 into a frequency spectrum for each frame. The frequency spectrum includes an amplitude spectrum and a phase spectrum. Hereinafter, the frequency spectrum for the audio signal input from the first microphone 201 is also referred to as a first frequency spectrum, and the frequency spectrum for the audio signal input from the second microphone 202 is also referred to as a second frequency spectrum.

  The phase difference calculation unit 103 calculates the phase spectrum difference of each frequency component in the frequency spectrum for one frame based on the phase spectrum in the first frequency spectrum and the phase spectrum in the second frequency spectrum. Hereinafter, the phase spectrum difference is also referred to as a phase difference.

  The sound source region / sound source number detection unit 104 detects the phase difference region where the sound source exists and the number of sound sources based on the phase difference of each frequency component. The phase difference region is a phase difference range of the audio signal in the time region corresponding to an angle range indicating the arrival direction of the sound arriving at the microphone array. Hereinafter, the phase difference region where the sound source exists is also referred to as a sound source region. The sound source region / sound source number detection unit 104 includes a frequency component number calculation unit 104a, a phase difference component ratio calculation unit 104b, a component ratio transition information calculation unit 104c, and a component ratio holding unit 104d.

  The detection result output unit 105 outputs the detection result of the sound source region / number of sound sources detection unit 104 to an external device or the like.

  As described above, the sound source region / sound source number detection unit 104 includes the frequency component number calculation unit 104a, the phase difference component ratio calculation unit 104b, the component ratio transition information calculation unit 104c, and the component ratio holding unit 104d. .

  The frequency component calculation unit 104a calculates the number of frequency components in each phase difference region based on the phase difference between the frequency components in the phase spectrum for one frame.

  The phase difference component ratio calculation unit 104b calculates the ratio of the number of frequency components in each phase difference region in the phase spectrum for one frame based on the number of frequency components in each phase difference region calculated by the frequency component number calculation unit 104a. Hereinafter, the ratio of the number of frequency components in each phase difference region is also referred to as “phase difference component ratio” or “phase difference component ratio”.

  The component ratio transition information calculation unit 104c calculates the component ratio transition information of each phase difference region based on the phase difference component ratio in the current processing target frame and the past component ratio held by the component ratio holding unit 104d. .

  The component ratio holding unit 104d holds the component ratio of each phase difference region calculated by the phase difference component ratio calculation unit 104b.

  The sound source region / sound source number detection unit 104 detects a sound source region based on the number of frequency components in each phase difference region calculated by the frequency component number calculation unit 104a. The sound source region / sound source number detection unit 104 detects a change in the number of sound sources in each sound source region based on the component ratio transition information calculated by the component ratio transition information calculation unit 140c.

  FIG. 2A is a flowchart (part 1) for explaining sound source detection processing according to the first embodiment. FIG. 2B is a flowchart (part 2) illustrating the sound source detection process according to the first embodiment.

  When the sound source detection apparatus 1 starts the sound source detection process, as shown in FIG. 2A, first, the sound source detection apparatus 1 starts a process of acquiring audio signals from the first microphone 201 and the second microphone 202 and dividing them into frames (steps). S1). The audio signal receiving unit 101 performs the process in step S1. The audio signal reception unit 101 divides each acquired audio signal into frames of about 30 ms, for example.

  Next, the sound source detection device 1 initializes a variable x used for identifying a frame of the audio signal to 1 (step S2).

  Next, the sound source detection device 1 converts the audio signal of frame x into a frequency spectrum (step S3). The conversion unit 102 performs the process in step S3. The conversion unit 102 converts a time-domain audio signal (frame) into a frequency spectrum by, for example, fast Fourier transform.

  Next, the sound source detection device 1 calculates the phase difference of each frequency component based on the phase spectrum in the frequency spectrum (step S4). The phase difference calculation unit 103 performs the process in step S4.

  Next, the sound source detection device 1 calculates the number of frequency components in each phase difference region based on the phase difference between each frequency component for one frame (step S5). The processing in step S5 is performed by the frequency component number calculation unit 104a of the sound source region / sound source number detection unit 104. As described above, the phase difference region is a phase difference range of the audio signal in the time domain corresponding to the angle range indicating the arrival direction of the sound.

  Next, the sound source detection device 1 extracts the top N phase difference regions in descending order of the number of frequency components (step S6). The sound source area / sound source number detection unit 104 performs the process of step S6. The number N of phase difference regions extracted in the process of step S6 is an arbitrary integer of 2 ≦ N <M (M is the total number of phase difference regions).

  Next, the sound source detection device 1 determines whether or not the extracted phase difference regions are adjacent (Step S7). The determination in step S7 is performed by the sound source area / number of sound sources detection unit 104. If the extracted phase difference regions are not adjacent, the sound source region / sound source number detection unit 104 determines that a sound source exists in each of the extracted phase difference regions. Therefore, when the extracted phase difference regions are not adjacent (step S7; No), the sound source region / sound source number detection unit 104 detects a plurality of sound sources from the audio signal of frame x as shown in FIG. 2B. This is notified to the detection result output unit 105 (step S8).

  On the other hand, when the phase difference regions are adjacent to each other (step S7; Yes), the sound source detection device 1 next performs a sound source region / sound source number determination process (step S9). The sound source region / sound source number detection unit 104 performs the process of step S9. The sound source region / sound source number detection unit 104 first determines a sound source region based on the extracted phase difference region, and calculates a component ratio of each phase difference region based on the sound source region and the number of frequency components in each phase difference region. The phase difference component ratio calculation unit 104b performs processing for determining the sound source area and calculating the component ratio of each phase difference area. The phase difference component ratio calculation unit 104b causes the component ratio holding unit 104d to hold the calculated component ratio of each phase difference region in association with the frame number x. The sound source region / sound source number detection unit 104 calculates transition information (time change) of the phase difference component ratio of each phase difference region, and the number of sound sources has changed based on the amount of change in the phase difference component ratio of the sound source region. It is determined whether or not. The process of calculating the phase difference component ratio transition information is performed by the component ratio transition information calculation unit 104c. After determining whether or not the number of sound sources has changed, the sound source region / sound source number detection unit 104 outputs a detection result regarding the change in the number of sound sources to the detection result output unit 105.

  When the process of step S8 or S9 ends, the sound source detection device 1 outputs a sound source detection result for the audio signal of frame x (step S10). The detection result output unit 105 performs the process of step S10. The detection result output unit 105 transmits the sound source detection result to an external device such as a server or a storage device not shown in FIG.

  After outputting the detection result, the sound source detection device 1 determines whether or not to end the detection process (step S11). The sound source detection device 1 ends the detection process when, for example, a user (operator) of the sound source detection device 1 performs an operation to end the process using an input device or the like not shown in FIG. 1 (step S11). ; Yes). When the detection process is not terminated (step S11; No), the sound source detection device 1 updates the variable x to x + 1 (step S12) and repeats the processes of steps S3 to S10.

  The flowcharts shown in FIGS. 2A and 2B are merely examples of sound source detection processing, and some of the processing can be changed or omitted as necessary. For example, the sound source detection processing may be performed by pipelining the processing of steps S3 to S10 on the audio signal of each frame.

  The sound source region / sound source number detection unit 104 performs the sound source region / sound source number determination processing (step S9) in the sound source detection processing according to the present embodiment as described above. The sound source region / sound source number detection unit 104 performs the processing shown in FIGS. 3A and 3B as the processing in step S9.

  FIG. 3A is a flowchart (part 1) for explaining the contents of the sound source region / sound source number determination process according to the first embodiment. FIG. 3B is a flowchart (part 2) illustrating the content of the sound source region / sound source number determination process according to the first embodiment.

  In the sound source region / sound source number determination process, as shown in FIG. 3A, the sound source region / sound source number detection unit 104 first calculates the phase difference based on the number of frequency components in the N phase difference regions extracted in step S6. An average position is calculated (step S901). The processing in step S901 is performed by the phase difference component ratio calculation unit 104b of the sound source region / number of sound sources detection unit 104. The phase difference component ratio calculation unit 104b calculates the phase difference average position PP using, for example, the following formula (1).

In Equation (1), PC _n is the center value of the phase difference in the extracted phase difference region, and FF _n is the number of frequency components in the phase difference region.

  Next, the phase difference component ratio calculation unit 104b determines a phase difference region (sound source region) where a sound source exists based on the calculated phase difference average position PP (step S902). In the process of step S902, the phase difference component ratio calculation unit 104b determines a phase difference area whose center value of the phase difference area is closest to the phase difference average position PP as a sound source area.

  Next, the phase difference component ratio calculation unit 104b calculates the phase difference component ratio of each phase difference region in the phase spectrum of the frame x (step S903). In the process of step S903, the phase difference component ratio calculation unit 104b calculates the phase difference component ratio R (j, x) using, for example, the following equation (2).

  The variable j in Equation (2) is a value that identifies the phase difference region. Further, α in the formula (2) is an arbitrary value satisfying 0 <α <1. The phase difference component ratio R (j, x−1) in the phase spectrum of the frame x−1 is a positive value. Therefore, when the phase difference component ratio R (j, x) is calculated using Expression (2), the phase difference component ratio R (j, x) of the sound source region approaches 1.0, and the calculated non-sound source region The phase difference component ratio R (j, x) approaches 0.0. When the frame number x is 1, a predetermined value is used as the phase difference component ratio R (j, x−1) = R (j, 0).

  In the process of step S903, the phase difference component ratio calculation unit 104b holds (stores) the calculated phase difference component ratio R (j, x) in the component ratio holding unit 104d.

  When the process of step S903 is completed, the sound source region / sound source number detection unit 104 next checks the temporal change of the phase difference component ratio of each phase difference region (step S904). The process of step S904 is performed by the component ratio transition information calculation unit 104c. The component ratio transition information calculation unit 104c reads out the phase difference component ratios for a predetermined number of frames before the frame x-1 from the component ratio holding unit 104d, and changes the time difference of the phase difference component ratio of each phase difference region up to the frame x. Investigate.

  Next, the sound source region / sound source number detection unit 104 determines whether or not there is a time change exceeding a predetermined change amount in the phase difference component ratio of the sound source region (step S905). When the change in the phase difference component ratio of the sound source region is equal to or less than the predetermined change amount (step S905; No), the sound source region / sound source number detection unit 104 determines that there is no change in the number of sound sources. In this case, the sound source region / sound source number detection unit 104 notifies the detection result output unit 105 that there is no change in the number of sound sources (step S906), and ends the sound source region / sound source number determination process as shown in FIG. 3B. (return).

  On the other hand, when there is a change exceeding the predetermined amount in the phase difference component ratio of the sound source region (step S905; Yes), the sound source region / sound source number detection unit 104 next performs the phase difference of the sound source region as shown in FIG. 3B. It is determined whether or not the component ratio has increased (step S907). When the phase difference component ratio of the sound source region increases (step S907; Yes), the sound source region / sound source number detection unit 104 notifies the detection result output unit 105 that the number of sound sources in the sound source region has increased (step S908). Then, the sound source area / sound source number determination process is terminated. When the phase difference component ratio of the sound source region decreases (step S907; No), the sound source region / sound source number detection unit 104 notifies the detection result output unit 105 that the number of sound sources in the sound source region has decreased (step S907). S909), the sound source region / sound source number determination process is terminated.

  Thus, in the sound source region / sound source number determination process according to the present embodiment, a sound source region is determined based on the number of frequency components in each phase difference region, and 1 based on the number of frequency components in the sound source region and each phase difference region. The component ratio of each phase difference region in the phase spectrum for the frame is calculated. This component ratio becomes larger as the phase difference region has a larger number of frequency components. In the sound source region / sound source number determination process according to the present embodiment, as described above, the component ratio of the sound source region approaches 1.0 and the component ratio of the non-sound source region approaches 0.0 so that the component ratio of the non-sound source region approaches 0.0. The component ratio is calculated.

  When a new sound source is generated in the sound source region, since the sound from the new sound source is added to the sound coming from the sound source region, the number of frequency components in the sound source region is increased. In contrast, the number of frequency components in the phase difference region (non-sound source region) other than the sound source region hardly changes before and after the time when a new sound source is generated in the sound source region. Therefore, when a new sound source is generated in the sound source region, the phase difference component ratio of the sound source region is increased. Therefore, when the increase amount of the phase difference component ratio in the sound source region exceeds the predetermined time change amount, it can be determined that a new sound source has occurred in the sound source region. On the other hand, when the number of sound sources in the sound source area decreases, the phase difference component ratio in the sound source area decreases. Therefore, when the amount of decrease in the phase difference component ratio in the sound source region exceeds a predetermined time change amount, it can be determined that the number of sound sources in the sound source region has decreased.

FIG. 4 is a diagram illustrating a setting example of the phase difference region.
The sound source detection device 1 according to the present embodiment, for example, as illustrated in FIG. 4, based on audio signals from the first microphone 201 and the second microphone 202 disposed on the installation surface 3 at a distance d. Detect the direction and number of sound sources. At this time, the direction of the sound source is divided into a plurality of small angle ranges in the angle range of the arrival direction of the sound that can be collected by the first microphone 201 and the second microphone 202, and the sound source is divided into any of the plurality of small angle ranges. Detects whether or not exists. For example, as shown in FIG. 4, −90 ≦ θ ≦ 90 (the normal direction of the installation surface 3 passing through the midpoint P between the first microphone 201 and the second microphone 202 is θ = 0 (degrees). Is divided into seven small angle ranges PA1 to PA7. Based on the phase difference between the audio signal collected by the first microphone 201 and the audio signal collected by the second microphone 202, which of the seven small angle ranges PA1 to PA7 has a sound source? To decide.

  The audio signal collected by the first microphone 201 and the audio signal collected by the second microphone 202 have a phase difference corresponding to the direction of the sound source viewed from the first microphone 201 and the second microphone 202. Arise. The distance from the first microphone 201 and the second microphone 202 to the sound source is sufficiently longer than the distance (inter-microphone distance) d between the first microphone 201 and the second microphone 202. Therefore, the sound path from the sound source to the first microphone 201 and the sound path from the sound source to the second microphone 202 are substantially parallel. Accordingly, the time difference Δt (= T1−T2) between the time T1 when the sound emitted from the sound source reaches the first microphone 201 and the time T2 when the sound reaches the second microphone 202 is Δt = It is approximated by (d · sin θ) / c. Here, θ is an angle formed by the normal direction of the installation surface 3 passing through the midpoint P between the first microphone 201 and the second microphone 202 and the direction of the sound source viewed from the midpoint P. C is the speed of sound.

  That is, there is a phase difference corresponding to the time difference Δt between the audio signal collected by the first microphone 201 and the audio signal collected by the second microphone 202. The time difference Δt has a correlation with the direction (angle θ) of the sound source viewed from the first microphone 201 and the second microphone 202. The direction (angle) of the sound source viewed from the first microphone 201 and the second microphone 202 is also determined by the phase difference between the audio signal collected by the first microphone 201 and the audio signal collected by the second microphone 202. θ). Therefore, in this specification, the small angle ranges PA1 to PA7 used for determining the direction of the sound source are referred to as a phase difference region.

  Here, consider the case where the first sound source 401 exists in the phase difference area PA4 as shown in FIG. The phase difference between the audio signal collected by the first microphone 201 and the audio signal collected by the second microphone 202 is the same as that of the first sound source 401 viewed from the first microphone 201 and the second microphone 202. There is a correlation with direction. Therefore, the direction of the first sound source 401 can be detected based on the phase difference between the audio signal collected by the first microphone 201 and the audio signal collected by the second microphone 202.

  Further, when the second sound source 402 is generated in the phase difference area PA4 while the first sound source 401 is emitting sound, in the sound source detection method of the present embodiment, the transition of the phase difference component ratio of the phase difference area PA4 is performed. Generation of the second sound source 402 can be detected based on the information.

  In the example shown in FIG. 4, the angle range of −90 ≦ θ ≦ 90 (degrees) is divided into seven phase difference areas PA1 to PA7. May be set as appropriate according to the desired detection accuracy. In addition, the angle range representing the sound arrival direction is not limited to −90 ≦ θ ≦ 90 (degrees), but depends on the directivity of the first microphone 201 and the second microphone 202 to be used and the desired sound source detection range. It can be changed as appropriate.

FIG. 5 is a diagram illustrating an example of variation in phase difference.
The sound collected by the first microphone 201 and the second microphone 202 often includes a plurality of frequency components. Then, the phase difference between sounds arriving at the first microphone 201 and the second microphone 202 from a certain angle θ (≠ 0 degrees) differs depending on the frequency of the sound. Therefore, when detecting the direction of the sound source based on the phase difference of the audio signal, the frequency spectrum (phase spectrum) of each audio signal is obtained, and the direction of the sound source is detected based on the phase spectrum difference for each frequency component.

  The relationship between the frequency component (frequency bin) in the phase spectrum and the phase spectrum difference (phase difference) can be expressed by, for example, the scatter diagram shown in FIG. The scatter diagram shown in FIG. 5 shows an example in which a predetermined frequency range (for example, 0 to 4000 Hz) in the collected audio signal is divided into 128 components.

  When the sound source exists in the direction of the angle θy when viewed from the first microphone 201 and the second microphone 202, the phase difference of each frequency component can theoretically be represented by the thick straight line shown in FIG. However, in reality, the phase difference in each frequency component varies as shown by the black circles in FIG. 5 due to the influence of sound (echoic sound) reflected by an object existing around the sound source. Therefore, in the sound source detection process of the present embodiment, as shown in FIG. 2A, the number of frequency components of the phase difference existing in each of the phase difference areas PA1 to PA7 is calculated based on the phase difference of each frequency component (step S4). Then, based on the number of frequency components in each phase difference region, the phase difference region where the sound source exists is determined (steps S6 to S9). For example, the phase difference areas PA1 to PA7 are overlaid on the scatter diagram shown in FIG. 5, and in which of the phase difference areas PA1 to PA7 the phase difference of each frequency component in the phase spectrum for one frame exists. Count. Then, a phase difference region (sound source region) in which a sound source exists is determined based on a phase difference region having a large number of frequency components. For example, in the example shown in FIG. 5, the number of frequency components having a phase difference in the phase difference area PA6 is the largest.

  FIG. 6 is a diagram for explaining a method of determining a sound source region. The center value of the phase difference in FIG. 6 is the center value of each phase difference region. In FIG. 6, the center value of each phase difference region in the frequency component having the highest frequency is used.

  When converting a time-domain audio signal for one frame into a frequency spectrum, for example, a frequency range of 0 to 4000 kHz is divided into 128 frequency components. When the phase difference of the 128 frequency components is included in any of the seven phase difference areas PA1 to PA7, for example, a counting result as shown in FIG. 6 is obtained. In the example shown in FIG. 6, the total number of frequency components is 100, but this is not included in any of the phase difference regions PA1 to PA7 due to the influence of the above-described variation in phase difference. This is due to the presence of frequency components.

  When determining the phase difference region where the sound source exists based on the number of frequency components in each phase difference region, first, the top N phase difference regions are extracted in descending order of the number of frequency components (step S6). When N = 2, phase difference areas PA4 and PA5 are extracted from the counting result shown in FIG. Since the extracted phase difference areas PA4 and PA5 are adjacent to each other, the sound source detection device 1 next performs a sound source area / number of sound sources determination process (step S9). The sound source detection device 1 performs processing as shown in FIGS. 3A and 3B as sound source region / sound source number determination processing.

  In the sound source region / sound source number determination process, first, a phase difference average position is obtained from the extracted phase difference regions PA4 and PA5 (step S901). The phase difference average position is calculated using Equation (1). In the example shown in FIG. 6, the center value PC and the number of frequency components FF of the phase difference area PA4 are 0.0 and 23, respectively. The center value PC and the frequency component number FF of the phase difference area PA5 are 1.0 and 18, respectively. Accordingly, the phase difference average position PP is about 0.4 as shown in the following formula (3).

PP = (0.0 × 23 + 1.0 × 18) / (23 + 18)
= 18/41
≒ 0.439 ≒ 0.4 (3)

  The center value of the phase difference region closest to the average position PP = 0.4 calculated by the equation (3) is the center value 0.0 of the phase difference region PA4. Therefore, in the process of step S902, the phase difference component ratio calculation unit 104b determines the phase difference area PA4 as an area where the sound source exists (sound source area).

  Although illustration is omitted, when the top two phase difference areas are extracted in the order of the number of frequency components, non-adjacent phase difference areas such as phase difference areas PA4 and PA6 are extracted, for example. Sometimes. In such a case, in the sound source detection processing according to the present embodiment, since it is determined that a sound source exists in each of the phase difference areas PA4 and PA6, the two phase difference areas PA4 and PA6 become sound source areas.

  In this way, when the sound source region is determined from the phase difference regions having a large number of frequency components for each frame, information representing the time change of the sound source region as shown in FIG. 7 is obtained.

FIG. 7 is a diagram illustrating an example of a time change of the sound source region.
When the sound source is in the phase difference area PA4 and is present at a position close to the phase difference area PA5, the sound source area in each frame is determined by the processing in steps S6, S901, and S902. For example, as shown in FIG. Results. However, although it can be estimated from the time change of the sound source area shown in FIG. 7 that there is a sound source in the phase difference area PA4, it is not known whether there is a change in the number of sound sources existing in the phase difference area PA4. Therefore, in the sound source detection device 1 of the present embodiment, the component ratio of each phase difference region (that is, the ratio of the number of frequency components in each phase difference region in the number of frequency components for one frame) is calculated, and the number of sound sources is calculated from the time change of the component ratio. Is determined (steps S903 to S909). Each phase difference component ratio is calculated using Equation (2).

FIG. 8 is a graph illustrating an example of a temporal change in the phase difference component ratio.
The graph of FIG. 8 shows an example of the time change of the phase difference component ratio calculated based on the audio signal picked up at times t2 to t5 when the positional relationship between the sound source and the microphone is the relationship shown in FIG. ing. Note that the first sound source 401 emits sound in the entire section from time t2 to t5, and the second sound source 402 emits sound only in the section from time t3 to t4.

  In the period from time t2 to t3, only the first sound source 401 that is in the phase difference area PA4 and located near the phase difference area PA5 emits sound. In this case, as shown in FIG. 6, the number of frequency components in the phase difference areas PA4 and PA5 is larger than the number of frequency components in the other phase difference areas. Therefore, the time change of the sound source region in the section from time t2 to time t3 is the time change from time t0 to time t1 shown in FIG. That is, in the period from time t2 to t3, the frame in which the phase difference area PA4 or PA5 is specified as the sound source area occupies the majority, and the number of frames in which the other phase difference areas are specified as the sound source area is small. In the section from time t2 to t3, the phase difference area PA4 is most frequently specified as the sound source area. Thus, the phase difference component ratio R (PA4) of the phase difference area PA4 is the highest in the most part of the section of the time difference t2 to t3 calculated using the equation (2).

  Next, in the period from time t3 to t4, the first sound source 401 and the second sound source 402 that are in the phase difference area PA4 and located near the phase difference area PA5 emit sound. Therefore, also in this case, the time change of the sound source region specified by the sound source detection device 1 is the change shown in FIG. Accordingly, the phase difference component ratio in the section from time t3 to t4 is such that the component ratio R (PA4) of the phase difference area PA4 and the component ratio R (PA5) of the phase difference area PA5 are larger than the component ratios of the other phase difference areas. Become.

  However, in the section from time t3 to t4, the second sound source 402 also emits sound. Further, the second sound source 402 is located closer to the first microphone 201 and the second microphone 202 than the first sound source 401. Therefore, the influence of the reverberation sound of the sound emitted by the second sound source 402 on the sound signals collected by the first microphone 201 and the second microphone 202 is the influence of the reverberation sound of the sound emitted by the first sound source 401. Smaller than. Therefore, each frequency component of the sound emitted by the second sound source 402 included in the sound signal collected by the first microphone 201 and the second microphone 202 is each frequency component of the sound emitted by the first sound source 401. Compared to the above, the variation in the phase difference is reduced. That is, when the first sound source 401 and the second sound source 402 emit sound, the number of frequency components in the phase difference area PA4 is larger than when only the first sound source 401 emits sound. Therefore, in the section from time t3 to t4, as shown in FIG. 8, compared to the section from time t2 to t3 when only the first sound source 401 is emitting sound, the phase difference component ratio R ( PA4) rises. Therefore, when the increase amount of the phase difference component ratio of the phase difference region specified in the sound source region exceeds the predetermined change amount, it can be determined that the number of sound sources in the sound source region has increased. In particular, the amount of utterance of a person in a noisy environment tends to increase as the amount of noise increases due to the Lombard effect. Therefore, for example, when the first sound source 401 is a television and the second sound source 402 is a person, the influence of the change in the volume of the first sound source 401 on the phase difference component ratio is small. Therefore, according to the sound source detection method according to the present embodiment, even when the second sound source 402 is generated in substantially the same direction as the first sound source 401 under the situation where the first sound source 401 emits sound, It is possible to detect that the second sound source 402 has occurred.

  Further, when the first sound source 401 and the second sound source 402 emit sound as in the period from time t4 to t5, the state changes from the state in which only the first sound source 401 emits sound. As shown in FIG. 5, the phase difference component ratio R (PA4) of the sound source region (phase difference region PA4) decreases. Therefore, when the amount of decrease in the phase difference component ratio of the phase difference region specified in the sound source region exceeds a predetermined amount of change, it can be determined that the number of sound sources in the sound source region has decreased.

  In the example shown in FIG. 8, for example, the phase difference component ratio of the phase difference area PA4, which is the sound source area, has a relatively large short-term fluctuation in the period from time t2 to t3 and from time t4 to t5. It can be seen. In order to distinguish such short-term fluctuations from fluctuations associated with the increase or decrease in the number of sound sources, for example, the temporal change of the phase difference component ratio R (PA5) in the phase difference area PA5 adjacent to the phase difference area PA4 that is the sound source area is also included. The increase / decrease in the number of sound sources may be determined in consideration. Since the phase difference area PA5 is close to the sound sources 401 and 402, the phase difference component ratio R (PA5) of the phase difference area PA5 is larger than the phase difference component ratios of the other non-sound source areas. The phase difference area PA5 is a phase difference area where no sound source actually exists. Therefore, when a new sound source is generated in the phase difference area PA4, the phase difference component ratio R (PA5) of the phase difference area PA5 decreases corresponding to the increase of the phase difference component ratio R (PA4) of the phase difference area PA4. . Conversely, when the number of sound sources in the phase difference area PA4 decreases, the phase difference component ratio R (PA5) in the phase difference area PA5 increases corresponding to the decrease in the phase difference component ratio R (PA4) in the phase difference area PA4. . Therefore, by determining that the number of sound sources has changed when the phase difference component ratio of the phase difference region PA5 changes in the direction opposite to the direction of change of the phase difference component ratio of the phase difference region PA4 that is the sound source region, The determination accuracy can be increased.

  Further, when determining the presence or absence of a change in the number of sound sources based on the temporal change in the phase difference component ratio, for example, the determination may be made on the basis of the component ratio thresholds THp1 and THp2 as shown in FIG. In FIG. 8, two threshold values THp1 and THp2 for the component ratio are set to divide the component ratio into three sections. Here, an area where the component ratio is equal to or higher than the first threshold value THp1 is a high-rate area, an area where the second threshold value THp2 is less than the first threshold value THp1 is an intermediate ratio area, and an area where the component ratio is equal to or lower than the second threshold value THp2 is a low-rate area And When these three areas are used, the time change of the phase difference component ratio of each phase difference region shown in FIG. 8 can be expressed as shown in FIG.

  FIG. 9 is a diagram for explaining a method for determining a change in the number of sound sources based on a time change in a component ratio area.

  In the time variation of the phase difference component ratio shown in FIG. 8, the phase difference component ratios of the phase difference regions PA1 to PA3, PA6, and PA7 are low rate areas in all the intervals from time t2 to t5. Therefore, in the phase difference areas PA1 to PA3, PA6, and PA7 where no sound source exists, the phase difference component ratio areas in each frame are all “low (low rate area)” as shown in FIG. On the other hand, in the phase difference area PA4 where the sound source is present, it is “middle (medium rate area)” from time t2 to immediately before time t3, and changes to “high (high rate area)” at time t3. Further, in the phase difference area PA5 adjacent to the phase difference area PA4, it is “middle” from time t2 to immediately before time t3, and changes to “low” at time t3. Therefore, it is determined whether or not the number of sound sources in the phase difference area PA4 has increased according to whether or not the area of the phase difference component ratio has changed and whether the same area has continued for a predetermined number of frames (L frames) or more after the change. It is possible.

  Further, as shown in FIG. 9, the phase difference component ratio area of the phase difference area PA4 changes from “middle” to “high” at time t3, and then changes from “high” to “middle” at time t4. is doing. Also in this case, the number of sound sources in the phase difference area PA4 is decreased depending on whether or not the area of the phase difference component ratio changes in a decreasing direction and the same area continues for a predetermined number of frames (L frames) or more after the change. It is possible to determine whether or not.

  Note that the thresholds THp1 and THp2 when the phase difference component ratio is divided into a plurality of sections are arbitrary, and the fluctuation range of the phase difference component ratio in the sound source region when only the first sound source 401 emits sound. It is possible to set based on

FIG. 10 is a diagram illustrating an example of a detection result.
In the sound source detection process of the present embodiment, as described above, a change in the number of sound sources in the sound source region is detected based on a time change in the phase difference component ratio of each phase difference region. Therefore, the detection result output unit 105 can output a detection result as shown in FIG. 10, for example. In the example shown in FIG. 10, the number of sound sources is 1 and the sound source region is the phase difference region PA4 from frame 1 to frame 879 immediately after the start of the sound source detection process. When a new sound source is generated in the phase difference area PA4 at a time corresponding to the frame 880, the sound source detection device 1 determines that the number of sound sources has increased by the process of step S9. Therefore, from frame 880 to frame 999, the sound source areas of two or two sound sources are both phase difference areas PA4. When one of the sound sources existing in the phase difference area PA4 stops generating sound at the time corresponding to the frame 1000, the sound source detection device 1 determines that the number of sound sources has decreased. Therefore, from the frame 1000 to the frame 1253, the number of sound sources is 1 again, and the sound source region is the phase difference region PA4.

  Thereafter, for example, at the time corresponding to the frame 1254, the sound source in the phase difference area PA2 starts to emit sound in addition to the sound source in the phase difference area PA4, and the top two phase difference areas extracted in step S6 are positioned. It is assumed that the phase difference area PA4 and the phase difference area PA2 are reached. In this case, since the phase difference area PA4 and the phase difference area PA2 are not adjacent to each other (step S7; No), the detection result output unit 105 is notified of the detection result that there are a plurality of sound sources (step S8). ). Therefore, in the detection results after frame 1254, the number of sound sources is 2, and the two sound source regions are the phase difference region PA4 and the phase difference region PA2.

  As described above, according to the present embodiment, it is possible to detect other sound sources that exist in a different direction from the first sound source that emits sound constantly, and of course exist in substantially the same direction as the first sound source. It is also possible to detect other sound sources.

  2A and 2B is merely an example of a sound source detection process. For example, one phase difference region may be extracted in step S6, and steps S7 and S8 may be omitted. Even when the extracted phase difference areas are not adjacent to each other, the sound source area / number of sound sources may be determined for each extracted phase difference area (sound source area).

  In addition, the sound source detection device 1 according to the present embodiment generates a sound source using not only audio signals from two microphones, the first microphone 201 and the second microphone 202, but also audio signals from a number of microphones. It may be detected.

[Second Embodiment]
FIG. 11 is a diagram illustrating a functional configuration of the sound source detection device according to the second embodiment.

  As shown in FIG. 11, the sound source detection device 1 according to the present embodiment includes an audio signal reception unit 101, a conversion unit 102, a phase difference calculation unit 103, a sound source region / sound source number detection unit 104, and a detection result output. Unit 105.

  The audio signal reception unit 101 receives an input of an audio signal from a microphone array including the first microphone 201 and the second microphone 202. The audio signal reception unit 101 divides the time-domain audio signals input from the first microphone 201 and the second microphone 202 into processing units (frames) for sound source detection processing.

  The conversion unit 102 converts the time-domain audio signal input from the first microphone 201 and the second microphone 202 into a frequency spectrum for each frame. The frequency spectrum includes an amplitude spectrum and a phase spectrum.

  The phase difference calculation unit 103 calculates the phase difference of each frequency component in the frequency spectrum for one frame based on the phase spectrum in the first frequency spectrum and the phase spectrum in the second frequency spectrum. The first frequency spectrum is the frequency spectrum of the audio signal input from the first microphone 201, and the second frequency spectrum is the frequency spectrum of the audio signal input from the second microphone 202. The phase difference calculation unit 103 in the sound source detection device of the present embodiment includes a stationary noise estimation unit 103a. The stationary noise estimation unit 103a estimates the stationary noise of each frequency component in the frequency spectrum for one frame. The phase difference calculation unit 103 according to the present embodiment calculates only the phase difference of the frequency component larger than the stationary noise whose amplitude spectrum is estimated among all the frequency components in the frequency spectrum for one frame.

  The sound source region / sound source number detection unit 104 determines the phase difference region where the sound source exists and the number of sound sources based on the phase difference of each frequency component calculated by the phase difference calculation unit 103. The sound source region / sound source number detection unit 104 includes a frequency component number calculation unit 104a, a phase difference component ratio calculation unit 104b, a component ratio transition information calculation unit 104c, and a component ratio holding unit 104d. Each unit 104a to 104d of the sound source region / sound source number detection unit 104 in the sound source detection device 1 according to the present embodiment is equivalent to each unit 104a to 104d of the sound source region / sound source number detection unit 104 according to the first embodiment. It has a function.

  The detection result output unit 105 outputs the processing result of the sound source region / sound source number detection unit 104 to an external device or the like.

  The sound source detection device 1 of the present embodiment performs sound source detection processing according to the flowcharts shown in FIGS. 2A and 2B. At this time, the phase difference calculation unit 103 of the sound source detection device 1 performs the process shown in FIG. 12 as the process of calculating the phase spectrum difference (step S4).

  FIG. 12 is a flowchart for explaining the content of the process of calculating the phase spectrum difference according to the second embodiment.

  In the process of calculating the phase spectrum difference in the sound source detection process according to the present embodiment, the phase difference calculation unit 103 first initializes a variable i used for identifying a frequency component to 0 (step S401).

  Next, the phase difference calculation unit 103 estimates the amplitude level of stationary noise of the frequency component f (i) (step S402). The stationary noise estimation unit 103a performs the process of step S402. The stationary noise estimation unit 103a estimates the amplitude level of stationary noise based on a known stationary noise estimation method.

  Next, the phase difference calculation unit 103 compares the amplitude spectrum of the frequency component f (i) in the first frequency spectrum with the estimated amplitude level of stationary noise (step S403), and determines whether the amplitude spectrum is larger. Is determined (step S404).

  When the amplitude spectrum is larger (step S404; Yes), the phase difference calculation unit 103 calculates the phase spectrum difference of the frequency component f (i) (step S405). Thereafter, the phase difference calculation unit 103 determines whether or not there is a frequency component f (i) whose magnitude relationship between the amplitude spectrum and the estimated amplitude level of stationary noise has not been determined (step S406). On the other hand, when the amplitude spectrum is equal to or lower than the amplitude level of stationary noise (step S404; No), the phase difference calculation unit 103 omits the process of step S405 and performs the determination of step S406.

  When there is a frequency component f (i) whose magnitude relationship between the amplitude spectrum and the estimated steady-state amplitude level is undetermined (Yes in step S406), the phase difference calculation unit 103 updates the variable i to i + 1 ( Step S407) and the process returns to step S402. When the magnitude relationship between the amplitude spectrum and the stationary noise amplitude level is determined for all frequency components (step S406; No), the phase difference calculation unit 103 calculates the frequency component and phase for which the phase spectrum difference has been calculated. The spectral difference is output (step S408).

  As described above, in the sound source detection process according to the present embodiment, only the phase spectrum difference of the frequency component whose amplitude spectrum is larger than the stationary noise is calculated in the process of calculating the phase spectrum difference (step S4). Therefore, in the subsequent processing in the sound source detection processing, the phase difference component ratio is calculated based only on the phase spectrum difference of the frequency component whose amplitude spectrum is larger than the stationary noise, and the presence / absence of the change in the number of sound sources is determined. Therefore, according to the present embodiment, it is possible to detect the sound source region and the number of sound sources by excluding the frequency components buried in the stationary noise in the sound of the sound source to be detected. Accuracy is improved. In addition, by excluding frequency components buried in stationary noise in the sound of the sound source to be detected, it is possible to reduce the number of calculation processes in the phase spectrum difference calculation process, the phase difference component ratio calculation process, etc. Become. Therefore, according to the present embodiment, it is possible to reduce the processing load of the sound source detection process without reducing the detection accuracy of the sound source region and the number of sound sources.

[Third Embodiment]
FIG. 13 is a diagram illustrating a functional configuration of a sound source detection device according to the third embodiment.

  As illustrated in FIG. 13, the sound source detection device 1 according to the present embodiment includes an audio signal reception unit 101, a conversion unit 102, a phase difference calculation unit 103, a sound source region / sound source number detection unit 104, and a detection result output. Unit 105.

  The audio signal reception unit 101 receives an input of an audio signal from a microphone array including the first microphone 201 and the second microphone 202. The audio signal reception unit 101 divides the time-domain audio signals input from the first microphone 201 and the second microphone 202 into processing units (frames) for sound source detection processing.

  The conversion unit 102 converts the time-domain audio signal input from the first microphone 201 and the second microphone 202 into a frequency spectrum for each frame. The frequency spectrum includes an amplitude spectrum and a phase spectrum.

  The phase difference calculation unit 103 calculates the phase spectrum difference (phase difference) of each frequency component in the frequency spectrum for one frame based on the phase spectrum in the first frequency spectrum and the phase spectrum in the second frequency spectrum. To do. The first frequency spectrum is the frequency spectrum of the audio signal input from the first microphone 201, and the second frequency spectrum is the frequency spectrum of the audio signal input from the second microphone 202. The phase difference calculation unit 103 may calculate the phase difference of all frequency components in the frequency spectrum for one frame, or calculates only the phase difference of frequency components larger than the amplitude level of stationary noise estimated by the amplitude spectrum. May be.

  The sound source region / sound source number detection unit 104 determines the phase difference region where the sound source exists and the number of sound sources based on the phase difference of each frequency component calculated by the phase difference calculation unit 103. Although omitted in FIG. 13, the sound source region / sound source number detection unit 104 includes a frequency component number calculation unit 104a, a phase difference component ratio calculation unit 104b, a component ratio transition information calculation unit 104c, and a component ratio holding unit 104d. And including.

  The detection result output unit 105 outputs the result of the sound source detection process to the external device based on the processing result of the sound source region / sound source number detection unit 104. The detection result output unit 105 in the sound source detection device 1 of the present embodiment includes a duration measurement unit 105a, an abnormal state detection unit 105b, an attention signal output unit 105c, and a detected sound source information holding unit 105d.

  The duration measuring unit 105a measures a duration in a state where no sound source is detected (sound source non-detection duration).

  The abnormal state detection unit 105b detects an abnormal state generated in a space for detecting a sound source based on the non-detection duration of the sound source.

  The caution signal output unit 105c outputs a caution signal notifying the abnormal state to the external device when the abnormal state detection unit 105b detects the abnormal state.

  The detected sound source information holding unit 105d holds (stores) information about the detected sound source region and the number of sound sources, information indicating whether the duration is being measured, and the like.

  The sound source detection device 1 of the present embodiment performs sound source detection processing according to the flowcharts shown in FIGS. 2A and 2B. At this time, the detection result output unit 105 of the sound source detection device 1 performs, for example, the process illustrated in FIG. 14 as the process of outputting the detection result (step S10).

  FIG. 14 is a flowchart for explaining the contents of a process for outputting a detection result according to the third embodiment.

  In the process of outputting the detection result according to the present embodiment, the detection result output unit 105 first holds the detection result of the sound source region and the number of sound sources acquired from the sound source region / sound source number detection unit 104 (step S1001). Next, the detection result output unit 105 determines whether or not a sound source has been detected based on the detection result of the sound source region (step S1002). The determination in step S1002 is performed by the abnormal state detection unit 105b.

  When no sound source is detected (step S1002; No), the abnormal state detection unit 105b next determines whether or not the sound source non-detection duration is being measured (step S1003). In the determination in step S1003, the abnormal state detection unit 105b checks whether or not the non-detection duration of the sound source is measured in the duration measurement unit 105a. When the non-detection duration is not being measured (step S1003; No), the abnormal state detection unit 105b causes the duration measurement unit 105a to start measuring the non-detection duration (step S1004). When the process of step S1004 is performed, the detection result output unit 105 ends the process of outputting the detection result (return).

  If the sound source is not detected and the non-detection duration is being measured (step S1003; Yes), the abnormal state detection unit 105b then determines whether or not the non-detection duration is equal to or greater than the time threshold TH. Is determined (step S1005). When the non-detection duration is equal to or greater than the time threshold TH (step S1005; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output a warning signal (step 1006). In step S1006, the attention signal output unit 105c performs, for example, a process of calling a pre-registered telephone number or a process of transmitting an e-mail addressed to a pre-registered mail address. When the process of step S1006 is performed, the detection result output unit 105 ends the process of outputting the detection result.

  On the other hand, when the non-detection duration is shorter than the time threshold value TH (step S1005; No), the abnormal state detection unit 105b determines that no abnormality has been detected. In this case, the detection result output unit 105 ends the process of omitting step S1006 and outputting the detection result.

  Even when it is determined in step S1002 that a sound source has been detected (step S1002; Yes), the abnormal state detection unit 105b next determines whether or not the non-detection duration is being measured (step S1007). Also in the determination of step S1007, the abnormal state detection unit 105b checks whether or not the duration measurement unit 105a measures the non-detection duration of the sound source. When the non-detection duration is being measured (step S1007; Yes), the abnormal state detection unit 105b causes the duration measurement unit 105a to finish measuring the non-detection duration of the sound source (step S1008). When the process of step S1008 is performed, the detection result output unit 105 ends the process of outputting the detection result. If the non-detection duration is not being measured (step S1007; No), the detection result output unit 105 ends the process of outputting the detection result by omitting the process of step S1008.

  For example, when a space for detecting a sound source is in a house or the like, a person or device in the space emits sound, so that the person or device is detected as a sound source. Therefore, a state where a sound source is not detected for a long time in a house means that a person or various devices living in the space does not emit a sound for a long time. The reason why the sound source is not detected for a long time in the house is that the person living in the house is absent (absence) and that the person in the house cannot emit sound. It is also possible.

  In the sound source detection process according to the present embodiment, in the process of outputting the detection result (step S10), the duration of the state in which no sound source is detected (sound source non-detection duration) is measured, and the non-detection duration is equal to or greater than the time threshold. Then, it detects that an abnormality has occurred. In addition, the detection result output unit 105 according to the present embodiment performs a process of calling a pre-registered telephone number or a process of transmitting a mail addressed to a pre-registered mail address when an abnormality occurs. I do. Therefore, according to the present embodiment, it is possible to notify a person who is remote from the space where the sound source is detected, of the occurrence of an abnormal state in which the sound source is not detected for a long time.

  Note that the flowchart of FIG. 14 is merely an example of a process for outputting a detection result, and the content of the process can be changed as appropriate. For example, when the duration measurement unit 105a can measure the duration (sound source detection duration) in a state where the sound source is detected in addition to the sound source non-detection duration, the detection result output by the detection result output unit 105 The process shown in FIGS. 15A and 15B may be output.

  FIG. 15A is a flowchart (part 1) illustrating a modification of the process of outputting the detection result. FIG. 15B is a flowchart (part 2) illustrating a modified example of the process of outputting the detection result.

  In the modification of the process of outputting the detection result, the detection result output unit 105 first holds the detection result of the sound source region and the number of sound sources acquired from the sound source region / sound source number detection unit 104 as shown in FIG. 15A ( Step S1011). Next, the detection result output unit 105 determines whether or not a sound source has been detected based on the detection result of the sound source region (step S1012). The determination in step S1012 is performed by the abnormal state detection unit 105b.

  When no sound source is detected (step S1012; No), the abnormal state detection unit 105b next determines whether or not the non-detection duration of the sound source is being measured in the duration measurement unit 105a (step S1013). ). When the non-detection duration is not being measured (step S1013; No), the abnormal state detection unit 105b determines that no abnormality has been detected. In this case, the detection result output unit 105 next causes the duration measurement unit 105a to start measuring the non-detection duration (step S1014). After the process of step S1014, the abnormal state detection unit 105b determines whether the duration measurement unit 105a is measuring the detection duration of the sound source (step S1015). When the detection duration is being measured (step S1015; Yes), the abnormal state detection unit 105b causes the duration measurement unit 105a to finish measuring the detection duration (step S1016). When the process of step S1016 is performed, the detection result output unit 105 ends the process of outputting the detection result (return). If the sound source is not detected and the detection duration time is not being measured (step S1015; No), the detection result output unit 105 ends the process of outputting the detection result by omitting the process of step S1016.

  On the other hand, when the sound source is not detected and the non-detection duration is being measured (step S1013; Yes), the abnormal state detection unit 105b then has a non-detection duration equal to or greater than the first time threshold value TH1. It is determined whether or not (step S1017). When the non-detection duration is equal to or longer than the first time threshold value TH1 (step S1017; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output the first attention signal (step S1018). The attention signal output unit 105c transmits, for example, an e-mail including a message indicating that a state in which no sound source is detected continues for a predetermined period as a first attention signal to a pre-registered mail address. When the non-detection duration is shorter than the first time threshold TH1 (step S1017; No), the abnormal state detection unit 105b determines that no abnormality has been detected. In this case, the detection result output unit 105 omits the process of step S1018 and ends the process of outputting the detection result.

  If it is determined in step S1012 that a sound source has been detected (step S1012; Yes), as shown in FIG. 15B, the abnormal state detection unit 105b next determines whether there is a sound source region for which the detection duration is not measured. Is determined (step S1019). When there is a sound source region for which the detection duration is not measured (step S1019; Yes), the abnormal state detection unit 105b then detects the detection duration of the sound source region for which the detection duration is not measured by the duration measurement unit 105a. Is started (step S1020). After step S1020, the abnormal state detection unit 105b determines whether or not the non-detection continuation period for the sound source region where the measurement of the detection continuation time has started is being measured (step S1021). When the non-detection duration is being measured (step S1021; Yes), the abnormal state detection unit 105 causes the duration measurement unit 105a to finish measuring the non-detection duration (step S1022). Thereafter, the abnormal state detection unit 105b determines whether there is a sound source region in which the number of sound sources has increased (step S1023).

  When the sound source is detected and the detection durations of all detected sound source regions are measured (step S1019; No), the abnormal state measuring unit 105b omits the processes of steps S1020 to S1022, The determination in S1023 is performed. Further, when there is no sound source region for which the non-detection duration is being measured (step S1021; No), the abnormal state detection unit 105b omits the process of step S1022 and performs the determination of step S1023.

  The abnormal state detection unit 105b performs the determination in step S1023 based on the detection result regarding the number of sound sources. If there is a sound source region in which the number of sound sources has increased (step S1023; Yes), the abnormal state detection unit 105b next determines whether the elapsed time since the number of sound sources has increased is equal to or greater than the second time threshold value TH2. It is determined whether or not (step S1024). When the elapsed time is equal to or greater than the second time threshold TH2 (step S1024; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output a second attention signal (step S1025). The attention signal output unit 105c transmits, for example, a mail including a message indicating that the state in which the number of sound sources has increased continues for a predetermined period as a second attention signal to a previously registered mail address. When the process of step S1025 is performed, the detection result output unit ends the process of outputting the detection result as illustrated in FIG. 15A.

  If there is no sound source area where the number of sound sources has increased (step S1023; No), the abnormal state detection unit 105b determines that no abnormality has been detected. In this case, the detection result output unit 105 omits the processes of steps S1024 and S1025 and ends the process of outputting the detection result. Also, when the elapsed time after the increase in the number of sound sources in the sound source region where the number of sound sources has increased is shorter than the second time threshold value TH2 (step S1024; No), the abnormal state detection unit 105b is also in the abnormal state detection unit. 105b determines that no abnormality has been detected. Therefore, also in this case, the detection result output unit 105 omits the process of step S1025 and ends the process of outputting the detection result.

  In the modification of the process of outputting the detection results shown in FIGS. 15A and 15B, the detection result output unit 105 outputs the first attention signal when the non-detection duration of the sound source is equal to or greater than the first time threshold value TH1. . Therefore, also in the above modification, it is possible to notify the occurrence of an abnormal state in which a sound source is not detected for a long time to a person who is remote from the space where the sound source is detected.

  In the above modification, the detection result output unit 105 is in a state where the number of sound sources in the sound source region has increased, and when the elapsed time since the increase in the number of sound sources becomes equal to or greater than the second time threshold value TH2. 2 attention signal is output.

  For example, when a space for detecting a sound source is in a house or the like, a person or device in the space emits sound, so that the person or device is detected as a sound source. Therefore, when a device that can be a sound source such as a ventilation fan is operated on a daily basis in a house, the device may be detected as the first sound source for a long time. Under such circumstances, when the second sound source is generated in substantially the same direction as the first sound source when viewed from the first microphone 201 and the second microphone 202, the sound source detection device 1 has the first sound source. It is detected that the number of sound sources in the sound source region to be increased (the second sound source has been generated). The sound source detected as the second sound source may be a person living in a house or a device different from the first sound source. When the sound source detected as the second sound source is a person living in a house or a device operated by the person, it is unlikely that the person or device continues to emit sound for a long time. Therefore, if the state where the number of sound sources in the sound source area increases continues for a long time, there is a possibility that some abnormality has occurred in the sound source area. Therefore, by outputting a caution signal when the number of sound sources in the sound source region increases for a long time, the user of the sound source detection device 1 may have an abnormality in which an undefined sound source continues for a long time. Sex can be known at an early stage, and it becomes possible to deal with abnormalities at an early stage.

[Fourth Embodiment]
FIG. 16 is a diagram illustrating a functional configuration of a sound source detection device according to the fourth embodiment.

  As illustrated in FIG. 16, the sound source detection device 1 according to the present embodiment includes an audio signal reception unit 101, a conversion unit 102, a phase difference calculation unit 103, a sound source region / sound source number detection unit 104, and a detection result output. Unit 105 and a sound source information management unit 106.

  The audio signal reception unit 101 receives an input of an audio signal from a microphone array including the first microphone 201 and the second microphone 202. The audio signal reception unit 101 divides the time-domain audio signals input from the first microphone 201 and the second microphone 202 into processing units (frames) for sound source detection processing.

  The conversion unit 102 converts the time-domain audio signal input from the first microphone 201 and the second microphone 202 into a frequency spectrum for each frame. The frequency spectrum includes an amplitude spectrum and a phase spectrum.

  The phase difference calculation unit 103 calculates the phase spectrum difference (phase difference) of each frequency component in the frequency spectrum for one frame based on the phase spectrum in the first frequency spectrum and the phase spectrum in the second frequency spectrum. To do. The first frequency spectrum is the frequency spectrum of the audio signal input from the first microphone 201, and the second frequency spectrum is the frequency spectrum of the audio signal input from the second microphone 202. The phase difference calculation unit 103 may calculate the phase difference of all frequency components in the frequency spectrum for one frame, or calculates only the phase difference of frequency components larger than the amplitude level of stationary noise estimated by the amplitude spectrum. May be.

  The sound source region / sound source number detection unit 104 determines the phase difference region where the sound source exists and the number of sound sources based on the phase difference of each frequency component calculated by the phase difference calculation unit 103. Although omitted in FIG. 16, the sound source region / sound source number detection unit 104 includes a frequency component number calculation unit 104a, a phase difference component ratio calculation unit 104b, a component ratio transition information calculation unit 104c, and a component ratio holding unit 104d. And including.

  The detection result output unit 105 outputs the result of the sound source detection process to the external device based on the processing result of the sound source region / sound source number detection unit 104. The detection result output unit 105 includes a duration measurement unit 105a, an abnormal state detection unit 105b, an attention signal output unit 105c, and a detected sound source information holding unit 105d.

  The sound source information management unit 106 manages information (sound source information) about a specific sound source in the space where the sound source is detected. For example, when a space for detecting a sound source is in a house, the specific sound source is a television, a ventilation fan, a water supply, or the like. The sound source information management unit 106 includes a sound source information registration unit 106a and a sound source information holding unit 106b.

  The sound source information registration unit 106a generates sound source information for the sound source based on the phase difference (phase spectrum difference) for each frequency component in a state where only a specific sound source emits sound. The sound source information to be generated includes information on a phase difference region (sound source region) where a specific sound source exists. The sound source information registration unit 106a holds (stores) the generated sound source information in the sound source information holding unit 106b. The sound source information held by the sound source information holding unit 106b is referred to when the abnormal state detection unit 105b of the detection result output unit 105 detects an abnormal state.

  The sound source detection device 1 of the present embodiment generates and registers sound source information about a specific sound source in a space for detecting a sound source, for example, separately from the sound source detection processing according to the flowcharts shown in FIGS. 2A and 2B. Process. When performing processing for registering sound source information, a space for detecting a sound source is set to a state in which only a specific sound source emits sound. Thereafter, the operator of the sound source detection device 1 operates an operation unit not shown in FIG. 16 to operate the sound source detection device 1 in a mode for registering sound source information. At this time, the sound source detection device 1 converts the audio signal input from the first microphone 201 and the second microphone 202 into a frequency spectrum, and calculates a phase difference for each frequency component. The conversion unit 102 performs processing for converting the audio signal into a frequency spectrum. The phase difference calculation unit 103 performs processing for calculating the phase difference for each frequency component. When operating in a mode for registering sound source information, the phase difference calculation unit 103 transmits the calculated phase difference of each frequency component to the sound source information management unit 106. The sound source information management unit 106 generates sound source information based on the phase difference of each frequency component received from the phase difference calculation unit 103, and causes the sound source information holding unit 106b to hold the generated sound source information. The sound source information management unit 106 (sound source information registration unit 106a) calculates, for example, the number of frequency components in each phase difference region based on the phase difference of each frequency component, and specifies the phase difference region having the largest number of frequency components as the registration target. Determine the sound source region for the sound source. Note that the sound source region / sound source number detection unit 104 may perform the process of determining the sound source region for the specific sound source instead of the sound source information management unit 106 (sound source information registration unit 106a).

  When sound source detection processing using the sound source detection device 1 is started after the sound source information is registered according to the above procedure, the operator of the sound source detection device 1 operates an operation unit not shown in FIG. The apparatus 1 is operated in a mode for performing sound source detection processing. When the sound source detection process is started, the sound source detection apparatus 1 performs the sound source detection process according to the flowchart shown in FIGS. 2A and 2B. At this time, the detection result output unit 105 of the sound source detection device 1 performs the process shown in FIGS. 17A to 17D as the process of outputting the detection result (step S10).

  FIG. 17A is a flowchart (part 1) illustrating the content of a process for outputting a detection result according to the fourth embodiment. FIG. 17B is a flowchart (part 2) illustrating the content of the process of outputting the detection result according to the fourth embodiment. FIG. 17C is a flowchart (part 3) illustrating the content of the process of outputting the detection result according to the fourth embodiment. FIG. 17D is a flowchart (part 4) illustrating the content of the process of outputting the detection result according to the fourth embodiment.

  In the process of outputting the detection result according to the present embodiment, the detection result output unit 105 first holds the detection result of the sound source region and the number of sound sources acquired from the sound source region / sound source number detection unit 104 as shown in FIG. 17A. (Step S1031). The detection result output unit 105 associates the detection result of the sound source region and the number of sound sources with the frame number x, and holds them in the detected sound source information holding unit 105d.

  Next, the detection result output unit 105 compares the detected sound source region with the sound source region included in the registered sound source information (step S1032). The abnormal state detection unit 105b performs the process in step S1032. The abnormal state detection unit 105b reads the sound source information held by the sound source information holding unit 106b of the sound source information management unit 106, and includes a phase difference region registered as a sound source region in the sound source information and a sound source region / sound source number detection unit. The phase difference area detected as a sound source area in 104 is compared.

  Next, the abnormal state detection unit 105b determines whether or not there is a phase difference region in which a sound source is detected in a phase difference region that is not registered in the sound source region (step S1033). When a sound source is detected from a phase difference region that is not registered in the sound source region (step S1033; Yes), the abnormal state detection unit 105b next performs the processes of steps S1034 to S1037. Steps S1034 to S1037 are processing performed to detect a first abnormal state in which an unknown sound source exists for a predetermined time in a phase difference region that is not registered in the sound source region. On the other hand, when no sound source is detected from the phase difference region that is not registered in the sound source region (step S1033; No), the abnormal state detection unit 105b performs the processing of steps S1038 to S1041. Steps S1038 to S1041 are processes performed to detect a second abnormal state in which a state in which no sound source is detected from a phase difference region that is not registered in the sound source region continues for a predetermined time or longer.

  When performing the process of detecting the first abnormal state, the abnormal state detection unit 105b sets the phase difference area in which the sound source is detected among the phase difference areas not registered in the sound source area as processing targets, and performs steps S1034 to S1037. Perform the process. In this case, the abnormal state detection unit 105b first determines whether or not there is a phase difference region in which the detection duration of the sound source is not measured in the corresponding phase difference region (phase difference region to be processed) ( Step S1034).

  If there is no phase difference region in which the detection duration is not measured (step S1034; No), the abnormal state detection unit 105b then has a phase difference region in which the detection duration is equal to or greater than the first time threshold value TH1. It is determined whether or not (step S1036). If there is a phase difference region where the detection duration is not measured (step S1034; Yes), the abnormal state detection unit 105b starts measuring the detection duration for the corresponding phase difference region in the duration measurement unit 105a. (Step S1035). Thereafter, the abnormal state detection unit 105b determines whether or not there is a phase difference region whose detection duration is equal to or greater than the first time threshold value TH1 (step S1036).

  In step S1036, the abnormal state detection unit 105b sets a phase difference region in which a sound source is detected among phase difference regions not registered in the sound source region as a determination target. If there is a phase difference region in which the detection duration is equal to or greater than the first time threshold TH1 (step S1036; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output the first attention signal (step S1037). ). In the process of step S1037, the attention signal output unit 105c transmits, for example, an e-mail including a message notifying that an unknown sound source has been present for a predetermined time or more to a pre-registered mail address. Process. Thereafter, the abnormal state detection unit 105b determines whether or not there is a phase difference region in which the sound source is not detected in the phase difference regions registered in the sound source region illustrated in FIG. 17C (step S1042). When there is no phase difference region where the detection duration is equal to or greater than the first time threshold value TH1 (S1036; No), the abnormal state detection unit 105b omits the process of step S1037, and then performs the determination of step S1042. Do.

  On the other hand, when performing the process of detecting the second abnormal state, the abnormal state detection unit 105b sets all the phase difference areas not registered in the sound source area as processing targets, and performs the processes of steps S1038 to S1041 illustrated in FIG. 17B. I do. In this case, the abnormal state detection unit 105b first determines whether there is a phase difference region in which the non-detection duration of the sound source is not measured in the phase difference regions that are not registered in the sound source region (step S1038). ).

  When the non-detection continuation time is measured in all the phase difference regions to be processed (step S1038; No), the abnormal state detection unit 105b then has the non-detection continuation time equal to or greater than the second time threshold value TH2. It is determined whether or not there is a phase difference region (step S1040). Further, when there is a phase difference region where the non-detection duration is not measured (step S1038; Yes), the abnormal state detection unit 105b causes the duration measurement unit 105a to measure the non-detection duration for the corresponding phase difference region. Is started (step S1039). In step S1039, if there is a phase difference region for which the sound source detection duration is being measured in the phase difference region to be processed, the abnormal state detection unit 105b ends the measurement of the detection duration for the phase difference region. Let Thereafter, the abnormal state detection unit 105b determines whether or not there is a phase difference region whose non-detection duration is equal to or greater than the second time threshold value TH2 (step S1040).

  In step S1040, the abnormal state detection unit 105b sets all phase difference areas not registered in the sound source area as determination targets. If there is a phase difference region where the non-detection duration is equal to or greater than the second time threshold TH2 (step S1040; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output the second attention signal (step S1040). S1041). In the process of step S1041, the attention signal output unit 105c registers in advance an e-mail including a message notifying that a sound source has not been detected for a predetermined time or more from a phase difference region not registered in the sound source region, for example. Process to send to a certain mail address. Thereafter, the abnormal state detection unit 105b determines whether or not there is a phase difference region in which the sound source is not detected in the phase difference regions registered in the sound source region illustrated in FIG. 17C (step S1042). If there is no phase difference region where the non-detection duration is equal to or greater than the second time threshold value TH2 (S1040; No), the abnormal state detection unit 105b omits the process of step S1041, and then determines whether the step S1042 I do.

  The process for detecting the first abnormal state and the process for detecting the second abnormal state are processes for detecting an abnormal state for a phase difference region that is not registered in the sound source region. On the other hand, the process after step S1042 is a process which detects the abnormal state about the phase difference area | region registered into the sound source area | region.

  In step S1042, the abnormal state detection unit 105b determines whether there is a phase difference region in which no sound source is detected in the phase difference regions registered in the sound source region. When there is a phase difference region in which no sound source is detected in the phase difference region registered in the sound source region (step S1042; Yes), the abnormal state detection unit 015b then performs steps S1043 to S1046 illustrated in FIG. 17C. Perform the process. Steps S1043 to S1046 are processes performed to detect a third abnormal state in which no sound source is detected for a predetermined period or more from the phase difference region registered in the sound source region. After steps S1043 to S1046, the abnormal state detection unit 015b performs the processes of steps S1047 and S1048 and the processes of steps S1049 to S1051 shown in FIG. 17D. Steps S <b> 1047 and S <b> 1048 are processes for starting the measurement of the detection duration for the phase difference region where the detection duration of the sound source is not measured. Steps S1049 to S1051 are processes performed to detect a fourth abnormal state in which an unknown sound source continues to exist for a predetermined period or more from the phase difference region registered in the sound source region. On the other hand, when the sound source is detected from all the phase difference regions registered in the sound source region (step S1042; No), the abnormal state detection unit 105b omits the processing of steps S1043 to S1046 and performs the processing of steps S1047 to S1051. I do.

  When performing the process of detecting the third abnormal state, the abnormal state detection unit 105b sets the phase difference area in which the sound source is not detected among the phase difference areas registered in the sound source area as processing targets, and performs steps S1043 to S1046. Perform the process. In this case, the abnormal state detection unit 105b first determines whether or not there is a phase difference region in which the non-detection duration of the sound source is not measured in the corresponding phase difference region (processing target phase difference region). (Step S1043).

  When there is a phase difference region where the non-detection duration is not measured (step S1043; Yes), the abnormal state detection unit 105b starts measurement of the non-detection duration for the corresponding phase difference region in the duration measurement unit 105a. (Step S1044). In step S1044, when there is a phase difference region for which the detection duration of the sound source is being measured in the phase difference region to be processed, the abnormal state detection unit 105b ends the measurement of the detection duration for the phase difference region. Let Thereafter, the abnormal state detection unit 105b determines whether or not there is a phase difference region whose non-detection duration is equal to or greater than the third time threshold TH3 (step S1045). Further, when the non-detection continuation time of all corresponding phase difference regions is measured (step S1043; No), the abnormal state detection unit 105b omits the process of step S1044 and performs the determination of step S1045.

  In step S <b> 1045, the abnormal state detection unit 105 b sets a phase difference area in which no sound source is detected among the phase difference areas registered in the sound source area as a determination target. When there is a phase difference region whose non-detection duration is equal to or greater than the third time threshold TH3 (step S1045; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output a third attention signal (step S1045). S1046). In the process of step S1046, the attention signal output unit 105c performs, for example, a process of transmitting an e-mail including a message notifying that a registered sound source is not detected for a predetermined period or more to a pre-registered mail address. Do. Thereafter, the abnormal state detection unit 105b determines whether or not there is a phase difference region in which the detection duration of the sound source is not measured in the phase difference regions registered in the sound source region shown in FIG. 17D ( Step S1047). If there is no phase difference region where the non-detection duration is equal to or greater than the third time threshold TH3 (S1045; No), the abnormal state detection unit 105b omits the process of step S1046, and then determines in step S1047. I do.

  On the other hand, when the process of detecting the third abnormal state is not performed (step S1042; No), the abnormal state detection unit 105b omits the processes of steps S1043 to S1046 illustrated in FIG. 17C, and then performs the process of step S1047. Make a decision.

  In step S1047, the abnormal state detection unit 105b determines whether there is a phase difference region in which the detection duration of the sound source is not measured in the phase difference regions registered in the sound source region. When there is a phase difference region in which the detection duration is not measured (step S1047; Yes), the abnormal state detection unit 105b causes the duration measurement unit 105a to start measuring the detection duration for the corresponding phase difference region ( Step S1048). In step S1048, when there is a phase difference region for which the non-detection duration of the sound source is being measured in the phase difference region to be processed, the abnormal state detection unit 105b measures the non-detection duration for the phase difference region. End. After that, the abnormal state detection unit 105b determines whether there is a phase difference region in which the number of sound sources is increased in the phase difference regions registered in the sound source region (step S1049). When the detection duration time is measured in all the phase difference regions registered in the sound source region (step S1047; No), the abnormal state detection unit 105b omits the process of step S1048, and then performs step S1049. Judgment is made.

  In step S1049, the abnormal state detection unit 105b determines a phase difference region in which a sound source is detected from among the phase difference regions registered in the sound source region as a determination target. The abnormal state detection unit 105b determines whether there is a phase difference region (sound source region) in a state where the number of sound sources has increased, based on the detection result of the sound source region and the number of sound sources. When there is a phase difference region in a state where the number of sound sources has increased (step S1049; Yes), the abnormal state detection unit 105b next has an elapsed time after the increase in the number of sound sources is equal to or greater than a fourth time threshold value TH4. Is determined (step S1050). The abnormal state detection unit 105b calculates, for example, an elapsed time after the number of sound sources increases based on the history of the detection results of the sound source region and the number of sound sources held by the detected sound source information holding unit 105d, Compare with the time threshold TH4. When the elapsed time from the increase in the number of sound sources is equal to or greater than the fourth time threshold TH4 (step S1050; Yes), the abnormal state detection unit 105b causes the attention signal output unit 105c to output a fourth attention signal ( Step S1051). In the process of step S1051, the attention signal output unit 105c, for example, registers in advance an e-mail including a message notifying that an unknown sound source continues to exist in the phase difference region registered in the sound source region for a predetermined time or more. Process to send to a certain mail address. When the process of step S1051 is performed, the detection result output unit 105 ends the process of outputting the detection result (return).

  On the other hand, when there is no phase difference region in which the number of sound sources has increased (step S1049; No), the abnormal state detection unit 105b says that unknown sound sources continue to exist in the phase difference region registered in the sound source region for a predetermined time or more. It is determined that the fourth abnormal state has not been detected. In this case, the detection result output unit 105 omits the processes of steps S1050 and S1051, and ends the process of outputting the detection result. Even when the elapsed time after the number of sound sources in the phase difference region in the state where the number of sound sources has increased is shorter than the fourth time threshold value TH4 (step S1050; No), the abnormal state detection unit 105b It is determined that the abnormal state 4 is not detected. In this case, the detection result output unit 105 omits the process of step S1051 and ends the process of outputting the detection result.

  As described above, in the sound source detection processing according to the present embodiment, an abnormality in a space for detecting a sound source based on a time change of a sound source detection result in a phase difference region different from a phase difference region registered in advance as a sound source region. Detect state. Therefore, if you want to detect the sound source in the house, register the phase difference area where the sound source fixed at a specific position such as TV, ventilation fan, water supply, etc. is registered as the sound source area. The occurrence of a different sound source can be detected at an early stage. In the sound source detection processing according to the present embodiment, a caution signal is output even when a sound source is not detected for a long time from a phase difference region other than the phase difference region registered as the sound source region. Therefore, for example, even when an abnormality occurs in a person living in a house (a space where a sound source is detected) and an action to emit a sound cannot be performed, the abnormality can be detected early.

  Furthermore, in the sound source detection processing according to the present embodiment, a caution signal is output even when the state in which the number of sound sources has increased in the phase difference region registered in the sound source region continues for a long time. Therefore, an abnormal state in which an unregistered sound source continues to exist for a long time in substantially the same direction as the registered sound source when viewed from the first microphone 201 and the second microphone 202 can be detected at an early stage.

  In addition, in the sound source detection processing according to the present embodiment, a caution signal is output when a state in which no sound source is detected from the phase difference region registered in the sound source region continues for a long time. When the space for detecting a sound source is in a house, for example, a device that is frequently used in daily life, such as a television, a ventilation fan, and a water supply, is registered as a sound source. Therefore, if a frequently used device registered in the sound source is not detected as a sound source for a long time, the registered device does not operate due to some abnormality, or a device registered by a person living in the house is used. It is possible that it is not possible. Therefore, when a state in which no sound source is detected from the phase difference region registered in the sound source region continues for a long time as in the present embodiment, the user generates an abnormality in the registered sound source. The possibility can be known early.

  The functional configuration of the sound source detection device 1 shown in the first to fourth embodiments is merely an example, and other configurations may be used as long as the sound source detection processing described in each embodiment can be executed. . For example, the sound source detection device 1 can be configured to acquire an image of a space in which a sound source imaged by the imaging device is detected, and output the acquired image together with a caution signal to an external device when an abnormal state is detected. It is.

  The flowcharts shown in FIGS. 2A and 2B are merely examples of sound source detection processing, and the processing content and processing procedure can be changed as appropriate. Similarly, the flowcharts shown in FIGS. 3A and 3B are merely examples of the sound source number specifying process, and the processing content and processing procedure can be changed as appropriate. Furthermore, the flowchart shown in FIG. 12 is merely an example of processing for calculating a phase spectrum difference, and the processing content and processing procedure can be changed as appropriate. In addition, the flowcharts shown in FIGS. 14, 15A and 15B, and FIGS. 17A to 17D are merely examples of processing for outputting detection results, and the processing content and processing procedure can be changed as appropriate.

  The sound source detection device 1 according to the first to fourth embodiments can be realized using, for example, a computer and a program executed by the computer. Hereinafter, a sound source detection apparatus implemented using a computer and a program will be described with reference to FIG.

FIG. 18 is a diagram illustrating a hardware configuration of a computer.
As illustrated in FIG. 18, the computer 7 includes a central processing unit (CPU) 701, a main storage device 702, an auxiliary storage device 703, an input device 704, and an output device 705. The computer 7 also includes an interface device 706, a communication control device 707, and a storage medium driving device 708. These elements 701 to 708 in the computer 7 are connected to each other by a bus 710 so that data can be exchanged between the elements.

  The CPU 701 is an arithmetic processing unit that controls the overall operation of the computer 7 by executing various programs including an operating system.

  The main storage device 702 includes a read only memory (ROM) and a random access memory (RAM) not shown. In the ROM of the main storage device 702, for example, a predetermined basic control program read by the CPU 701 when the computer 7 is started is recorded in advance. The RAM of the main storage device 702 is used as a working storage area as necessary when the CPU 701 executes various programs. The RAM of the main storage device 702 can be used for storing, for example, a frequency spectrum, a phase difference of each frequency component, a phase difference component ratio, and the like regarding the audio signal to be processed.

  The auxiliary storage device 703 is a storage device having a larger capacity than the main storage device 702 such as a hard disk drive (HDD) or a solid state drive (SSD). The auxiliary storage device 703 can store various programs executed by the CPU 701, various data, and the like. The auxiliary storage device 703 can be used, for example, for storing a sound source detection program including the processes shown in FIGS. 2A and 2B. Further, the auxiliary storage device 703 can be used for storing, for example, a frequency spectrum for a processing target audio signal, a phase difference for each frequency component, a phase difference component ratio, sound source information for a specific sound source, and the like.

  The input device 704 is, for example, a keyboard device or a touch panel device. When an operator (user) of the computer 7 performs an operation such as pressing the input device 704, the input device 704 transmits input information associated with the operation content to the CPU 701. The input device 704 is used, for example, for starting the computer 7 (sound source detection device 1), ending the operation, switching the operation mode, and the like.

  The output device 705 is, for example, a liquid crystal display or a printer. The output device 705 is used, for example, for displaying the operation state of the computer 7 (sound source detection device 1).

  The interface device 706 is a device that connects the computer 7 to other electronic devices, and includes a Universal Serial Bus (USB) standard connector and the like. A device that can be connected to the computer 7 by the interface device 706 includes a microphone array 2 including a plurality of microphones.

  The communication control device 707 is a device that controls various communications between the computer 7 and other communication terminals 9 via the network 8 such as a telephone network or the Internet. The communication control device 707 is used, for example, for outputting caution information when an abnormal state is detected.

  The storage medium driving device 708 reads programs and data recorded in a portable storage medium (not shown), and writes data stored in the auxiliary storage device 703 to the portable storage medium. As the portable storage medium, for example, a flash memory equipped with a USB standard connector can be used. Further, as a portable storage medium, an optical disc such as a Compact Disk (CD), a Digital Versatile Disc (DVD), and a Blu-ray Disc (Blu-ray is a registered trademark) can be used.

  In the computer 7, the CPU 701 reads out and executes the program including the processes of FIGS. 2A and 2B from the auxiliary storage device 703 and the like, and detects the sound source region and the number of sound sources based on the sound signal input from the microphone array 2. I do. In addition, the computer 7 performs processing for detecting an abnormal state based on the detection result of the sound source region and the number of sound sources. When an abnormal state occurs, the computer 7 uses the communication control device 707 to pay attention to the predetermined communication terminal 9. Output a signal.

  Note that the computer 7 used as the sound source detection device 1 does not need to include all the components shown in FIG. 18, and some of the components can be omitted depending on applications and conditions. For example, the computer 7 may be one in which the storage medium driving device 708 is omitted. For example, the computer 7 may be one in which the microphone array 2 is directly connected to the printed circuit board and the interface device 706 is omitted.

The following additional notes are disclosed for each of the embodiments described above.
(Appendix 1)
Based on the frequency spectrum of a plurality of audio signals acquired from a plurality of microphones of the microphone array device, a phase difference calculation unit that calculates a phase difference for each frequency component in the audio signal;
Based on the calculated phase difference for each frequency component, and a sound source detector that detects the direction and number of sound sources in which the sound source exists,
The sound source detector
A frequency component for calculating the number of frequency components for each phase difference region based on a plurality of phase difference regions divided based on the direction of arrival of sound arriving at the microphone and the calculated phase difference for each frequency component A number calculator,
A phase difference component ratio calculation unit that calculates a ratio of the number of frequency components of the plurality of phase difference regions in the audio signal based on the calculated number of frequency components for each phase difference region;
A transition information calculation unit that calculates transition information of the frequency component number ratio based on a temporal change in the frequency component number ratio;
The sound source detection unit determines a phase difference region where the sound source exists based on the number of frequency components for each phase difference region, and also includes a phase difference where the sound source exists based on transition information of the ratio of the number of frequency components. Determine the number of sound sources in the region,
A sound source detection device characterized by that.
(Appendix 2)
The sound source detection unit determines that the number of sound sources in the phase difference region has changed when the amount of change in the ratio of the number of frequency components in the phase difference region where the sound source exists exceeds a predetermined amount of change.
The sound source detection apparatus according to Supplementary Note 1, wherein
(Appendix 3)
The sound source detector
When the ratio of the number of frequency components in the phase difference region where the sound source exists is equal to or greater than a predetermined threshold, it is determined that the number of sound sources in the phase difference region has increased.
The sound source detection apparatus according to Supplementary Note 1, wherein
(Appendix 4)
The sound source detector
A predetermined number of phase difference regions are extracted from the plurality of phase difference regions in descending order of the number of frequency components (S6), and the phase difference in which the sound source exists based on the extracted number of frequency components in the predetermined number of phase difference regions Determine the area,
The sound source detection apparatus according to Supplementary Note 1, wherein
(Appendix 5)
A stationary noise estimation unit for estimating an amplitude level of stationary noise included in the audio signal acquired from the microphone;
The phase difference calculating unit calculates a phase difference of a frequency component having an amplitude spectrum larger than an amplitude level of the stationary noise among the frequency components in the frequency spectrum of the audio signal;
The sound source detection apparatus according to Supplementary Note 1, wherein
(Appendix 6)
Based on the phase difference region where the sound source exists and the detection result of the number of sound sources, further comprising an abnormal state detection unit for detecting an abnormal state in the detection target space of the sound source,
The sound source detection apparatus according to Supplementary Note 1, wherein
(Appendix 7)
The abnormal state detection unit detects the abnormal state based on a duration of a state in which the sound source is not detected;
The sound source detection apparatus according to appendix 6, wherein
(Appendix 8)
The abnormal state detection unit,
Detecting the abnormal state based on a duration of a state in which the number of sound sources is increased in a phase difference region where the sound source exists;
The sound source detection apparatus according to appendix 6, wherein
(Appendix 9)
A sound source information holding unit for holding sound source information including information indicating a phase difference region where the sound source exists,
The abnormal state detection unit detects an abnormal state in the detection target space of the sound source based on the detection result of the phase difference region where the sound source exists and the number of sound sources, and the sound source information.
The sound source detection apparatus according to appendix 6, wherein
(Appendix 10)
The abnormal state detection unit,
When a state in which a sound source is not detected from other phase difference areas excluding a phase difference area where the sound source exists in the sound source information continues for a predetermined time or more, it is determined as the abnormal state.
The sound source detection device according to appendix 9, wherein
(Appendix 11)
The abnormal state detection unit,
In the phase difference region other than the phase difference region where the sound source exists in the sound source information, there is a phase difference region in which the state in which the sound source is detected continues for a predetermined time or more. To determine,
The sound source detection device according to appendix 9, wherein
(Appendix 12)
The abnormal state detection unit,
A state in which the sound source is not detected by the sound source detection unit in the phase difference region where the sound source exists in the sound source information, and the sound source is not detected in the phase difference region where the sound source is not detected Is determined to be in the abnormal state when is continued for a predetermined time,
The sound source detection device according to appendix 9, wherein
(Appendix 13)
The abnormal state detection unit,
In the phase difference region where the sound source exists in the sound source information, when there is a phase difference region in which the state where the number of sound sources is increased continues for a predetermined time or more, it is determined as the abnormal state.
The sound source detection device according to appendix 9, wherein
(Appendix 14)
Computer
A plurality of audio signals acquired from a plurality of microphones of the microphone array device are converted into a frequency spectrum to calculate a phase difference for each frequency component,
Based on the plurality of phase difference regions divided based on the direction of arrival of the sound arriving at the microphone and the calculated phase difference for each frequency component, the number of frequency components for each phase difference region is calculated,
Based on the calculated number of frequency components for each phase difference region, determine the phase difference region where the sound source exists,
Based on the frequency component for each phase difference region, the ratio of the number of frequency components of the plurality of phase difference regions in the audio signal is calculated,
Determining the number of sound sources in the phase difference region where the sound source exists, based on the time change of the ratio of the number of frequency components;
A sound source detection method characterized by executing processing.
(Appendix 15)
In the process of determining the number of sound sources, when the amount of change in the ratio of the number of frequency components in the phase difference region where the sound source exists exceeds a predetermined amount of change, the computer counts the number of sound sources in the phase difference region. Is determined to have changed,
15. The sound source detection method according to appendix 14, wherein
(Appendix 16)
In the process of determining the number of sound sources, the computer increases the number of sound sources in the phase difference region when the ratio of the number of frequency components in the phase difference region in which the sound source exists is equal to or greater than a predetermined threshold. To determine,
15. The sound source detection method according to appendix 14, wherein
(Appendix 17)
In the process of determining the phase difference area where the sound source exists, the computer extracts a predetermined number of phase difference areas from the plurality of phase difference areas in descending order of the number of frequency components, and extracts the predetermined number of phase differences. Determining a phase difference region where the sound source exists based on the number of frequency components in the region;
15. The sound source detection method according to appendix 14, wherein
(Appendix 18)
In the process of calculating the phase difference, the computer estimates an amplitude level of stationary noise included in the audio signal acquired from the microphone for each frequency component, and among the frequency components in the frequency spectrum of the audio signal Calculating a phase difference of frequency components whose amplitude spectrum is larger than the amplitude level of the stationary noise;
15. The sound source detection method according to appendix 14, wherein
(Appendix 19)
The computer further executes a process of detecting an abnormal state in the detection target space of the sound source based on a detection result of the phase difference region where the sound source exists and the number of sound sources. The described sound source detection method.
(Appendix 20)
In the process of detecting the abnormal state, the computer detects the abnormal state based on a duration of a state in which the sound source is not detected.
The sound source detection method according to appendix 19, characterized by:
(Appendix 21)
In the process of detecting the abnormal state, the computer detects the abnormal state based on a duration of a state in which the number of sound sources has increased in a phase difference region where the sound source exists.
The sound source detection method according to appendix 19, characterized by:
(Appendix 22)
In the process of detecting the abnormal state, the computer includes a sound source including information indicating a phase difference region where the sound source exists and a detection result of the number of sound sources, and a phase difference region where the sound source is stored in a storage unit. Detecting an abnormal state in the detection target space of the sound source based on the information;
The sound source detection method according to appendix 19, characterized by:
(Appendix 23)
In the process of detecting the abnormal state, the computer detects the abnormal state when a state in which no sound source is detected from a phase difference region other than the phase difference region where the sound source exists in the sound source information continues for a predetermined time or more. It is determined that
The sound source detection method according to Supplementary Note 22, wherein
(Appendix 24)
In the process of detecting the abnormal state, the computer continues the state in which the sound source is detected in a phase difference region other than the phase difference region where the sound source exists in the sound source information for a predetermined time or more. When there is a phase difference region, it is determined that the state is abnormal.
The sound source detection method according to Supplementary Note 22, wherein
(Appendix 25)
In the process of detecting the abnormal state, the computer includes a phase difference region in which the sound source is not detected by the sound source detection unit in the phase difference region where the sound source exists in the sound source information, and the sound source is detected. When the state where the sound source is not detected in the phase difference region that has not been continued for a predetermined time, it is determined that the abnormal state,
The sound source detection method according to Supplementary Note 22, wherein
(Appendix 26)
In the process of detecting the abnormal state, the computer includes a phase difference region in which the number of sound sources has increased for a predetermined time or more in the phase difference region in which the sound source exists in the sound source information. If it is determined that the abnormal state,
The sound source detection method according to Supplementary Note 22, wherein
(Appendix 27)
A plurality of audio signals acquired from a plurality of microphones of the microphone array device are converted into a frequency spectrum to calculate a phase difference for each frequency component,
Based on the plurality of phase difference regions divided based on the direction of arrival of the sound arriving at the microphone and the calculated phase difference for each frequency component, the number of frequency components for each phase difference region is calculated,
Based on the calculated number of frequency components for each phase difference region, determine the phase difference region where the sound source exists,
Based on the number of frequency components for each phase difference region, calculate the ratio of the number of frequency components in the plurality of phase difference regions in the audio signal,
Determining the number of sound sources in the phase difference region where the sound source exists, based on the time change of the ratio of the frequency components;
A program that causes a computer to execute processing.

DESCRIPTION OF SYMBOLS 1 Sound source detection apparatus 101 Audio | voice signal reception part 102 Conversion part 103 Phase difference calculation part 104 Sound source area / sound source number detection part 104a Frequency component number calculation part 104b Phase difference component ratio calculation part 104c Component ratio transition information calculation part 104d Component ratio holding part 105 detection result output unit 105a duration measurement unit 105b abnormal state detection unit 105c attention signal output unit 105d detection sound source information holding unit 106 sound source information management unit 106a sound source information registration unit 106b sound source information holding unit 2 microphone array 201, 202 microphone 401, 402 Sound source 7 Computer 701 CPU
702 Main storage device 703 Auxiliary storage device 704 Input device 705 Output device 706 Interface device 707 Communication device 708 Storage medium drive device 710 Bus 8 Network 9 Communication terminal

Claims (8)

Based on the frequency spectrum of a plurality of audio signals acquired from a plurality of microphones of the microphone array device, a phase difference calculation unit that calculates a phase difference for each frequency component in the audio signal;
Based on the calculated phase difference for each frequency component, and a sound source detector that detects the direction and number of sound sources in which the sound source exists,
The sound source detector
A frequency component for calculating the number of frequency components for each phase difference region based on a plurality of phase difference regions divided based on the direction of arrival of sound arriving at the microphone and the calculated phase difference for each frequency component A number calculator,
A phase difference component ratio calculation unit that calculates a ratio of the number of frequency components of the plurality of phase difference regions in the audio signal based on the calculated number of frequency components for each phase difference region;
A transition information calculation unit that calculates transition information of the frequency component number ratio based on a temporal change in the frequency component number ratio;
The sound source detection unit determines a phase difference region where the sound source exists based on the number of frequency components for each phase difference region, and also includes a phase difference where the sound source exists based on transition information of the ratio of the number of frequency components. Determine the number of sound sources in the region,
A sound source detection device characterized by that. The sound source detection unit determines that the number of sound sources in the phase difference region has changed when the amount of change in the ratio of the number of frequency components in the phase difference region where the sound source exists exceeds a predetermined amount of change.
The sound source detection device according to claim 1. The sound source detection unit determines that the number of sound sources in the phase difference region has increased when a ratio of the number of frequency components in the phase difference region in which the sound source exists is equal to or greater than a predetermined threshold.
The sound source detection device according to claim 1. A stationary noise estimation unit for estimating an amplitude level of stationary noise included in the audio signal acquired from the microphone;
The phase difference calculating unit calculates a phase difference of a frequency component having an amplitude spectrum larger than an amplitude level of the stationary noise among the frequency components in the frequency spectrum of the audio signal;
The sound source detection device according to claim 1. Based on the phase difference region where the sound source exists and the detection result of the number of sound sources, further comprising an abnormal state detection unit for detecting an abnormal state in the detection target space of the sound source,
The sound source detection device according to claim 1. A sound source information holding unit for holding sound source information including information indicating a phase difference region where the sound source exists,
The abnormal state detection unit detects an abnormal state in the detection target space of the sound source based on the detection result of the phase difference region where the sound source exists and the number of sound sources, and the sound source information.
The sound source detection device according to claim 5. Computer
A plurality of audio signals acquired from a plurality of microphones of the microphone array device are converted into a frequency spectrum to calculate a phase difference for each frequency component,
Based on the plurality of phase difference regions divided based on the direction of arrival of the sound arriving at the microphone and the calculated phase difference for each frequency component, the number of frequency components for each phase difference region is calculated,
Based on the calculated number of frequency components for each phase difference region, determine the phase difference region where the sound source exists,
Based on the number of frequency components for each phase difference region, calculate the ratio of the number of frequency components in the plurality of phase difference regions in the audio signal,
Determining the number of sound sources in the phase difference region where the sound source exists, based on the time change of the ratio of the number of frequency components,
A sound source detection method characterized by executing processing. A plurality of audio signals acquired from a plurality of microphones of the microphone array device are converted into a frequency spectrum to calculate a phase difference for each frequency component,
Based on the plurality of phase difference regions divided based on the direction of arrival of the sound arriving at the microphone and the calculated phase difference for each frequency component, the number of frequency components for each phase difference region is calculated,
Based on the calculated number of frequency components for each phase difference region, determine the phase difference region where the sound source exists,
Based on the number of frequency components for each phase difference region, calculate the ratio of the number of frequency components in the plurality of phase difference regions in the audio signal,
Determining the number of sound sources in the phase difference region where the sound source exists, based on the time change of the ratio of the number of frequency components;
A program that causes a computer to execute processing.

Images