CN107705785A - Sound localization method, intelligent sound box and the computer-readable medium of intelligent sound box - Google Patents

Abstract

The invention provides a sound source localization method for an intelligent speaker, an intelligent speaker and a computer-readable medium. The method includes: if it is determined that the voice signal sent by the target sound source needs to be collected, acquiring at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying a preset wake-up word sent by the target sound source; Two signal receiving modules in the module pair receive the time difference of the first voice signal; according to the first voice signal and the time difference of each signal receiving module pair receiving the first voice signal, determine the orientation of the target sound source that sends out the first voice signal. The technical solution of the present invention can locate the target sound source in a scene with many sound sources. In this way, the smart speaker can only collect the voice signal of the target sound source in the positioning direction, and then provide the user corresponding to the target sound source. services; and can effectively enrich the functions of smart speakers, making the use of smart speakers more flexible and convenient.

Description

Sound source localization method for smart speaker, smart speaker and computer readable medium

【Technical field】

The invention relates to the technical field of computer applications, in particular to a sound source localization method for a smart speaker, a smart speaker and a computer-readable medium.

【Background technique】

With the development of science and technology, smart devices gradually enter users' homes, forming an intelligent furniture environment. For example, smart speakers, as an intelligent device in smart homes, can help users check music, weather, chat, dialogue, etc., so smart speakers need to have voice recognition, semantic analysis, content services, speech generation, voice TTS ( TextToSpeech; TTS) broadcast feedback and other functions.

In the prior art, the smart speakers are all provided with a default wake-up word, and the smart speakers can be in a dormant state when not working. When the user needs to start the smart speaker, he can call the wake-up word of the smart speaker through voice, and the smart speaker detects that its own wake-up word is awakened, and then starts to work. Receive the voice query (Query) input by the user, then perform voice recognition, semantic analysis, query the query result of the voice query, and then generate the speech of the feedback information according to the query result, and perform TTS conversion on the speech of the feedback information, and send it to the Users broadcast feedback information.

However, in the prior art, the smart speaker cannot locate the sound source. If multiple users around the smart speaker call the smart speaker at the same time, the multiple users are equivalent to multiple sound sources, which will cause confusion in the work of the smart speaker and cannot be realized. Voice Query query.

【Content of invention】

The invention provides a method for locating a sound source of an intelligent speaker, the intelligent speaker and a computer-readable medium, which are used for realizing the location of the sound source by the intelligent speaker.

The present invention provides a sound source localization method for an intelligent speaker, the method comprising:

If it is determined that the voice signal sent by the target sound source needs to be collected, at least two groups of signal receiving modules on the smart speaker are obtained to receive the first voice signal carrying the preset wake-up word sent by the target sound source; the preset wake-up word is used waking up the smart speaker for the target sound source;

Acquiring the time difference between the two signal receiving modules in each group of the signal receiving module pairs receiving the first voice signal;

The orientation of the target sound source that sends out the first voice signal is determined according to the first voice signal and the time difference between each of the signal receiving modules receiving the first voice signal.

Further optionally, in the above-mentioned method, according to the first voice signal and the time difference between each of the signal receiving modules receiving the first voice signal, determine the target voice that emits the first voice signal. After the location of the source, the method also includes:

Rotate the positioning indicator mark to the azimuth of the target sound source and/or light up the positioning indicator light to the azimuth of the target sound source to inform the user corresponding to the target sound source that the azimuth of the target sound source has been fixed Sure.

Further optionally, in the method as described above, obtaining the time difference between two signal receiving modules in each pair of signal receiving modules receiving the first voice signal specifically includes:

Taking the first group of signal receiving module pairs in the at least two groups of signal receiving module pairs as a reference object, selecting the candidate direction θ of the target sound source;

For each pair of signal receiving modules, according to the candidate direction θ of the target sound source, obtain the time difference t0 at which two of the signal receiving modules in the corresponding pair of signal receiving modules receive the first speech signal, wherein Said t0 is a function of said θ.

Further optionally, in the above-mentioned method, according to the first voice signal and the time difference between each of the signal receiving modules receiving the first voice signal, determine the target voice that emits the first voice signal. The location of the source, including:

According to the time difference t0 of each group of signal receiving modules receiving the first voice signal, the first voice signals received by the two signal receiving modules in each group of the signal receiving module pairs are timed. alignment processing;

calculating the correlation of each group of the signal receiving modules with respect to the corresponding two aligned first speech signals;

superimposing the corresponding correlations of each group of the signal receiving modules to obtain the total correlation;

Obtaining the candidate direction θ corresponding to the target sound source when the total correlation takes a maximum value is the target direction of the target sound source.

Further optionally, in the above method, according to the time difference t0 of each pair of signal receiving modules receiving the first voice signal, the signal received by the two signal receiving modules in each pair of signal receiving modules The first speech signal is aligned in time, specifically including:

Delaying the time difference t0 of the first received first voice signal among the two first voice signals received by each pair of signal receiving modules, or delaying the two received voice signals by each pair of signal receiving modules The first voice signal received later in the first voice signal is advanced by the time difference t0, so that the two first voice signals are aligned in time.

Further optionally, in the above-mentioned method, before acquiring at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source, the method further includes:

Determine the need to collect the voice signal from the target sound source;

Further optionally, determining that the voice signal emitted by the target sound source needs to be collected includes:

acquiring the first voice signal carrying the preset wake-up word sent by the target sound source;

extracting the preset wake-up word from the first voice signal;

extracting voiceprint features of the first voice signal;

judging whether the preset wake-up word matches the voiceprint feature of the first voice signal according to the pre-stored correspondence between the preset wake-up word and the voiceprint feature of the target sound source;

If they match, it is determined that the voice signal from the target sound source needs to be collected.

Further optionally, in the above method, before acquiring the first voice signal carrying the preset wake-up word sent by the target sound source, the method further includes:

receiving a second voice signal carrying the preset wake-up word input by a user voice corresponding to the target sound source;

extracting the preset wake-up word from the second voice signal;

extracting the voiceprint feature of the second speech signal as the voiceprint feature of the target sound source;

A correspondence relationship between the preset wake-up word and the voiceprint feature of the target sound source is established and stored.

The present invention provides a kind of intelligent sound box, and described intelligent sound box comprises:

The signal acquisition module is used to obtain at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source if it is determined that the voice signal sent by the target sound source needs to be collected; The preset wake-up word is used for the target sound source to wake up the smart speaker;

A time difference acquiring module, configured to acquire the time difference between two signal receiving modules in each group of signal receiving module pairs receiving the first voice signal;

The positioning module is configured to determine the azimuth of the target sound source that sends out the first voice signal according to the first voice signal and the time difference between each of the signal receiving modules receiving the first voice signal.

Further optionally, the above smart speaker also includes:

The positioning indication module is used to rotate the positioning indication mark to the azimuth of the target sound source and/or light the positioning indicator light to the azimuth of the target sound source, so as to inform the user corresponding to the target sound source that the target sound source The direction of the sound source has been determined.

Further optionally, in the smart speaker as described above, the time difference acquisition module is specifically used for:

Taking the first group of signal receiving module pairs in the at least two groups of signal receiving module pairs as a reference object, selecting the candidate direction θ of the target sound source;

For each pair of signal receiving modules, according to the candidate direction θ of the target sound source, obtain the time difference t0 at which two of the signal receiving modules in the corresponding pair of signal receiving modules receive the first speech signal, wherein Said t0 is a function of said θ.

Further optionally, in the smart speaker as described above, the positioning module is specifically used for:

According to the time difference t0 of each group of signal receiving modules receiving the first voice signal, the first voice signals received by the two signal receiving modules in each group of the signal receiving module pairs are timed. alignment processing;

calculating the correlation of each group of the signal receiving modules with respect to the corresponding two aligned first speech signals;

superimposing the corresponding correlations of each group of the signal receiving modules to obtain the total correlation;

Obtaining the candidate direction θ corresponding to the target sound source when the total correlation takes a maximum value is the target direction of the target sound source.

Further optionally, in the smart speaker as described above, the positioning module is specifically configured to combine the first received first voice signal among the two first voice signals received by each of the signal receiving modules Delay the time difference t0, or advance the time difference t0 to the first speech signal received by each of the two first speech signals received by each of the signal receiving modules, so that the two first Speech signals are aligned in time.

Further optionally, in the smart speaker as described above, the smart speaker further includes:

A determination module, configured to determine the need to collect the voice signal from the target sound source;

Further, the determining module is specifically used for:

acquiring the first voice signal carrying the preset wake-up word sent by the target sound source;

extracting the preset wake-up word from the first voice signal;

extracting voiceprint features of the first voice signal;

judging whether the preset wake-up word matches the voiceprint feature of the first voice signal according to the pre-stored correspondence between the preset wake-up word and the voiceprint feature of the target sound source;

If they match, it is determined that the voice signal from the target sound source needs to be collected.

Further optionally, in the smart speaker as described above, the smart speaker further includes:

A receiving module, configured to receive a second voice signal carrying the preset wake-up word input by the user's voice corresponding to the target sound source;

An extraction module, configured to extract the preset wake-up word from the second voice signal;

The extraction module is further configured to extract the voiceprint feature of the second speech signal as the voiceprint feature of the target sound source;

The establishment module is used to establish and store the corresponding relationship between the preset wake-up word and the voiceprint feature of the target sound source.

The present invention also provides a smart speaker, including a plurality of microphones for sending and receiving signals; the smart speaker also includes:

one or more processors;

memory for storing one or more programs,

When the one or more programs are executed by the one or more processors, the one or more processors implement the sound source localization method for the smart speaker as described above.

The present invention also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the method for locating a sound source of a smart speaker as described above is realized.

The sound source localization method of the smart speaker, the smart speaker and the computer-readable medium of the present invention, if it is determined that the voice signal sent by the target sound source needs to be collected, the voice signal sent by the target sound source is received by at least two groups of signal receiving modules on the smart speaker. Carrying the first voice signal of the preset wake-up word; obtaining the time difference between the two signal receiving modules in each group of signal receiving module pairs receiving the first voice signal; according to the first voice signal and each signal receiving module pair receiving the first voice signal The time difference is used to determine the azimuth of the target sound source that sends out the first voice signal. The technical solution of the present invention can locate the target sound source in a scene with many sound sources. In this way, the smart speaker can only collect the voice signal of the target sound source in the positioning direction, and then provide the user corresponding to the target sound source. services; and can effectively enrich the functions of smart speakers, making the use of smart speakers more flexible and convenient.

【Description of drawings】

FIG. 1 is a flow chart of Embodiment 1 of a sound source localization method for a smart speaker according to the present invention.

Fig. 2 is an application scene diagram of the sound source localization method of the smart speaker according to the present invention.

FIG. 3 is another application scene diagram of the sound source localization method for a smart speaker according to the present invention.

FIG. 4 is a flow chart of Embodiment 2 of the sound source localization method for a smart speaker according to the present invention.

FIG. 5 is a structural diagram of Embodiment 1 of the smart speaker of the present invention.

FIG. 6 is a structural diagram of Embodiment 2 of the smart speaker of the present invention.

FIG. 7 is a structural diagram of Embodiment 3 of the smart speaker of the present invention.

Fig. 8 is an example diagram of a smart speaker provided by the present invention.

【detailed description】

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

FIG. 1 is a flow chart of Embodiment 1 of a sound source localization method for a smart speaker according to the present invention. As shown in Figure 1, the sound source localization method of the smart speaker of the present embodiment may specifically include the following steps:

100. If it is determined that the voice signal sent by the target sound source needs to be collected, acquire at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source;

The execution subject of the sound source localization method for a smart speaker in this embodiment is a smart speaker. The preset wake-up word in this embodiment is used for the target sound source to wake up the smart speaker. The target sound source in this embodiment is preferably a user interacting with the smart speaker. In order to receive the user's voice query and broadcast feedback information to the user based on the user's voice query, the smart speaker in this embodiment is provided with a signal receiving module and a signal sending module. For example, the signal receiving module and the signal sending module on the smart speaker can be integrated, such as a microphone integrated in the smart speaker, to receive signals from all directions and broadcast feedback information to all directions. Optionally, in this embodiment, an even number of microphones may be arranged symmetrically on the smart speaker, for example, four microphones or four groups of microphones may be uniformly arranged around the smart speaker, and each group may include two.

In the usage scenario of this embodiment, the smart speaker can be called by multiple sound sources. Since the smart speaker cannot process multiple voice queries at the same time, the smart speaker can determine the voice signal from the target sound source that needs to be collected through its own analysis, for example , it may be determined that the first received speech signal needs to be collected, or the target sound source among multiple sound sources is obtained through other strategies, and it is determined that the speech signal emitted by the target sound source needs to be collected. At this time, it is necessary to further locate the target sound source. In this embodiment, it is first necessary to obtain at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source, so as to use each The group signal receiving module locates the target sound source on the received first voice signal.

The pair of signal receiving modules in this embodiment may be a pair of microphones on the smart speaker, and each pair of microphones includes two microphones, and the two microphones are any two microphones on the smart speaker. The localization is not accurate enough because a set of microphone pairs may localize target sound sources from two symmetrical directions. In this embodiment, in order to locate the azimuth of the target sound source, at least two groups of microphone pairs need to be selected to locate the azimuth of the target sound source. The relationship between the at least two microphone pairs selected in this embodiment is not limited. For example, one group may be two adjacent microphones, and the other pair may be adjacent or two opposite microphones located on a diagonal. Since different microphones are at different positions from the target sound source, different microphones receive different timings of the same voice signal from the target sound source.

In this embodiment, if it is determined that the voice signal sent by the target sound source needs to be collected, it is necessary to obtain at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source, for example, If the preset wake-up word of the smart speaker is "Dabai", and the user sends out a voice signal of "Dabai, Dabai", it may be the first voice signal sent by the target sound source. That is to say, in this embodiment, when locating the target sound source, the target sound source can use the preset wake-up word to wake up the first voice signal of the smart speaker for positioning, without the need to collect other voices of the target sound source Signal.

101. Obtain the time difference between two signal receiving modules in each group of signal receiving module pairs receiving the first voice signal;

For example, if the signal receiving module pair is a microphone pair, since different microphones have different distances from the target sound source, there is a time difference between the two microphones in each microphone pair receiving the first voice signal of the target sound source. Optionally, this step 101 may specifically include: taking the first group of signal receiving module pairs in at least two groups of signal receiving module pairs as a reference object, selecting the candidate direction θ of the target sound source; for each signal receiving module pair, according to For the candidate direction θ of the target sound source, obtain the time difference t0 between two signal receiving modules in the corresponding signal receiving module pair receiving the first speech signal, where t0 is a function of θ.

Specifically, since the microphone pair cannot accurately know the time difference, in this embodiment, the candidate direction θ of the target sound source can be preset first, and then based on the candidate direction θ of the target sound source, it can represent that the two microphones in each microphone pair receive The time difference t0 of the first speech signal of the target sound source. The target sound source in this embodiment is a far-field sound source, that is, the distance between the target sound source and the microphones is far greater than the distance between the microphones. mode to each microphone. At this time, the first group of microphone pairs among the at least two groups of microphone pairs may be selected as a reference object, and the candidate direction of the target sound source relative to the first group of microphone pairs of the reference object is taken as θ. Since the distance between the two microphones in each group of microphone pairs in at least two groups of microphone pairs is known, according to the candidate direction of the target sound source and the distance between the two microphones in each group of microphone pairs, it can be identified The time difference t0 of the first speech signal received by the two microphones in each group of microphone pairs, at this time t0 is a function of the candidate direction θ of the target sound source, and the candidate direction θ of the target sound source can be 0-360 degrees in the space At any angle, that is, the target sound source can be located in any direction from 0 to 360 degrees in the space.

For example, FIG. 2 is an application scene diagram of the sound source localization method for a smart speaker according to the present invention. As shown in Figure 2, A, B and C are the microphones on the smart speaker respectively. In this embodiment, A and B, and A and C can be used to form a microphone pair. The form is transmitted to each microphone. As shown in Figure 2, with the microphone pair A and B as the reference object, the candidate direction of the target sound source can be taken as θ, and then the auxiliary line BO is perpendicular to the parallel wave direction of the target sound source, and AO is the arrival of the first voice signal The distance difference between microphone B and reaching microphone A, AO distance Δd=L×cosθ, where L is equal to the distance between microphones A and B. Further, the time difference t0 between the microphones A and B receiving the first voice signal from the target sound source is equal to the distance of AO divided by the sound velocity V, so the time difference t0 can be expressed as: t0=Δd/V=L×cosθ/V, That is, the time difference t0 is a function of θ.

At this time, for another group of microphone pairs A and C in FIG. 2 , the distance of AC is also equal to L. On the smart speaker, the connection line between microphone pair A and C is perpendicular to the connection line between microphone pair A and B. At this time, according to the geometric relationship of the triangle, the time difference t0 between microphone A and C receiving the first voice signal from the target sound source can be Expressed as: t0=L×sinθ/V.

For example, FIG. 3 is another application scene diagram of the sound source localization method for a smart speaker according to the present invention. Similar to the processing method in FIG. 2 above, the time difference t0 between microphones A and B receiving the first voice signal from the target sound source can be expressed as: t0=Δd/V=L×cosθ/V. And if the straight line where microphones A and C are located is parallel to the parallel wave of the first voice signal, then the time difference t0 between microphones A and C receiving the first voice signal from the target sound source is equal to the length of AC and the length L' of AC Divide by the speed of sound V.

The above Figures 2 and 3 are only examples of two special scenarios. In practical applications, for at least two sets of signal receiving module pairs of smart speakers in any scenario, the first set of signal receiving module pairs can always be selected as the reference object , to obtain the candidate direction θ of the target sound source, and according to the positional relationship of each group of signal receiving module pairs on the smart speaker, the time difference between the two signal receiving modules in each group of signal receiving module pairs receiving the first voice signal can be used as the target The candidate direction θ of the sound source is indicated.

102. Determine an azimuth of a target sound source that sends out the first voice signal according to the first voice signal and the time difference between each signal receiving module for receiving the first voice signal.

When the signal receiving module pair is a microphone pair, for each microphone pair, the time difference of the first voice signal received by the two microphones can be expressed as a function of the candidate direction θ of the target sound source, and the candidate direction θ can be selected from 0-360 degrees Angle in any orientation within the range. Moreover, the first voice signals received by the two microphones can be aligned in time, so that the two first voice signals should have the strongest correlation. Then, by traversing the angles of each azimuth, the angle with the greatest correlation between the two first speech signals is obtained, which is the azimuth angle of the target sound source.

That is, step 102 may specifically include the following steps:

(a1) According to the time difference t0 of each group of signal receiving modules receiving the first voice signal, the first voice signals received by the two signal receiving modules in each signal receiving module are aligned in time;

In this embodiment, during alignment processing, each signal receiving module can delay the time difference t0 of the first received first voice signal among the two first voice signals received by each signal receiving module, or the two first received voice signals received by each signal receiving module The first voice signal received later in the voice signal is advanced by the time difference t0, so that the two first voice signals are aligned in time.

(a2) calculating the correlation of each group of signal receiving modules to the corresponding two aligned first speech signals;

(a3) superimposing the corresponding correlations of each signal receiving module to obtain the total correlation;

(a4) Obtain the candidate direction θ corresponding to the target sound source when the total correlation takes the maximum value as the target direction of the target sound source.

If only a pair of microphone pairs are selected when determining the orientation of the target sound source that sends out the first voice signal, such as microphones A and C in Figure 2, at this time, as shown in Figure 2, there may still be a For a candidate target sound source that is symmetrical about AC with the target sound source, the correlation of the first speech signal after alignment processing can also reach a maximum value, and at this time, the orientation of the target sound source cannot be uniquely determined. However, if the microphone pair A and B are selected to jointly determine the orientation of the target sound source, the orientation of the target sound source can be uniquely determined. Therefore, in this embodiment, it is necessary to obtain at least two groups of signal receiving module pairs, such as microphone pairs, in order to uniquely determine the orientation of the target sound source that sends out the first voice signal.

Specifically, for each pair of signal receiving modules, the first voice signals received by the two signal receiving modules in the group of signal receiving module pairs are aligned, and the alignment processing method refers to the above description. For example, for a pair of microphones a and b, a first receives the first voice signal Y1, and b receives the first voice signal Y2 later, the time difference is t0, and the first voice signal Y1 received by a can be delayed by t0, or The first speech signal Y2 received by b may be advanced by t0. In this embodiment, if the first voice signal received earlier is delayed, the delayed first voice signal is Y1'. The correlation of Y1' and Y2 can then be calculated. For each group of microphone pairs, the corresponding correlation can be obtained in the above way, and then the correlations corresponding to each group of microphone pairs can be added to obtain the total correlation; since Y1' is delayed by t0, t0 is about θ function, therefore, the above step (a4) can determine the value of the total correlation corresponding to each candidate direction θ corresponding to the target sound source by traversing each θ, and obtain the target sound source when the total correlation takes the maximum value The corresponding candidate direction θ is the target direction of the target sound source.

That is to say, if the candidate direction θ of the selected target sound source is exactly the real direction of the sound source, then at this time, align the first speech signals received by the two microphones on the time axis according to the time difference calculated by the candidate direction θ Afterwards, there should be the strongest correlation between the two signals. Conversely, if the selected candidate direction θ is not the real direction of the sound source, then after the time difference calculated according to the candidate direction θ is aligned, the correlation between the first speech signals received by the two microphones becomes weaker. Therefore, by detecting the correlation of the aligned first speech signals of the two microphones corresponding to each candidate direction θ, the possibility of the first speech signal coming from each candidate direction θ can be judged. The stronger the correlation, the stronger the sound. It may be incident from the θ direction.

Further optionally, after step 102 "according to the first voice signal and the time difference between each signal receiving module for receiving the first voice signal, determine the orientation of the target sound source that sends out the first voice signal", it may also include: rotating the positioning indicator mark To the azimuth of the target sound source and/or to light up the positioning indicator light to the azimuth of the target sound source, to inform the user corresponding to the target sound source that the azimuth of the target sound source has been determined.

That is to say, after the smart speaker locates the target sound source, it needs to give certain feedback to the user of the target sound source, so as to inform the user that the target sound source has been positioned, and the user can interact with the smart speaker. Smart speakers provide services. In this embodiment, the smart speaker can be provided with a positioning indicator mark, which can be a rotatable pointer. After the smart speaker locates the target sound source, it can rotate the positioning indicator mark to the direction of the target sound source. , so that the user in this direction can see that the target sound source has been located. Or the smart speaker can also be provided with a positioning indicator light. For example, the positioning indicator light can be set on the pointer of the positioning indicator mark. In this way, after the smart speaker locates the direction of the target sound source, it can also light up to the direction of the target sound source. The positioning indicator light is used to inform the user corresponding to the target sound source that the direction of the target sound source has been determined. The above two methods can be used alone or in combination.

In the sound source localization method of the smart speaker in this embodiment, if it is determined that the voice signal sent by the target sound source needs to be collected, at least two groups of signal receiving modules on the smart speaker receive the first voice signal carrying the preset wake-up word sent by the target sound source. A voice signal; obtain the time difference between the two signal receiving modules in each group of signal receiving modules receiving the first voice signal; determine the first voice according to the first voice signal and the time difference between each signal receiving module receiving the first voice signal The azimuth of the signal's target sound source. The technical solution of this embodiment can locate the target sound source in a scene where there are many sound sources. In this way, the smart speaker can only collect the voice signal of the target sound source in the positioning direction, and then provide the user corresponding to the target sound source. Provide services; and can effectively enrich the functions of smart speakers, making the use of smart speakers more flexible and convenient.

FIG. 4 is a flow chart of Embodiment 2 of the sound source localization method for a smart speaker according to the present invention. As shown in FIG. 4 , the sound source localization method for a smart speaker in this embodiment further introduces the technical solution of the present invention in more detail on the basis of the technical solution of the above-mentioned embodiment. As shown in Figure 4, the sound source localization method of the smart speaker of the present embodiment may specifically include the following technical solutions:

200. Receive a second voice signal carrying a preset wake-up word input by a user voice corresponding to a target sound source;

201. Extract a preset wake-up word from the second voice signal;

202. Extract the voiceprint feature of the second voice signal as the voiceprint feature of the target sound source;

203. Establish and store the correspondence between the preset wake-up word and the voiceprint feature of the target sound source;

In the application scenario of the method for localizing a sound source of a smart speaker in this embodiment, the smart speaker may support adding and setting a preset wake-up word. For example, on the basis that the wake-up word of the smart speaker in the prior art cannot be changed, in this embodiment, the owner of the smart speaker can set a preset wake-up word in the smart speaker for himself or a family member. For example, if the default wake-up word of the smart speaker is small A, and the owner is the first person to talk to the smart speaker after purchasing the smart speaker, the default wake-up word can be the owner's private wake-up word. With the consent of the owner, the owner can also allow other family members to set their private wake-up words on the smart speaker. For example, in a similar manner, when the user corresponding to the target sound source sets a private wake-up word, the user corresponding to the target sound source can voice input the second voice signal carrying the preset wake-up word. Lovely is the preset wake-up word set by the user of the target sound source for the smart speaker. At this time, the smart speaker extracts the preset wake-up word from the second voice signal; and extracts the voiceprint feature of the second voice signal as the voiceprint feature of the target sound source; then establishes and stores the preset wake-up word and the voice print feature of the target sound source Correspondence relationship of texture features. That is to say, the voiceprint feature of the voice signal calling the preset wake-up word must be the voiceprint feature in the corresponding relationship, or the wake-up word carried in the voice signal called by using the voiceprint feature must be the preset wake-up word in the corresponding relationship. Set a wake-up word, otherwise the smart speaker can ignore it.

The above process of establishing and storing the correspondence between the preset wake-up word and the voiceprint feature of the target sound source can be performed in advance, so that the correspondence can be directly used for subsequent detection.

204. Obtain a first voice signal carrying a preset wake-up word sent by the target sound source;

205. Extract a preset wake-up word from the first voice signal;

206. Extracting voiceprint features of the first voice signal;

207. According to the correspondence between the pre-stored preset wake-up words and the voiceprint features of the target sound source, determine whether the preset wake-up words match the voiceprint features of the first voice signal; if they match, perform step 208; otherwise, perform step 207. 209;

208. Determine that the voice signal from the target sound source needs to be collected, and end.

That is to say, according to the corresponding relationship between the preset wake-up words obtained in step 203 and the voiceprint features of the target sound source, it is judged whether the preset wake-up words obtained in step 205 match the voiceprint features extracted in step 206. If they match, it is determined that the voice signal emitted by the target sound source needs to be collected at this time, that is, the technical solution of the above-mentioned embodiment shown in FIG. 1 can be implemented.

209. It is determined that the preset wake-up word does not match the voiceprint feature, and no operation is performed temporarily.

Or optionally, if the smart speaker determines that the current preset wake-up word does not match the voiceprint features, the smart speaker can also make a certain voice prompt at this time, "Sorry, the wake-up word you used is wrong, and we cannot provide you with services temporarily." ” and similar prompt messages.

The sound source localization method for the smart speaker of this embodiment can further use the voiceprint and the preset wake-up word together to realize the determination of the voice signal by adopting the above solution. In the method of this embodiment, for the same smart speaker, different users can set different wake-up words, and the corresponding relationship between the user's wake-up word and the user's voiceprint feature can be stored in the smart speaker, so that each Users can only use their private wake-up words to wake up the smart speaker and interact with it. In this way, the smart speaker has a good recognition of the voiceprint characteristics and wake-up words of each voice signal, which can not only enhance the smart speaker's recognition of voice signals Positioning accuracy, but also can greatly enhance the user experience.

FIG. 5 is a structural diagram of Embodiment 1 of the smart speaker of the present invention. As shown in FIG. 5 , the smart speaker in this embodiment may specifically include: a signal acquisition module 10 , a time difference acquisition module 11 and a positioning module 12 .

Wherein the signal acquisition module 10 is used to obtain at least two groups of signal receiving modules on the smart speaker to receive the first voice signal of the preset wake-up word sent by the target sound source if it is determined that the voice signal sent by the target sound source needs to be collected; preset The wake-up word is used for the target sound source to wake up the smart speaker;

The time difference acquiring module 11 is used to acquire the time difference when two signal receiving modules in each group of signal receiving modules acquired by the signal acquiring module 10 receive the first voice signal;

The positioning module 12 is used to determine the orientation of the target sound source that sends out the first voice signal according to the first voice signal acquired by the signal acquisition module 10 and the time difference between each signal receiving module receiving the first voice signal acquired by the time difference acquisition module 11 .

For the intelligent speaker of this embodiment, the implementation principle and technical effect of sound source localization by using the above-mentioned modules are the same as those of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

FIG. 6 is a structural diagram of Embodiment 2 of the smart speaker of the present invention. As shown in FIG. 6 , the smart speaker of this embodiment may further include the following technical solutions on the basis of the technical solution of the above-mentioned embodiment shown in FIG. 5 .

As shown in FIG. 6 , the smart speaker of this embodiment may further include a positioning indication module 13 .

The positioning indicator module 13 is used to turn the positioning indicator mark to the orientation of the target sound source positioned by the positioning module 12 and/or light up the positioning indicator light to the orientation of the target sound source positioned by the positioning module 12, so as to inform the corresponding position of the target sound source. User, the orientation of the target sound source has been determined.

Further optionally, in the smart speaker of this embodiment, the time difference acquisition module 11 is specifically used for:

Taking the first group of signal receiving module pairs in at least two groups of signal receiving module pairs as a reference object, selecting the candidate direction θ of the target sound source;

For each pair of signal receiving modules, according to the candidate direction θ of the target sound source, the time difference t0 between two signal receiving modules in the corresponding signal receiving module pair receiving the first voice signal is obtained, where t0 is a function of θ.

Further optionally, in the smart speaker of this embodiment, the positioning module 12 is specifically used for:

According to the time difference t0 of each group of signal receiving modules receiving the first voice signal, the first voice signals received by the two signal receiving modules in each group of signal receiving modules are aligned in time;

Calculate the correlation of each group of signal receiving modules to the corresponding two aligned first speech signals;

Superimpose the corresponding correlations of each group of signal receiving modules to obtain the total correlation;

The candidate direction θ corresponding to the target sound source when the total correlation takes the maximum value is the target direction of the target sound source.

Further optionally, in the smart speaker of this embodiment, the positioning module 12 is specifically configured to delay the first voice signal received first among the two first voice signals received by each signal receiving module by a time difference t0, or to delay each signal The receiving module advances the time difference t0 of the first speech signal received later among the two received first speech signals, so that the two first speech signals are aligned in time.

Further optionally, as shown in Figure 6, the smart speaker of this embodiment also includes:

The determining module 14 is used for determining that the voice signal emitted by the target sound source needs to be collected.

Further optionally, in the smart speaker of this embodiment, the determining module 14 is specifically used for:

Obtaining a first voice signal carrying a preset wake-up word sent by a target sound source;

extracting a preset wake-up word from the first voice signal;

Extracting the voiceprint feature of the first voice signal;

According to the corresponding relationship between the pre-stored preset wake-up word and the voiceprint feature of the target sound source, it is judged whether the preset wake-up word matches the voiceprint feature of the first voice signal;

If they match, it is determined that the voice signal from the target sound source needs to be collected.

Correspondingly, when the determination module 14 determines that it is necessary to collect the voice signal from the target sound source, the trigger signal acquisition module 10 acquires at least two groups of signal receiving modules on the smart speaker to receive the first voice carrying the preset wake-up word sent by the target sound source. Signal.

Further optionally, as shown in Figure 6, the smart speaker of this embodiment also includes:

The receiving module 15 is used to receive the second voice signal carrying the preset wake-up word input by the user's voice corresponding to the target sound source;

The extraction module 16 is used to extract the preset wake-up word from the second voice signal received by the receiving module 15;

The extraction module 16 is also used to extract the voiceprint feature of the second voice signal received by the receiving module 15 as the voiceprint feature of the target sound source;

The establishment module 17 is used to establish and store the correspondence between the preset wake-up words extracted by the extraction module 16 and the voiceprint features of the target sound source.

Correspondingly, the determination module 14 is configured to determine whether the preset wake-up word matches the voiceprint feature of the first voice signal according to the correspondence between the pre-stored preset wake-up word and the voiceprint feature of the target sound source established by the establishment module 17 .

For the intelligent speaker of this embodiment, the implementation principle and technical effect of sound source localization by using the above-mentioned modules are the same as those of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

FIG. 7 is a structural diagram of Embodiment 3 of the smart speaker of the present invention. As shown in FIG. 7 , the smart speaker in this embodiment includes multiple microphones (not shown in the figure) for sending and receiving signals. For example, multiple microphones of the smart speaker may be evenly distributed on the casing of the smart speaker, and the multiple microphones are used to receive the user's voice query, and are also used to broadcast feedback information to the user based on the user's voice query. The smart speaker of this embodiment also includes: one or more processors 30, and a memory 40, the memory 40 is used to store one or more programs, when the one or more programs stored in the memory 40 are used by one or more processors 30, so that one or more processors 30 implement the sound source localization method for the smart speaker in the embodiment shown in Fig. 1-Fig. 4 above. The embodiment shown in FIG. 7 takes a plurality of processors 30 as an example.

For example, FIG. 8 is an example diagram of a smart speaker provided by the present invention. Figure 8 shows a block diagram of an exemplary smart speaker 12a suitable for use in implementing embodiments of the present invention. The smart speaker 12a shown in FIG. 8 is only an example, and should not limit the functions and scope of use of this embodiment of the present invention.

As shown in FIG. 8, the smart speaker 12a of this embodiment is represented in the form of a general-purpose computing device. Components of the smart speaker 12a may include, but are not limited to: one or more processors 16a, a system memory 28a, and a bus 18a connecting different system components (including the system memory 28a and the processor 16a).

Bus 18a represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus structures. These architectures include, by way of example, but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.

Smart speaker 12a typically includes various computer system readable media. These media can be any available media that can be accessed by the smart speaker 12a, including volatile and non-volatile media, removable and non-removable media.

System memory 28a may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30a and/or cache memory 32a. The smart speaker 12a may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34a may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 8, commonly referred to as a "hard drive"). Although not shown in FIG. 8, a disk drive for reading and writing to removable nonvolatile disks (e.g., "floppy disks") may be provided, as well as for removable nonvolatile optical disks (e.g., CD-ROM, DVD-ROM or other optical media) CD-ROM drive. In these cases, each drive may be connected to bus 18a via one or more data media interfaces. The system memory 28a may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of the above-described embodiments of FIGS. 1-6 of the present invention.

A program/utility 40a having a set (at least one) of program modules 42a may be stored, for example, in system memory 28a, such program modules 42a including - but not limited to - an operating system, one or more application programs, other Program modules, as well as program data, each or some combination of these examples may include implementations of network environments. The program module 42a generally executes the functions and/or methods described in the embodiments of FIG. 1 to FIG. 6 described in the present invention.

The smart speaker 12a may also communicate with one or more external devices 14a (such as a keyboard, pointing device, display 24a, etc.), communicate with one or more devices that enable a user to interact with the smart speaker 12a, and/or communicate with Any device (eg, network card, modem, etc.) that enables the smart speaker 12a to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 22a. Moreover, the smart speaker 12a can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through the network adapter 20a. As shown, network adapter 20a communicates with other modules of smart speaker 12a via bus 18a. It should be understood that although not shown, other hardware and/or software modules may be used in conjunction with smart speaker 12a, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

The processor 16a executes various functional applications and data processing by running the programs stored in the system memory 28a, for example, implementing the sound source localization method for the smart speaker shown in the above-mentioned embodiments.

The present invention also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the method for localizing the sound source of the smart speaker as shown in the above-mentioned embodiments is realized.

The computer-readable medium in this embodiment may include the RAM 30a in the system memory 28a in the embodiment shown in FIG. 8 above, and/or the cache memory 32a, and/or the storage system 34a.

With the development of science and technology, the transmission channels of computer programs are no longer limited to tangible media, and can also be directly downloaded from the Internet or obtained in other ways. Therefore, the computer-readable medium in this embodiment may include not only tangible media, but also intangible media.

The computer-readable medium of this embodiment may use any combination of one or more computer-readable mediums. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include: electrical connections with one or more leads, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a data signal carrying computer readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including - but not limited to - electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. .

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including - but not limited to - wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out the operations of the present invention may be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural programming language—such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated units implemented in the form of software functional units may be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, and include several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) or a processor (processor) execute the methods described in various embodiments of the present invention. partial steps. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other various media that can store program codes. .

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (16)

1. a sound source localization method of intelligent speaker, is characterized in that, described method comprises: If it is determined that the voice signal sent by the target sound source needs to be collected, at least two groups of signal receiving modules on the smart speaker are obtained to receive the first voice signal carrying the preset wake-up word sent by the target sound source; the preset wake-up word is used waking up the smart speaker for the target sound source; Acquiring the time difference between the two signal receiving modules in each group of the signal receiving module pairs receiving the first voice signal; The orientation of the target sound source that sends out the first voice signal is determined according to the first voice signal and the time difference between each of the signal receiving modules receiving the first voice signal. 2. The method according to claim 1, wherein, according to the first voice signal and each of the signal receiving modules receiving the time difference of the first voice signal, it is determined to send the first voice signal. After describing the orientation of the target sound source, the method also includes: Rotate the positioning indicator mark to the azimuth of the target sound source and/or light up the positioning indicator light to the azimuth of the target sound source to inform the user corresponding to the target sound source that the azimuth of the target sound source has been fixed Sure. 3. The method according to claim 1, characterized in that, obtaining the time difference at which two of the signal receiving modules in each of the signal receiving modules receive the first voice signal, specifically includes: Taking the first group of signal receiving module pairs in the at least two groups of signal receiving module pairs as a reference object, selecting the candidate direction θ of the target sound source; For each pair of signal receiving modules, according to the candidate direction θ of the target sound source, obtain the time difference t0 at which two of the signal receiving modules in the corresponding pair of signal receiving modules receive the first speech signal, wherein Said t0 is a function of said θ. 4. The method according to claim 3, wherein, according to the first voice signal and each of the signal receiving modules receiving the time difference of the first voice signal, it is determined to send the first voice signal. Describe the direction of the target sound source, including: According to the time difference t0 of each group of signal receiving modules receiving the first voice signal, the first voice signals received by the two signal receiving modules in each group of the signal receiving module pairs are timed. alignment processing; calculating the correlation of each group of the signal receiving modules with respect to the corresponding two aligned first speech signals; superimposing the corresponding correlations of each group of the signal receiving modules to obtain the total correlation; Obtaining the candidate direction θ corresponding to the target sound source when the total correlation takes a maximum value is the target direction of the target sound source. 5. method according to claim 4, is characterized in that, according to each described signal receiving module to receive the time difference t0 of described first speech signal, two described signal receiving modules in each described signal receiving module are received The first voice signal received by the module is aligned in time, specifically including: Delaying the time difference t0 of the first received first voice signal among the two first voice signals received by each pair of signal receiving modules, or delaying the two received voice signals by each pair of signal receiving modules The first voice signal received later in the first voice signal is advanced by the time difference t0, so that the two first voice signals are aligned in time. 6. The method according to claim 1, wherein, before obtaining at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source, the method further include: Determine the need to collect the voice signal from the target sound source; Further, it is determined that the voice signal emitted by the target sound source needs to be collected, specifically including: acquiring the first voice signal carrying the preset wake-up word sent by the target sound source; extracting the preset wake-up word from the first voice signal; extracting voiceprint features of the first voice signal; judging whether the preset wake-up word matches the voiceprint feature of the first voice signal according to the pre-stored correspondence between the preset wake-up word and the voiceprint feature of the target sound source; If they match, it is determined that the voice signal from the target sound source needs to be collected. 7. The method according to claim 6, wherein before acquiring the first voice signal carrying the preset wake-up word sent by the target sound source, the method further comprises: receiving a second voice signal carrying the preset wake-up word input by a user voice corresponding to the target sound source; extracting the preset wake-up word from the second voice signal; extracting the voiceprint feature of the second speech signal as the voiceprint feature of the target sound source; A correspondence relationship between the preset wake-up word and the voiceprint feature of the target sound source is established and stored. 8. A smart speaker, characterized in that the smart speaker includes: The signal acquisition module is used to obtain at least two groups of signal receiving modules on the smart speaker to receive the first voice signal carrying the preset wake-up word sent by the target sound source if it is determined that the voice signal sent by the target sound source needs to be collected; The preset wake-up word is used for the target sound source to wake up the smart speaker; A time difference acquiring module, configured to acquire the time difference between two signal receiving modules in each group of signal receiving module pairs receiving the first voice signal; The positioning module is configured to determine the azimuth of the target sound source that sends out the first voice signal according to the first voice signal and the time difference between each of the signal receiving modules receiving the first voice signal. 9. The smart speaker according to claim 8, further comprising: The positioning indication module is used to rotate the positioning indication mark to the azimuth of the target sound source and/or light the positioning indicator light to the azimuth of the target sound source, so as to inform the user corresponding to the target sound source that the target sound source The direction of the sound source has been determined. 10. The smart speaker according to claim 8, wherein the time difference acquisition module is specifically used for: Taking the first group of signal receiving module pairs in the at least two groups of signal receiving module pairs as a reference object, selecting the candidate direction θ of the target sound source; For each pair of signal receiving modules, according to the candidate direction θ of the target sound source, the time difference t0 at which two of the signal receiving modules in the corresponding pair of signal receiving modules receive the first speech signal is obtained, wherein Said t0 is a function of said θ. 11. The smart speaker according to claim 10, wherein the positioning module is specifically used for: According to the time difference t0 of each group of signal receiving modules receiving the first voice signal, the first voice signals received by the two signal receiving modules in each group of the signal receiving module pairs are timed. alignment processing; calculating the correlation of each group of the signal receiving modules with respect to the corresponding two aligned first speech signals; superimposing the corresponding correlations of each group of the signal receiving modules to obtain the total correlation; Obtaining the candidate direction θ corresponding to the target sound source when the total correlation takes a maximum value is the target direction of the target sound source. 12. The smart speaker according to claim 11, wherein the positioning module is specifically configured to place the first received first voice signal among the two first voice signals received by each of the signal receiving modules. A voice signal is delayed by the time difference t0, or the first voice signal received by each of the two first voice signals received by each of the signal receiving modules is advanced by the time difference t0, so that the two received The first speech signals are aligned in time. 13. The smart speaker according to claim 8, wherein the smart speaker further comprises: A determination module, configured to determine the need to collect the voice signal from the target sound source; Further, the determining module is specifically used for: acquiring the first voice signal carrying the preset wake-up word sent by the target sound source; extracting the preset wake-up word from the first voice signal; extracting voiceprint features of the first voice signal; judging whether the preset wake-up word matches the voiceprint feature of the first voice signal according to the pre-stored correspondence between the preset wake-up word and the voiceprint feature of the target sound source; If they match, it is determined that the voice signal from the target sound source needs to be collected. 14. The smart speaker according to claim 13, wherein the smart speaker further comprises: A receiving module, configured to receive a second voice signal carrying the preset wake-up word input by the user's voice corresponding to the target sound source; An extraction module, configured to extract the preset wake-up word from the second voice signal; The extraction module is further configured to extract the voiceprint feature of the second speech signal as the voiceprint feature of the target sound source; The establishment module is used to establish and store the corresponding relationship between the preset wake-up word and the voiceprint feature of the target sound source. 15. A smart speaker, including a plurality of microphones for sending and receiving signals; it is characterized in that, the smart speaker also includes: one or more processors; memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method according to any one of claims 1-7. 16. A computer-readable medium, on which a computer program is stored, wherein, when the program is executed by a processor, the method according to any one of claims 1-7 is implemented.