WO2008001766A1 - Music game device - Google Patents

Abstract

Provided is a music game device capable of effectively realizing a function for reflecting a music feature and a user voice in the game contents. The music game device includes: a line input terminal (4) for acquiring a music reproduction signal; a microphone (6) for acquiring a user voice and converting it into a voice signal; a signal processing unit (10) for outputting at least one of a differential value and an integration value concerning a particular frequency component of each signal; judging units (30, 31) for judging a feature of the music to be reproduced by the music reproduction signal according to a signal outputted from the signal processing unit in response to the music reproduction signal and classifying the voice signal according to the signal outputted from the signal processing unit in response to the voice signal; and a game control unit (32) for reflecting the music feature judged by the judging units (30, 31) and the voice signal classification result in the game contents.

Description

 Specification

 Music game machine

 Technical field

 [0001] The present invention relates to a music game machine having a function of capturing a user's voice and reflecting it in game content.

 Background art

 [0002] As a game machine of this type, there is a game machine that has a dedicated processing unit that analyzes an audio signal input from a microphone and reflects the analysis result of the processing unit in the form of a character (for example, (See Patent Document 1).

 Patent Document 1: Japanese Unexamined Patent Publication No. 2001-29649

 Disclosure of the invention

 Problems to be solved by the invention

[0003] A music game machine that takes in a music playback signal output from a music playback device, determines the characteristics of the music to be played back by the music playback signal, for example, Giannole, and reflects the determination result in the game content is considered. Has been. In such a music game machine, when a dedicated processing unit for analyzing an audio signal is added in addition to the components for determining the characteristics of the music, a functionally redundant portion is added and wasted. May occur.

 [0004] Accordingly, an object of the present invention is to provide a music game machine capable of realizing the function of reflecting each of the music features and the user's voice in the game content without waste.

 Means for solving the problem

[0005] The music game machine of the present invention is output from a signal processing unit that outputs a signal corresponding to at least one of a differential value and an integral value related to a specific frequency component of an input signal, and a music reproduction device. A music playback signal input unit that takes in a music playback signal and inputs it to the signal processing unit, a voice input unit that takes a user's voice and converts it into a voice signal, and inputs the voice signal to the signal processing unit, and In response to music playback signal input Based on the signal output from the signal processing unit, the characteristics of the music to be played back by the music playback signal are determined, and the signal output from the signal processing unit in response to the input of the audio signal A discriminating unit that classifies the audio signal based on the discriminating unit, and a game control unit that reflects each of the characteristics of the music discriminated by the discriminating unit and the classification result of the audio signal in the game content. Solve the problem.

 In the music game machine of the present invention, the signal processing unit receives at least one of a differential value and an integral value related to a specific frequency component with respect to an input of a music playback signal or a user-powered audio signal from a music playback device. A signal corresponding to either one is output. Based on the output signal, the feature of the music is discriminated by the discriminating unit, or the audio signal is classified, and each of the feature and the classification result is reflected in the game content by the game control unit. Features such as music genre and tempo correlate with the differential value or integral value of the specific frequency component of the music playback signal, so if the correlation is used, the signal corresponding to the input of the music playback signal It is possible to discriminate music features based on signals output from the processing unit. On the other hand, audio signals are also common to music playback signals in that they are sound waveforms. For this reason, the differential value or integral value of a specific frequency component of the speech signal shows changes that correlate with various factors such as the utterance content, that is, the tone, strength, inflection, and word difference. . Focusing on these changes, even if advanced speech analysis such as speech recognition is not desired, speech signals can be classified to some extent according to the utterance content. As a result, at least the signal processing unit is shared, and in some cases, part of the processing function of the determination unit is also shared, so that more game functions can be provided to the user at a low cost.

[0007] In one embodiment of the music game machine of the present invention, the signal processing unit outputs a signal corresponding to a differential value and an integral value of the specific frequency component, and the determination unit includes the signal processing unit. The signal output from the signal is taken in a predetermined sampling unit time, and it is determined whether or not the value of each signal exceeds a predetermined level within the sampling unit time, and it is determined that a value exceeding the predetermined level is detected. A data generation unit for generating analysis data obtained by counting the number of times every predetermined sampling period and for each signal; and determining characteristics of the music based on the total value described in the analysis data; and A data analysis unit for classifying the signals. According to this aspect, the degree of variation of the differential value and the integral value output by the signal processing unit is reflected in the total value of the analysis data created by the determination unit. These variations correlate with characteristics such as music genre and tempo, or changes in the content of the user's utterances, so the aggregated values can be used to distinguish music characteristics or classify user voices. can do. Since both the differential value and the integral value related to the specific frequency component are output from the signal processing unit, the aggregate value related to the differential value increases when the signal strength frequently changes, and the overall signal level is increased. When the fluctuation is large, the total value for the integral value becomes large. By using such a tendency of change, it is possible to discriminate music characteristics more delicately. Further, by utilizing the function of generating the analysis data of the data generation unit for the classification of the audio signal, the additional burden necessary for providing the function of classifying the audio signal can be further reduced.

In one form of the music game machine of the present invention, the signal processing unit includes a signal corresponding to at least one of a differential value and an integral value of the low frequency component of the input signal, and a differential value of the high frequency component of the input signal. And a signal corresponding to at least one of the integral values, and the determination unit captures each of the signals output from the signal processing unit by a predetermined sampling unit time, and outputs the sampling unit. It is determined whether or not the value of each signal exceeds a predetermined level in time, and the analysis data obtained by collecting the number of times that the value exceeding the predetermined level is detected for each predetermined sampling period and for each signal is obtained. A data generation unit for generating the data, and a data analysis unit for discriminating the characteristics of the music and classifying the audio signal based on the total value described in the analysis data It may have. According to this form, the feature of music can be discriminated by using the total value described in the analysis data, or the user's voice can be classified. In addition, a signal corresponding to at least one of the differential value and the integral value of the low frequency component and a signal corresponding to at least one of the differential value and the integral value of the high frequency component of the input signal are respectively transmitted from the signal processing unit. Since it is output, the tendency of each change of the low frequency component and the high frequency component can be grasped from the total value, and the music feature can be discriminated more finely. Then, the analysis data of the data generator By utilizing the function to generate the audio signal for the classification of the audio signal, the additional burden required to provide the function for classifying the audio signal can be further reduced.

 [0009] In a form in which the aggregate value described in the analysis data is used, the data analysis unit calculates an average value of each of the aggregate values, and evaluates variation of the obtained average value to thereby calculate the music. The speech signal may be classified based on the magnitude relation of the average value. The genre of music can be discriminated using the correlation existing between the genre of music and the variation of the differential value or integral value of the specific frequency component of the music playback signal. As for the classification of audio signals, the average value of the aggregated values is used in the same way as music feature discrimination, so it is possible to share the calculation function of the aggregated values. It is possible to further reduce the additional burden required to establish

 [0010] In the form in which the audio signal is classified based on the magnitude relationship of the average values, the data analysis unit corresponds to any of a plurality of patterns (patterns A to E) in which the magnitude relationship of the average values is predetermined. The audio signal may be classified according to whether or not. According to this embodiment, it is possible to easily classify the magnitude relation of the obtained average values by setting a plurality of patterns in advance.

 [0011] In a form in which the audio signals are classified using a plurality of patterns in which the magnitude relationship of the average values is determined in advance, the data analysis unit calculates the difference between the two average values as one of the average values. The branch reference value obtained by division may be further referenced to determine which of the plurality of patterns corresponds to the magnitude relationship of the average value. By setting the branch reference value, the number of patterns can be increased to further classify the audio playback signal.

[0012] In one embodiment of the music game machine of the present invention, the determination unit determines a character to appear on the game screen based on the classification result of the audio signal, and the game control unit is connected to the data analysis unit. The sound signal classification result may be reflected in the game content by causing the determined character to appear on the game screen. According to this embodiment, it is possible to provide the user with a game function of displaying a character corresponding to the content of the user's utterance on the screen. The invention's effect

 [0013] As described above, according to the music game machine of the present invention, both the audio playback signal output from the music playback device and the audio signal output from the audio input unit are used as the signal processing unit. Since the signal corresponding to at least one of the differential value and the integral value related to the specific frequency component of those signals is input and output, the characteristic of music and the differential value or integral of the specific frequency component of the music playback signal are output. While distinguishing music features using correlation with values, it is possible to appropriately classify audio signals by focusing on changes in differential values or integral values corresponding to the user's utterance content. As a result, at least the signal processing unit is shared, and in some cases, part of the processing function of the determination unit is also shared, so that more play functions can be provided to the user at a low cost.

 Brief Description of Drawings

 FIG. 1 is a diagram showing a state in which a portable game machine of the present invention is placed between a portable music player and an earphone.

 [Fig. 2] Block diagram of the part related to the discrimination of music in the control system of the game machine of FIG.

FIG. 3 is a functional block diagram of the control unit of FIG.

 FIG. 4 is a diagram showing a relationship between a music playback signal and a sampling period.

 FIG. 5 is a diagram showing an example of the relationship between the waveform of the integrated value and the sampling unit time within the sampling period.

 [Fig. 6] A diagram showing the contents of analysis data.

 FIG. 7 is a diagram showing the contents of calculation result identification data.

 FIG. 8 is a diagram showing the contents of discrimination reference data.

 FIG. 9 is a diagram showing the contents of history data.

 [Fig. 10] Flow chart showing the analysis data generation process executed by the control unit.

 FIG. 11 is a flowchart showing a data analysis process executed by the control unit.

 FIG. 12 is a diagram showing a pattern for classifying audio signals.

FIG. 13 is a view showing the contents of retained character data. FIG. 14 is a flowchart showing a data analysis processing routine executed by the control unit in the microphone mode.

 BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 shows a portable game machine according to one embodiment of the present invention. A game machine 1 as a music game machine is used in combination with a portable music player 100, and includes a case 2 and an LCD 3 as a display device attached to the front surface of the case 2. . The casing 2 is provided with a line input terminal 4, a phone terminal 5, and a microphone 6. The line input terminal 4 is connected to the line output terminal 101 of the portable music player 100 via the relay cable 102. The phone terminal 5 is connected to the earphone 103. That is, the game machine 1 of this embodiment is used by being interposed between the portable music player 100 and the audio output device to be combined therewith. The audio output device combined with the portable music player 100 is not limited to the earphone 103. That is, the portable music player 100 is not limited as long as it can output a music reproduction signal for audio conversion to various audio output devices such as a speaker and a headphone. The details are unquestionable. Furthermore, the music player is not limited to a portable type, and includes various devices that output music such as home audio, television, personal computer, and commercially available portable electronic games. The microphone 6 takes in the user's voice and converts it into a voice signal. In addition to the user's voice, the microphone 6 can be used to input ambient sounds, voice or music from electronic devices, etc.

[0016] The game machine 1 functions as a relay that passes the music playback signal output from the line input terminal 4 of the portable music player 100 to the earphone 103, and music output from the portable music player 100. It functions as a game machine that analyzes the playback signal and provides the user with a game according to the analysis result. FIG. 2 is a block diagram showing a configuration of a part related to the function of taking in and analyzing music reproduction signals and audio signals in the control system provided in the game machine 1. The game machine 1 includes a line input terminal 4 as a music playback signal input section and a bypass path R1 for passing an analog music playback signal and a voice signal from the microphone 6 as a voice input section to the phone terminal 5 and the line input terminal 4 And the music playback signal and audio signal captured from the microphone 6 via the branch path R2. It has a signal processing unit 10 for processing and a control unit 11 for capturing the output signal of the signal processing unit 10. Paths Rl and R2 are all represented by one in the force diagram composed of three lines of the right channel, the left channel, and the first channel. The line input terminal 4 and the microphone 6 are connected by a switching switch (not shown) that switches between them, and can switch between a music reproduction signal and an audio signal as an input signal. For example, when the line input terminal 4 is inserted from the relay cable 102, the switching switch connects the line input terminal 4 and the branch paths Rl and R2. If there is no plug, the switch connects microphone 6 and branch paths Rl and R2.

 [0017] The signal processing unit 10 includes a pair of low-pass filters (LPF) 12A and 12B that allow only a low-frequency component of a music reproduction signal captured from the line input terminal 4 or an audio signal converted by the microphone 6 to pass through. A high-pass filter (HPF) 13A that passes only the high-frequency component of the music playback signal or audio signal, a differentiation circuit 14 that differentiates the output signal of LPF12A, an integration circuit 15 that integrates the output signal of LPF12B, and an HPF13A A fine circuit 16 for differentiating the output signal, and A / D converters 17A to 17C for converting the output signals of the circuits 14 to 16 into digital signals and outputting them to the control unit 11 are provided. The frequency range that LPF 12A and 12B pass is set to 1000 Hz or less, for example, and the frequency range that HPF 13A passes is set to 1000 Hz or more, for example. The set value of the frequency range is not limited to these examples. For example, the frequency range that LPF12A and 12B pass can be set to 500Hz or less, and the frequency range that HPF13A can pass is set to 1000Hz or more. Further, the frequency ranges that LPF 12A and 12B pass may be set equal or different from each other. If the two pass frequency ranges coincide with each other, a single LPF may be provided instead of the LPFs 12A and 12B, and the output signal may be branched to the differentiation circuit 14 and the integration circuit 15.

The control unit 11 is configured as a computer unit that combines a microprocessing unit (MPU) and peripheral devices necessary for the operation of the MPU, for example, storage devices such as RAM and ROM. The control unit 11 is connected to the LCD 3 described above as a control target, and is connected to an input device 20 for giving game instructions and the like, and a speaker unit (SP) 21 for generating sound, sound effects, and the like. The Further, the phone terminal 5 is also connected to the connection path to the speaker unit 21. The control unit 11 provides various game functions to the user by executing processing such as displaying a game screen on the LCD 3. As a function attached to the game, the control unit 11 has a function of analyzing the output signal of the signal processing unit 10 when the music reproduction signal is input from the line input terminal 4 and determining the music signal. . FIG. 3 is a functional block diagram of the control unit 11. When the MPU (not shown) of the control unit 11 reads a predetermined control program from the storage device 25 and executes it, the control unit 11 has a data generation unit 30 as a determination unit and a logical unit inside the control unit 11. A data analysis unit 31 and a game control unit 32 are generated. The data generation unit 30 processes the output signal of the signal processing unit 10 to generate analysis data D1, and stores this in the storage device 25. The data analysis unit 31 reads the analysis data D1, refers to the calculation result identification data D2, determines the genre of music by a predetermined method, and updates the history data D4 according to the determination result. The discrimination reference data D3 recorded in the storage device 25 is referred to for the Giannole discrimination. The game control unit 32 executes the game according to a predetermined game program (not shown) while referring to the history data D4.

 [0020] Further, when the audio signal from the microphone 6 is processed by the signal processing unit 10, the control unit 11 analyzes the output signal and classifies the audio, and based on the classification result, the game screen of the LCD 3 It also has a function to make a character appear above. In order to realize this function, the control unit 11 generates the analysis data D1 by processing the output signal of the signal processing unit 10 in the data generation unit 30 in the same procedure as the Giannole discrimination function, and stores it in the storage device 25. To remember. The data analysis unit 31 reads the analysis data D1, classifies the voice signals by a predetermined method, and updates the retained character data D5 according to the classification result. The game control unit 32 executes the game with reference to the retained character data D5 in addition to the history data D4 described above.

Next, processing relating to genre discrimination by the game machine 1 will be described with reference to FIGS. FIG. 4 is an example of a waveform of a music playback signal input from the line input terminal 4 to the signal processing unit 10. In the signal processing unit 10, low frequency components of the music playback signal are extracted by the LPFs 12A and 12B, and high frequency components are extracted by the HPF 13A. The differential value of the extracted low-frequency component is output from the differentiation circuit 14, and the integrated value of the low-frequency component is output from the integration circuit 15. The differential value of the high frequency component is output from the differentiation circuit 16. The output differential value and integral value are converted into digital signals by the A / D converters 17A to 17C and input to the data generation unit 30 of the control unit 11. The data generation unit 30 includes a sampling period Tm shown in FIG. 4 and an example of an output waveform of the integration circuit 15 as a reference time for processing the differential value and the integration value output from the signal processing unit 10. Two types of time lengths are set for the sampling unit time Tn shown in the figure. The sampling period Tm is an integer multiple of the sampling unit time Tn. As an example, the sampling period Tm is set to 5 seconds, and the sampling unit time Tn is set to 20 milliseconds.

 [0022] In the data generation unit 30 of the control unit 11, the differential value and the integral value are taken in by the sampling unit time Tn, and whether the differential value and the integral value each exceed the predetermined level within the sampling unit time Tn. Power. Then, the analysis data D1 is generated by collecting the number of times it is determined that a value exceeding a predetermined level is detected for each sampling period Tm and for each differential value and integral value. For example, if the integrated value of the low-frequency component in one sampling period Tm set in FIG. 4 fluctuates as shown in FIG. 5, the data generation unit 30 sets the integrated value to a predetermined value within each sampling unit time Tn. It is monitored whether or not the threshold value TH is exceeded. If the integral value exceeds the threshold value TH, it is determined that the integral value has exceeded a predetermined level. However, the number is counted as 1 if it exceeds once even if the integral value exceeds the threshold TH within one sampling unit time Tn. This determination process is repeated every sampling unit time Tn within the sampling period Tm, and the number of times that it has been determined that the predetermined level has been exceeded when the sampling period Tm has elapsed is counted. If the sampling period Tm is 5 seconds and the sampling unit time Tn is 20 milliseconds, the minimum number of times in one period Tm is 0 and the maximum value is 250.

[0023] The data generation unit 30 of the control unit 11 individually executes the above-described processing for each of the differential value and the integral value, and sequentially collects the measured number of times for each sampling period Tm. The analysis data D1 is generated as shown in Fig. 6. In analysis data D1 in Fig. 6, channel chO corresponds to the output from differentiation circuit 14, channel chl corresponds to the output from integration circuit 15, and channel ch2 corresponds to the output from differentiation circuit 16. Sample numbers s mpl to smpN correspond to cycle numbers from the start point of the music playback signal. here It is assumed that the music playback signal is equivalent to N cycles in total. And the total value sumOX of channel chO in sample number smpX (where X is 1 to N) is that the differential value of the low frequency component exceeds the predetermined level TH in the Xth sampling period TmX from the start of processing. Indicates the number of times determined. For example, sumOl corresponds to the number of times that the differential value of the low frequency component was determined to exceed the threshold TH in the first sampling period. The same applies to the other channels chl to ch2.

The data analysis unit 31 of the control unit 11 calculates the average value M0 to M2 for each channel, that is, for each differential value and integral value, the low frequency component and the high frequency component for the total value described in the analysis data D1. Calculate the coefficient of variation CVO and CV2 of the aggregated value described in the analysis data D1 for the derivative value (see Fig. 6). Here, the coefficient of variation is a percentage of the value obtained by dividing the standard deviation of the aggregated value by the average value, and is a type of value used in statistical processing as a measure for evaluating the magnitude of data variation. . For example, if the standard deviation of the aggregate value is SD and the average value is M, the coefficient of variation is CV = (SD / M) x 100. Further, the data analysis unit 31 refers to the calculation result identification data D2, and obtains identification values dMO, dMl, dM2, dCVO, dC V2 corresponding to the average values MO, Ml, M2, and coefficient of variation CVO, CV2, respectively. To do. The calculation result identification data D2 is a set of tables in which the average values MO, Ml, M2 and the coefficients of variation CVO, CV2 are associated with the identification values dMO, dMl, dM2, dCVO, dCV2. The identification value is a value that represents each category when the range that the average value or coefficient of variation can take is divided by a predetermined number of steps. For example, in the average value MO table, as shown in FIG. 7, the range of values 0 to 250 that the average value MO can take is set to 4 by three threshold values a, b, and c (where a <b <c). Each category is represented by an identification value 0-3. Then, the data analysis unit 31 refers to the table in FIG. 7 to obtain any one value from 0 to 3 corresponding to the average value MO as the identification value dMO. A similar table is prepared for the average values Ml and M2 and the coefficient of variation CV0 and CV2, though not shown. The data analysis unit 31 acquires identification values dMl, dM2, dCV0, and dCV2 corresponding to the acquired average values Ml and M2 and variation coefficients CV0 and CV2 in the same procedure. The identification values dMl and dM2 corresponding to the average values M 1 and M2 are three levels 0 to 2, respectively, and the identification values dC V0 and dCV2 corresponding to the coefficient of variation CV0 and C V2 are two levels 0 and 1, respectively. Segmented It is. However, the number of division stages of each identification value may be changed as appropriate.

 [0025] The data analysis unit 31 arranges the acquired identification values dM0 to dM2 and dCV0 and dCV2 in the order of the identification values dM0, dMl, dM2, dCV0, and dCV2, thereby characterizing the waveform of the music playback signal. A numerical value is acquired as a judgment value. For example, if the identification value dMO is 1, dMl is 0, dM2 force, dCVO force 0, dCV2 force, 10001 can be obtained as a semi-IJ straightness. In this example, 144 judgment values are obtained. Note that the arrangement order of the identification values dM0 to dM2 and dCV0 and dCV2 for obtaining the determination value is not limited to this embodiment, and may be arbitrarily specified.

 [0026] Further, the data analysis unit 31 determines the genre of music to be reproduced by the music reproduction signal based on the above-described five-digit determination value. In this genre discrimination, discrimination reference data D3 is referred to. As illustrated in FIG. 8, in the determination reference data D3, music genres A to X and the above-described 144 determination values are described in association with each other. The genre here is a concept used to distinguish the contents of music such as classic, rock, ballad, and JAZZ. The data analysis unit 31 compares the determination reference data D3 with the genre that matches the obtained determination value as the genre corresponding to the music playback signal. For example, when the determination value is 10001, genre A is determined as the genre corresponding to the music playback signal as illustrated in FIG. In addition, after the determination of data is made, the data analysis unit 31 updates the history data D4 according to the determination result. For example, as shown in FIG. 9, the history data D4 is described in association with the genres A to X and the respective input counts Na to Nx, and the data analysis unit 31 adds 1 to the determined number of gains. The history data D4 is updated by calculation. Alternatively, a specific number may be provided in advance for the number of times the history data D2 is described, and the determined genre may be described in the history data D2 each time a determination result is output. In this case, if the number of descriptions exceeds a specific number, the oldest description will be deleted, and the history data D2 will be updated so that the latest discrimination result is described.

Next, with reference to FIG. 10 and FIG. 11, a procedure of processing executed by the control unit 11 in order to perform the above-described genre discrimination will be described. FIG. 10 shows an analysis data generation process that is executed by the control unit 11 (data generation unit 30) to generate the analysis data D1. Show. This routine is executed on condition that, for example, the differential value and the integral value are output from the signal processing unit 10 in a state where the user instructs genre discrimination from the input device 20 (see FIG. 2). The differential value and the integral value output from the signal processing unit 10 are sequentially accumulated in the internal buffer of the control unit 11 and subjected to processing by this routine.

 [0028] In the analysis data generation processing routine, in the first step S1, the control unit 11 sets the variable n that specifies the number of the channel ch to be processed to the initial value 0, and then continues to step S2. Then, capture the output signal (differential value or integral value) equivalent to the sampling unit time of channel chn. In the next step S3, the control unit 11 determines whether or not the fetched output signal exceeds a predetermined level. If it exceeds the predetermined level, the control unit 11 proceeds to step S4, adds 1 to the internal counter for channel chn, and then proceeds to step S5. On the other hand, if the predetermined level is exceeded in step S3, the control unit 11 skips step S4 and proceeds to step S5.

 [0029] In step S5, the control unit 11 determines whether or not 2 is set in the variable n.

 If not 2, add 1 to variable n in step S6 and return to step S2. On the other hand, if the variable n is 2 in step S5, the control unit 11 proceeds to step S7. By repeating the processing of steps S2 and S6, the output of each of the three channel ch0 ch2, that is, the differentiation circuit 14 for the low frequency component, the integration circuit 15 and the differentiation circuit 16 for the high frequency component is made to be equivalent to the sampling unit time. Inspected across the board.

[0030] In step S7, the control unit 11 determines whether or not the processing for the sampling period Tm has been completed. For example, if the number of times that step S5 is affirmed is equal to the value obtained by dividing the sampling period Tm by the sampling unit time Tn, it may be determined that the processing for the sampling period Tm has ended. If the determination in step S7 is negative, the control unit 11 returns to step S1 and proceeds to process the signal of the next sampling unit time stored in the internal buffer. On the other hand, if step S7 is affirmed, the control unit 11 proceeds to step S8, and the value recorded in the internal counter is the sum of the sample numbers smpX corresponding to the current sampling period sumOX sumlX sum2X (see Fig. 6) To the analysis data D1 of the storage device 25. If analysis data D1 does not exist yet Create new analysis data D1 and record the total value in association with the first sample number smp1.

 In the subsequent step S9, the control unit 11 resets the value of the internal counter to the initial value 0, and further determines whether or not the generation processing of the analysis data D1 is completed in the next step S10. For example, it can be determined that the processing is completed when a so-called silent state near the output force SO of all channels ch0 to ch2 continues for a predetermined time or more. If the processing is not completed, the control unit 11 returns to step S1. If it is determined that the processing has ended, the control unit 11 ends the analysis data generation processing routine. With the above processing, analysis data D1 as shown in FIG. 6 is generated.

 FIG. 11 shows a data analysis processing routine executed by the control unit 11 (data analysis unit 31) in order to discriminate music gains from the analysis data D1. This routine is executed after the end of the analysis data generation processing routine of FIG. In the data analysis processing routine, the control unit 11 sets the variable n for designating the number of the channel ch subject to data processing to the initial value 0 in the first step S21, and stores it in the subsequent step S22. The total value of the channel number ch n corresponding to the variable n is read from the analysis data D1 recorded in the device 25, and the average value thereof and the coefficient of variation with respect to the total value of the differential values of the low frequency component and the high frequency component are calculated. In the next step S23, the control unit 11 determines whether or not 2 is set to the variable n. If not 2, 1 is added to the variable n in step S24 and the process returns to step S22. On the other hand, if the variable n is 2 in step S23, the control unit 11 proceeds to step S25. Processing power of steps S22 to S24 S Repeatedly calculates the average values M0 to M2 of the three channels ch0 to ch2 and the coefficient of variation CV0 and CV2 for the sum of the differential values of the low and high frequency components. Is done.

In step S25, the control unit 11 refers to the calculation result identification data D4, and the identification values dM0, dMl, dM2, dCV0, respectively corresponding to the obtained average values M0 to M2 and the variation coefficients CV0 and CV2, Get dCV2. In the next step S26, the control unit 11 refers to the discrimination reference data D3 of the storage device 25 and refers to the genre corresponding to the 5-digit judgment value in which the identification values dM0, dMl, dM2, dC V0, dCV2 are arranged in order. Select the music genre by selecting. Furthermore, the control unit 11 determines in the next step S27. The history data D2 is updated so that 1 is added to the number of times of the separated genre, and then the data analysis processing routine is finished.

[0034] Next, with reference to Figs. 12 to 14, a process related to the microphone mode for analyzing the audio signal from the microphone 6 and classifying the audio will be described. Note that the microphone mode is activated when the user performs a predetermined operation for instructing the microphone mode to the input device 20 with the input from the microphone 6 to the signal processing unit 10 enabled, for example. This is a special mode.

 In the microphone mode, the audio signal from the microphone 6 is input to the signal processing unit 10 in the same manner as the music playback signal from the line input terminal 4. In the signal processing unit 10, the differential value and integral value for the low frequency component of the audio signal and the differential value for the high frequency component of the audio signal are converted into a digital signal and output. Based on these output signals, the data generation unit 30 of the control unit 11 generates analysis data D1. As in the case of the music playback signal described above, the analysis data D1 in this case is recorded for each of the channels ch0 to ch2 as the total value of the number of times that the differential value and the integral value exceed the threshold TH within the sampling period Tm. It is. That is, the processing content of the data generation unit 30 when an audio signal is input is the same as that when a music playback signal is input.

On the other hand, the data analysis unit 31 has different processing contents when processing the analysis data D1 corresponding to the music playback signal and when processing the analysis data D1 corresponding to the audio signal. That is, when processing the analysis data D1 corresponding to the audio signal, the data analysis unit 31 calculates the average values M0 to M2 for each channel for the total value described in the analysis data D1 (see FIG. 6). Then, the data analysis unit 31 compares the obtained average values M0 to M2 in magnitude relationship, so that the magnitude relationship between the average values M0, Ml, and M2 is any of the patterns A to E shown in FIG. The audio signal is classified according to whether it is applicable. For example, if average value M0> average value Ml> average value M2, the audio signal is classified into pattern B. The patterns A to E characterize the magnitude relationship between the average values M0 to M2 by vectors connecting the average values M0 to M2. However, when the magnitude relationship between the average values M0 to M2 is a predetermined relationship, a branch reference value F = (MO-Ml) ZM0 is calculated which indicates the ratio of the difference between the average value M0 and the average value Ml. , This branch reference value F is used for pattern branching Is done. In the example of FIG. 12, the branch reference value F is used for the branch between the pattern A and the pattern C and the branch between the pattern D and the pattern E.

 [0037] When the audio signal is classified into any of patterns A to E, data analysis unit 31 determines a character to appear on the game screen according to the classified pattern. For example, one or more different characters are associated with each of the patterns A to E in advance, and the character is determined by the data analysis unit 31 selecting a character corresponding to the classified pattern according to the correspondence. Is done. Here, the character is a character, character or symbol, etc. arranged in addition to humans, animals and plants, imaginary animals and plants, and displayed in LCD3, and the display content is not particularly limited. Les. The determined character is displayed on the game screen of the LCD 3 at an appropriate timing according to the game control by the game control unit 32. Further, the data analysis unit 31 updates the retained character data D5 according to the determined contents of the character. Once appearing, the character is counted as a character that can be held by the user and recorded in the held character data D5. For example, as shown in FIG. 13, the retained character data D5 is configured as a table in which the characters A to X that can be displayed on the game screen and the characters retained by the user are associated with each other. Updates the stored character data D5 by adding 1 to the number of characters that appear according to the pattern classification result. Depending on the content of the game, it may be set so that a plurality of the same characters can be held, or it may be set so that only one character can be held. If the number of characters that can be held is limited, the number of held character data D5 is not updated after the set number of characters.

 Next, with reference to FIG. 14, the procedure of processing executed by the control unit 11 to classify sound in the microphone mode will be described. Prior to the processing of FIG. 14, the control unit 11 (data generation unit 30) executes the analysis data generation processing routine shown in FIG. 10 in order to generate analysis data D1 corresponding to the audio signal. Is the same as the process for the music playback signal, so the explanation is omitted. When the analysis data D1 corresponding to the audio signal is generated, the control unit 11 subsequently executes a data analysis processing routine shown in FIG.

In the data analysis processing routine of FIG. 14, the control unit 11 performs the first step S3. In step 1, the variable n that specifies the number of the channel ch to be processed is set to the initial value 0, and in step S32, the analysis data D1 stored in the storage device 25 corresponds to the variable n. Read the total value of channel number chn and calculate the average value. In the next step S33, the control unit 11 determines whether or not 2 is set to the variable n. If not 2, 1 is added to the variable n in step S34 and the process returns to step S32. On the other hand, if the variable n is 2 in step S33, the control unit 11 proceeds to step S35. By repeating the processing of steps S32 to S34, the average value of each of the three channels ch0 to ch2 is calculated.

 [0040] In step S35, the control unit 11 compares the magnitude relationship of the average values M0 to M2. In the next step S36, the control unit 11 determines whether or not the average values M0 to M2 are in a predetermined magnitude relationship, that is, the relationship between the patterns A and C or the relationships between the patterns D and E described above. If there is a predetermined magnitude relationship, the control unit 11 calculates the branch reference value F in step S37, and then proceeds to step S38. If a negative determination is made in step S36, the control unit 11 skips step S37 and proceeds to step S38. In step S38, the control unit 11 compares the average values M0 to M2, and if the branch reference value F is calculated, the audio signal is converted into patterns A to E based on the branch reference value F (Fig. 12)). In subsequent step S39, the control unit 11 determines a character corresponding to the classified pattern as a character to appear on the game screen. Thereafter, the control unit 11 proceeds to step S40, updates the retained character data D5 so that 1 is added to the determined number of characters, and thereafter ends the data analysis processing routine.

[0041] The character determined as described above can be used in various ways in the game executed by the game control unit 32. For example, when the game control unit 32 executes a game for breeding a character, the game control unit 32 may cause the character described in the retained character data D5 to appear as a guest character different from the breeding character. Les. The guest character can be positioned as a character that the user calls as desired and displays it on the LCD 3, for example. A game machine that collects guest characters using voice input by making different guest characters appear according to the classification result of the audio signal. Functions can be provided to the user.

 [0042] In the game machine 1 of this embodiment, since the number of distinctions for each Giannole is recorded in the history data D4, the music the user listens to via the game machine 1 by referring to the history data D4 It is possible to analyze the frequency of each game, the user's preference, and reflect the genre discrimination result in the game content executed by the game control unit 32. For example, when the game control unit 32 executes a game for nurturing a character, the characteristics of the character, personality, etc. are changed according to the distribution of the number of discriminations for each genre described in the history data D4. The operation can be performed by the game control unit 32. Then, the user's voice can be classified using the functions of the signal processing unit 10, the data generation unit 30, and the data analysis unit 31 for use in discriminating music, and the classification result can be reflected in the game contents. . As a result, it is possible to realize the function of reflecting each of the music features and the user's voice in the game content without waste, so it is possible to increase more at a low cost without the need to add a dedicated processing circuit for voice discrimination. It is possible to provide a play function.

 [0043] The present invention can be implemented in various forms without being limited to the above forms.

 For example, the signal processing unit is not limited to the one provided with the differentiation circuit and integration circuit for the low frequency component of the audio reproduction signal or the audio signal and the differentiation circuit for the high frequency component, and at least one of the differentiation circuit and the integration circuit for the specific frequency component. Various modifications are possible as long as either one is provided. The differential value and integral value of the music playback signal reflects the tendency of music changes. By outputting at least one of the differential value and integral value from the signal processing unit, the characteristics of the music can be determined from the tendency of the change. And classify speech. In the above embodiment, the differential value is output for each of the low frequency component and the high frequency component. However, the integrated value of both frequency components may be output from the signal processing unit, or the differential value for a single frequency range may be output. The signal processing unit may output the value and the integral value. Furthermore, only one of the differential value or the integral value with respect to a single frequency range may be output from the signal processing unit.

In the above embodiment, the genre is discriminated as the music feature, but the present invention is not limited to this, and the discriminating unit may discriminate the music tempo and other various features. example For example, the tempo of music has a relatively strong correlation with the integrated value of the low frequency component of the music playback signal. That is, the low frequency component of the music playback signal is strongly influenced by the regular base rhythm included in the music, and the irregular waveform included in the low frequency component is dulled in the integrated waveform. Regular waveforms due to rhythm appear more clearly. For this reason, the peak interval of the integral waveform of the low frequency component, that is, the interval between the maximum values has a correlation with the tempo of music. The determination unit can determine the tempo of the music by obtaining the peak interval of the integral waveform for each predetermined sampling period. If the peak interval is irregular, the feature discriminating unit may calculate a statistical value such as an average value, median value, or mode value as the peak interval. In the integrated value waveform of the low frequency component of the audio signal, the irregular waveform included in the audio signal is dulled and a regular waveform showing regular fluctuation of the audio appears more clearly. Voices can be classified using such regular changes.

 [0045] In the above embodiment, the number of times that the integral value and the differential value of the low frequency component and the differential value of the high frequency component exceed a predetermined level within the sampling unit time is totaled, and the average value and fluctuation of the total value are counted. Although the coefficient is calculated to determine the degree of variation in the music reproduction signal waveform, the contents of the processing executed by the determination unit to determine the characteristics of the music are not limited to the example using only the average value and the variation coefficient. For example, the music genre etc. may be discriminated by further referring to various statistical values such as standard deviation, variance, and total of the total values. Any number of statistical values may be used. In addition, although only the variation coefficient of the differential value of the low frequency component and the high frequency component was calculated, the present invention is not limited to this, and the genre etc. is determined by calculating the coefficient of variation of each aggregate value of all differential values and integral values. You may use it. Although the 5-digit judgment value characterizing the waveform of the music playback signal was used for data analysis, the number of digits may be set according to various statistical values to be calculated. For example, if the average value and coefficient of variation of the integral and derivative values of the low-frequency component and the derivative value of the high-frequency component are calculated, the judgment value that characterizes the waveform of the music playback signal is 6 digits. You can also use various statistical values for voice classification.

[0046] In this embodiment, the game control unit reflects the music gain on the change of the character form or the like, or the character determined according to the audio signal classification result is displayed on the game screen. Although the example of making it appear was shown, the relationship between the music feature discrimination result or the audio signal classification result and the game content is not limited to these examples. For example, various changes such as changes in game difficulty, progress speed, and bonus occurrence probability may be generated in association with music characteristics or audio signal classification results.

 The signal processing unit may be configured as a hardware device combining circuit elements such as IC and LSI, or may be configured as a logical device combining MPU and software. Each of the data generation unit and the data analysis unit may also be configured as a hardware device. The signal input unit is not limited to the line input terminal. For example, a device that receives a playback signal transmitted using radio such as FM radio waves and converts it into a music playback signal may be used as the signal input unit.

Claims

The scope of the claims  [1] A signal processing unit that outputs a signal corresponding to at least one of a differential value and an integral value related to a specific frequency component of the input signal;  A music playback signal input unit for receiving a music playback signal output from a music playback device and inputting the music playback signal to the signal processing unit;  A voice input unit that captures a user's voice, converts the voice into a voice signal, and inputs the voice signal to the signal processing unit;  Based on the signal output from the signal processing unit corresponding to the input of the music playback signal, the characteristics of the music to be played back by the music playback signal are determined, and the input of the audio signal is supported. A discriminating unit for classifying the audio signal based on a signal output from the signal processing unit;  A game control unit that reflects the characteristics of the music determined by the determination unit and the classification result of the audio signal in the game content;  Music game machine equipped with.  [2] The signal processing unit outputs signals corresponding to the differential value and the integral value of the specific frequency component,  The determination unit captures a signal output from the signal processing unit for each predetermined sampling unit time, determines whether the value of each signal exceeds a predetermined level within the sampling unit time, and determines the predetermined A data generation unit that generates analysis data in which the number of times it is determined that a value exceeding the level is detected for each predetermined sampling period and for each signal; and the music based on the total value described in the analysis data The music game machine according to claim 1, further comprising: a data analysis unit that discriminates the characteristics of the voice signal and classifies the audio signal.  [3] The signal processing unit includes at least one of a signal corresponding to at least one of a differential value and an integral value of a low frequency component of the input signal, and a differential value and an integral value of a high frequency component of the input signal. Output signals corresponding to either of these, The discriminating unit takes in each of the signals output from the signal processing unit for a predetermined sampling unit time, and the value of each signal falls within a predetermined unit time within the sampling unit time. A data generation unit that generates analysis data that determines whether or not a value exceeding the predetermined level has been detected, and aggregates the number of times that the value exceeding the predetermined level is detected for each predetermined sampling period and for each signal; and The music game machine according to claim 1, further comprising: a data analysis unit that discriminates characteristics of the music based on a total value described in data and classifies the audio signal.  [4] The data analysis unit calculates an average value of each of the aggregated values, determines a genre as a feature of the music by evaluating variation of the obtained average value, and calculates the average value of the average value. 4. The music game machine according to claim 2 or 3, wherein the audio signals are classified based on a magnitude relationship.  5. The data analysis unit according to claim 4, wherein the data analysis unit classifies the audio signal according to which of a plurality of predetermined patterns the magnitude relationship of the average value corresponds to. Music game machine.  [6] The data analysis unit further refers to a branch reference value obtained by dividing the difference between two average values by one of the average values, and the magnitude relationship of the average values is determined by the plurality of patterns. 6. The music game machine according to claim 5, wherein the music game machine determines which one of them is applicable.  [7] The determination unit determines a character to appear on the game screen based on the classification result of the audio signal, and the game control unit determines the character determined by the data analysis unit as the game screen. The music game machine according to any one of claims 1 to 6, wherein a classification result of the audio signal is reflected in the game content by appearing in the game.