JP2003208161A - Robot apparatus and control method thereof - Google Patents

Abstract

(57) [Summary] [Problem] Since a specific sound is associated with various actions, it is possible for a user to predict in advance what kind of sound will be output from the movement of the robot, There was a risk of getting bored eventually. A robot generates music in accordance with the operation and its own internal state while causing the robot device to perform the operation, and outputs the music to the outside.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot apparatus and its control method, and is suitable for application to, for example, a bipedal robot.

[0002]

2. Description of the Related Art In recent years, various robots such as a humanoid type robot that imitates the appearance of a human and a four-legged walking pet robot that imitates the appearance of a dog or cat raised as a pet in a general household. Has been developed and commercialized.

As these robots, ones that can act autonomously in response to a command from the user and the surrounding situation, and one that is previously associated with the operation in response to the user's operation. There are various things, such as one that is designed to express one movement.

[0004]

By the way, a conventional robot has a predefined audio file inside and reproduces the audio file by associating it with a specific operation or the like. When it appears, the sound is output to the outside together with this.

However, according to such a conventional configuration, since a specific sound is associated with various operations as described above, the user is informed in advance what kind of sound is output from the movement of the robot at that time. There was a risk that I could get tired of it.

As one method for solving such a problem, a method of preparing a plurality of types of audio files corresponding to respective operations and reproducing them at random can be considered.

However, according to this method, there is a problem that a memory having a large capacity is required as a memory for storing an audio file and the program becomes complicated. Moreover, in such a method, there is no change in that some kinds of sounds are repeatedly output for one motion in any case, so that there is a problem that the user's boredom of the robot cannot be effectively prevented.

The present invention has been made in consideration of the above points, and proposes a robot apparatus and a control method thereof capable of significantly improving the entertainment property while preventing the configuration from becoming complicated and upsizing. Is.

[0009]

In order to solve such a problem, according to the present invention, the robot apparatus is provided with a music generating means for generating music according to the motion expressed by the motion demonstrating section and its own internal state. I did it. As a result, a memory for preliminarily storing data of music to be emitted when the robot device expresses an action is unnecessary, and various kinds of music are generated and emitted according to the internal state of the robot device. Therefore, it is possible to effectively prevent the user from getting tired of the robot device.

Further, according to the present invention, in the control method of the robot apparatus, while causing the robot apparatus to perform an action, music corresponding to the action and the internal state of the self is generated and the music is output to the outside. did. As a result, a memory for preliminarily storing data of music to be emitted when the robot device expresses an action is unnecessary, and various kinds of music are generated and emitted according to the internal state of the robot device. Therefore, it is possible to effectively prevent the user from getting tired of the robot device.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described in detail below with reference to the drawings.

(1) Configuration of Robot According to the Present Embodiment (1-1 Configuration of Robot) In FIGS. 1 and 2, reference numeral 1 denotes a bipedal robot as a whole according to the present embodiment, and a body unit 2, a head unit 3 is disposed on the upper part of the body portion 2, arm units 4A and 4B having the same configuration are disposed on the upper left and right portions of the body portion unit 2, and on the lower left and right sides of the body portion unit 2, respectively. The leg units 5A and 5B having the same configuration are attached to respective predetermined positions.

In the trunk unit 2, a frame 10 forming an upper trunk and a waist base 11 forming a lower trunk are connected to each other via a waist joint mechanism 12, and the lower waist of the trunk is connected. By driving the actuators A ₁ and A ₂ of the lumbar joint mechanism 12 fixed to the base 11, the upper trunk is independently rotated around the orthogonal roll axis 13 and pitch axis 14 shown in FIG. Is made possible.

The head unit 3 is attached to a central portion of the upper surface of a shoulder base 15 fixed to the upper end of the frame 10 via a neck joint mechanism 16, and the neck joint mechanism 16 is attached.
By driving each of the actuators A ₃ and A ₄ of the above, the actuator can be independently rotated around the orthogonal pitch axis 17 and yaw axis 18 shown in FIG.

Further, the arm units 4A and 4B are attached to the left and right of the shoulder base 15 via the shoulder joint mechanism 19, and drive the actuators A ₅ and A ₆ of the corresponding shoulder joint mechanism 19, respectively. Fig. 3
Can be independently rotated around the orthogonal pitch axis 20 and roll axis 21 shown in FIG.

In this case, each arm unit 4A, 4B is
An actuator A ₈ forming a forearm is connected to an output shaft of an actuator A ₇ forming an upper arm via an elbow joint mechanism 22, and a hand 23 is attached to the tip of the forearm. .

[0017] The arm units 4A, in 4B, FIG. 3 the forearm by driving the actuator _{A 7}
The actuator A is rotated around the yaw axis 24 shown in FIG.
By driving ₈ , the forearm can be rotated around the pitch axis 25 shown in FIG.

On the other hand, in each leg unit 5A, 5B, each leg unit 5A, 5B is attached to the waist base 11 under the torso via the hip joint mechanism 26, and each actuator of the corresponding hip joint mechanism 26 is A ₉ ~ A
_By driving each of them _11, the yaw shaft 27, the roll shaft 28, and the pitch shaft 29 shown in FIG. 3 which are orthogonal to each other can be independently rotated.

In this case, in each of the leg units 5A and 5B, a frame 32 forming the lower leg is connected to the lower end of a frame 30 forming the thigh via a knee joint mechanism 31, and the frame 32 is connected. Ankle joint mechanism 3 at the lower end of
It is configured by connecting the foot portion 34 via the terminal 3.

Thus, in each of the leg units 5A and 5B, the actuator A forming the knee joint mechanism 31 is formed.
By driving the _12, can rotate the lower leg around the pitch axis 35 shown in FIG. 3, and by driving the actuator A _{1 3,} A ₁₄ of the ankle joint mechanism 33 respectively, the foot portion 34 Can be independently rotated around the orthogonal pitch axis 36 and roll axis 37 shown in FIG.

On the other hand, on the back side of the waist base 11 forming the lower trunk of the body unit 2, as shown in FIG. 4, a main control unit 40 for controlling the operation of the robot 1 as a whole.
A control unit 42 including a peripheral circuit 41 such as a power supply circuit and a communication circuit, a battery 45 (FIG. 5), and the like is housed in a box.

The control unit 42 is provided in each constituent unit (body unit 2, head unit 3, each arm unit 4A, 4B and each leg unit 5A, 5B) and each sub-control is arranged. The sub-control units 43A to 43D are connected to the sub-control units 43A to 43D by supplying a necessary power supply voltage to the sub-control units 43A to 43D.
~ 43D can be communicated with.

Further, each of the sub-control units 43A to 43D has each of the actuators A ₁ to
Is connected to the A _14, it is configured so as to be able to drive each actuator A ₁ to A ₁₄ in the configuration unit to the state specified on the basis of the various control commands given from the main control unit 40.

Further, as shown in FIG. 5, the head unit 3 has a CCD functioning as "eyes" of the robot 1.
(Charge Coupled Device) A camera 50, an external sensor portion 53 including a microphone 51 and a touch sensor 52 that function as “ears”, a speaker 54 that functions as a “mouth”, and the like are arranged at predetermined positions, and a control unit is provided. Inside 42, an internal sensor unit 57 including a battery sensor 55, an acceleration sensor 56, and the like is provided.

The CCD camera 5 of the external sensor section 53
0 captures the surrounding situation and sends the obtained image signal S1A to the main control unit, while the microphone 51 causes the microphone 51 to "walk" or "prone" given as a voice input.
Alternatively, various command voices such as "follow the ball" are collected, and the voice signal S1B thus obtained is collected by the main control unit 4
It is designed to be sent to 0.

As is apparent from FIGS. 1 and 2, the touch sensor 52 is provided on the upper portion of the head unit 3, and the pressure applied by a physical action such as "stroking" or "striking" from the user. Is detected and the detection result is sent to the main controller 40 as a pressure detection signal S1C.

Further, the battery sensor 55 of the internal sensor unit 57 detects the remaining energy level of the battery 45 in a predetermined cycle, and sends the detection result to the main control unit 40 as a remaining battery level detection signal S2A, while the acceleration sensor 56 operates. Three
The acceleration in the axial direction (x-axis, y-axis, and z-axis) is detected in a predetermined cycle, and the detection result is sent to the main control unit 40 as an acceleration detection signal S2B.

The main control section 40 includes an external sensor section 53.
Image signal S1A, audio signal S1B, pressure detection signal S1C, etc. (hereinafter collectively referred to as external sensor signal S1) supplied from CCD camera 50, microphone 51, touch sensor 52, etc., and internal sensor unit 57. Based on the battery remaining amount detection signal S2A, the acceleration detection signal S2B, etc. (hereinafter collectively referred to as the internal sensor signal S2) supplied from the battery sensor 55, the acceleration sensor, etc. Determine the situation, commands from the user, and whether or not the user is working.

Then, the main control section 40 continues based on this judgment result, the control program stored in advance in the internal memory 40A, and various control parameters stored in the external memory 58 loaded at that time. The action is determined, and the control command based on the determination result is sent to the corresponding sub control unit 43A to 43D. As a result, based on this control command, the corresponding actuators A _{1 to} A ₁₄ are driven under the control of the sub-control units 43A to 43D, thus swinging the head unit 3 vertically and horizontally, and arm movement. The robot 1 expresses actions such as raising the sub-units 4A and 4B and walking.

Further, at this time, the main controller 40 outputs a voice based on the voice signal S3 to the outside by giving a predetermined voice signal S3 to the speaker 54 as necessary, or functions as an apparent "eye". A drive signal is output to an LED provided at a predetermined position of the head unit 3 to blink it.

In this way, the robot 1 can autonomously act on the basis of the surrounding and internal conditions, the instruction from the user and the presence or absence of the action.

(1-2) Process of Main Control Unit Here, the process of the main control unit 40 relating to the action generation of the robot 1 will be described.

As shown in FIG. 6, when the processing contents of the main control unit 40 relating to the action generation of the robot 1 are functionally classified, the state recognition unit 60 that recognizes the external and internal states.
And an emotion / instinct model 61 that determines the state of emotion and instinct based on the recognition result of the state recognition unit 60, and the recognition result of the state recognition unit 60 and the state of emotion / instinction determined in the emotion / instinct model 61. An action determining unit 62 that determines the next action based on the action determination unit 6 and an action generating unit 6 that actually causes the robot 1 to exert an action based on the determination result of the action determining unit 62.
It can be divided into three. Hereinafter, these state recognition units 6
0, the emotion / instinct model unit 61, the action determination unit 62, and the action generation unit 63 will be described.

(1-2-1) The processing state recognizing unit 60 of the state recognizing unit 60 specifies a specific one based on the external sensor signal S1 given from the external sensor unit 53 and the internal sensor signal S2 given from the internal sensor unit 57. The state is recognized, and the recognition result is used as the state recognition information D1 for the emotion / instinct model 61
And the action determination unit 62 is notified.

Specifically, the state recognition section 60 is provided with an image signal S1A provided from the CCD camera 50 of the external sensor section 53.
Is constantly monitored, and when a "red round object" or "object located in the traveling direction" is detected in the image based on the image signal S1A, "there is a ball", "there is an obstacle"
And the emotion / instinct model 61 and the action determining unit 62 are notified of the recognition result.

The state recognition unit 60 also includes a microphone 51.
Constantly monitors the audio signal S1B given by
When various voices such as “walk”, “prone”, and “follow the ball” are recognized by a voice recognition method such as the (Hidden Markov Model) method, the voice is recognized as an emotion / instinct model 61.
And the action determination unit 62 is notified.

Further, the state recognizing section 60 is provided with the touch sensor 5
2. The pressure detection signal S1C given from 2 is constantly monitored, and when a pressure equal to or higher than a predetermined threshold value and for a short time (for example, less than 2 seconds) is detected based on the pressure detection signal S1C, it is "struck" When a pressure lower than a predetermined threshold value and for a long time (for example, 2 seconds or more) is detected, it is recognized as “stroked (praised)”, and the recognition result is the emotion / instinct model unit 61 and the action. Notify the decision unit 62.

Further, the state recognizing section 60 includes an internal sensor section 5
Acceleration detection signal S given from the acceleration sensor 56 of No. 7
2B is constantly monitored, and when a fall or the like is detected based on the acceleration detection signal S2B, this is notified to the emotion / instinct model unit 61 and the action determination unit 62.

(1-2-2) Processing of Emotion / Instinct Model Unit 61 The emotion / instinct model unit 61 performs "joy", "sadness",
With respect to a total of six emotions of “surprise”, “fear”, “disgust”, and “anger”, a parameter indicating the strength of the emotion is held for each emotion. Then, the emotion / instinct model unit 61, based on the specific recognition result such as “praised” or “scold”, which gives the parameter value of each emotion as the state recognition information D1 from the state recognition unit 60, respectively. Change sequentially.

Specifically, the emotion / instinct model 61 is
Recognition result and action determination unit obtained based on the recognition information D1
Robot 1 notified from 62 as action determination information D2
Of the output behavior just before the action on the emotion
(Preset) and the suppression and control from other emotions
Performance based on the degree of
The emotional change amount ΔE [t] calculated by the formula,
The parameter value of the current emotion is used as E [t], recognition result, etc.
K is the coefficient that represents the rate of changing the emotion _eage
Then, in a predetermined cycle,

[0041]

[Equation 1]

Using, the parameter value E [t + 1] of the emotion in the next cycle is calculated.

Then, the emotion / instinct model unit 61 updates the parameter value of the emotion by replacing the calculation result with the current parameter value E [t] of the emotion.
Note that it is predetermined which emotion parameter value is updated for each recognition result or each output action. For example, when a recognition result such as “struck” is given, the emotion parameter of “anger” is given. When the value rises and a recognition result such as “struck” is given, the emotional parameter value of “joy” rises.

Similarly to this, the emotion / instinct model unit 6
1 holds five independent desires of "love desire", "search desire", "exercise desire", "charge desire" and "sleep desire", each of which has a parameter representing the strength of the desire. is doing. Then, the emotion / instinct model unit 61 sets the parameter values of each of these desires in the state recognition unit 60.
It is changed sequentially based on the recognition result from, the elapsed time, and so on.

More specifically, the emotion / instinct model unit 61 calculates a predetermined arithmetic expression for “love desire”, “search desire”, and “exercise desire” based on the output behavior of the robot 1, elapsed time, recognition result, and the like. The change amount ΔI of the desire calculated by
[K], the current parameter value of the desire is I [k], and a coefficient representing the sensitivity of the desire is k _i

[0046]

[Equation 2]

The parameter value I [k + 1] of the desire in the next cycle is calculated using and the calculation result is replaced with the current parameter value I [k] of the desire to update the parameter value of the desire. To do. In this case, which desire parameter value to change with respect to the output behavior, the recognition result, and the like is predetermined, and for example, the behavior determination unit 6
When there is a notification from 1 that some action has been taken, the parameter value of "motivation for exercise" is lowered.

Further, the emotion / instinct model part 61 is "appetite".
Regarding the battery remaining amount _BL based on the battery remaining amount detection signal S2A given via the state recognition unit 60.
With the following formula

[0049]

[Equation 3]

The parameter value I [k +
1] is calculated, and the calculation result is replaced with the current parameter value I [k] of appetite to update the parameter value of the “appetite”.

In the present embodiment, the parameter values of each emotion and each desire are regulated so as to vary within the range of 0 to 100, and the coefficients k _e and k _i are regulated.
The value of is also set individually for each emotion and each desire.

By the way, as described above, it is set in advance which parameter value of emotion or desire is increased or decreased for each recognition result or each output action. It is changed sequentially according to the result of one's own action.

In practice, the emotion / instinct model unit 61 receives the state recognition information D1 indicating that the robot 1 is looking at an item such as a color or a ball, which the robot 1 likes, from the state recognition unit 60. The parameter value of the emotion of "joy" is updated so as to be increased, but if the state recognition information D1 "struck" is given at this time,
Along with the lower the coefficient k _e of "joy", coefficient of "anger" k
Update these coefficient values to raise _e .

As a result, the robot 1 has a favorite color and
Many times, when I was watching a movie
Once done, the "joy" clerk for the color, item, etc.
A few k _eBecomes gradually smaller, and the coefficient of "anger" k_e
Gradually becomes larger, and eventually you will see the color and items etc.
And the "anger" parameter value was updated to be larger
As a result, the robot 1 seems to be angry as described later.
You will begin to express actions.

Further, the emotion / instinct model unit 61 is given the state recognition information D1 of "ball (large)" which means "there is a big ball" from the state recognition unit 60, and then the action determination unit 62. The action determination information D2 of "kick" meaning "kick the ball" is given, and after that, the state recognition unit 60 further indicates a state of "ball (small)" meaning "small ball exists". When the recognition information D1 is given, it is determined that "the ball has been successfully kicked", and the coefficient of "joy" for the item "ball" related to the phenomenon, k _e
And the coefficient k _i of the "motility desire" are respectively increased, these coefficient values are updated.

[0056] When this result, the robot 1 is also successful many times the action to kick the ball, and the coefficient k _i is increased gradually each of the "joy" coefficient k _e and the "movement greed" of for the "ball", before long ball Sees "joy" and "motivation"
By updating the parameter value so that it becomes large, the robot 1 can express a pleasing action when looking at the ball, or improve the ball play, as described later.

On the contrary, the emotion / instinct model unit 61
Is given the state recognition information D1 of "ball (large)" from the state recognition unit 60, the action determination information D2 of "kick" from the action determination unit 62, and then the state recognition information from the state recognition unit 60. When the state recognition information D1 “Ball (Large)” is given, it is determined that “kicking the ball has failed” and the coefficient of “joy” for the “ball” is k.
These coefficient values are updated so that _e and the coefficient k _i of "motility desire" are respectively lowered.

[0058] As a result, the robot 1 also fail many times the action to kick the ball, and the coefficient k _i decreases gradually each of the "joy" coefficient k _e and the "movement greed" of for the "ball", before long ball Sees "joy" and "motivation"
By updating the parameter value so that it becomes smaller, the robot 1 becomes unresponsive or does not play much when the ball is seen, as described later.

In this way, in the robot 1, the character can be changed according to the user's action, the result of his / her own action, and the like.

(1-2-3) Processing of the action deciding unit 62 On the other hand, the action deciding unit 62 receives the state recognition information D1 from the state recognizing unit 60, and a certain period of time has passed since the current action was taken. When any of the emotion or instinct parameter values in the emotion / instinct model unit 61 exceeds a threshold value, the control program stored in the internal memory 40A and the control parameter stored in the external memory 58 are changed. Based on this, the next action is determined.

Specifically, the action determining section 62 determines the next action.
As a method of determining the state, as shown in FIG.
ODE₀~ NODE_nAs one node NO
DE ₀From which node NODE₀~ NODE_nTransition to
Whether to move the node NODE₀~ NODE_nAt
Completed or each node NODE₀~ NODE_nConnect between
Ark ARC₀~ ARC_nRespectively set for
Transition probability P₀~ P_nProbabilistically determined based on
An algorithm called an automaton is used.

[0062] respectively associated with this case and the connection relationship between the nodes NODE ₀ ~NODE _n in the probability automaton, each arc _ARC 0 transitions for ～ARC _n probability _P 1 to P _n and each arc _ARC 0 ~ARC _n The motion described above is stored in the external memory 58 as a control parameter (behavior model).

Then, the action determining section 62, for example, recognizes the state.
When the state recognition information D1 is given from the section 60,
Node (NODE₀), A certain amount of time has passed since
When one of the emotions / instinct model section 61
When the parameter value of movement or instinct exceeds a threshold value,
Node of next transition destination in such probability automaton
(NODE₀~ NODE_n) Each arc ARC₀~ AR
C_nTransition probability P for ₀~ P_nProbabilistically based on
The node (NODE₀~ NODE
_n) And the original node (NODE₀) Is connected to the arc
(ARC₀~ ARC _n), The action associated with
Generate action as action decision information D2 as action to be shown
Notify the department.

(1-2-4) Processing of Action Generating Unit 63 When the action determining unit 62 receives the action determining information D2, the action generating unit 63 causes the robot to develop an action based on the action determining information D2. Control command C for
The OM is output to the corresponding sub control units 43A to 43D.

Specifically, the action generator 63 "walks",
The actuators A _{1 to} A ₁₄ (FIGS. 1 and 2) output shafts at which timings and how much the actuators A _{1 to} A ₁₄ (FIGS. 1 and 2) correspond to the motions such as “sitting” and “dancing” to cause the robot 1 to express the motions. Actuators A _{1 to} A _{1 4} for each operation, such as whether to rotate and drive only the angle
The external memory 58 has a file (hereinafter, referred to as a motion file) defining the time-series control contents of the above.

Then, the action generation section 63 has the action determination section 62.
Each time the action decision information D2 is given from the control command C, the corresponding motion file is sequentially reproduced and the control command C based on the control parameter stored in the motion file is reproduced.
The OM is generated and the control command COM is sent to the corresponding sub control units 43A to 43D.

As a result, based on this control command COM, the corresponding sub-control units 43A to 43D sequentially drive the corresponding actuators A _{1 to} A ₁₄ , and thus the robot 1 develops such an operation.

Further, the action generator 63 uses the WAVE of various sounds.
The external memory 58 has a plurality of audio files, which are files, and a plurality of LED drive files in which the drive data of the LEDs functioning as the above-mentioned "eyes" are stored in the external memory 58. The audio file and / or the LED drive file associated with the motion file are simultaneously played back to the robot 1 to cause the robot 1 to output a voice from the speaker 54 (FIG. 5) or to drive the LED to blink. .

In this way, the main control section 40 is configured to allow the robot 1 to act autonomously in accordance with external and internal conditions, instructions from the user and presence / absence of an action.

(2) Process of Main Control Unit 40 in Dance Mode Next, the process of the main control unit 40 in the dance mode of the robot 1 will be described.

In the case of the robot 1, in addition to the autonomous mode in which the robot 1 behaves autonomously in accordance with the external and internal conditions, the command from the user and the presence / absence of an action from the user as described above, the operation mode is previously set to the programmer or the user. It has a dance mode for performing the dance created by. Then, in this dance mode, the robot 1 self-generates music according to the emotional state at that time, and emits it to the outside.

In this case, the control relating to the action generation of the robot 1 in the dance mode is mainly performed by the action generation unit 63 (FIG. 6) described above.

In practice, the action generation section 63 has the state recognition section 6
When the state recognition result that the voice input “Dance” is input from 0 is notified via the action determination unit 62, the operation mode is switched to the dance mode.

Then, the action generator 63 causes the robot 1 to perform a dance by playing back a previously registered motion file for dance, and the emotion / instinct model 61 (FIG. 6) shows the parameter values of each emotion. Based on the above, music is generated at a rhythm according to the speed of movement of the robot 1, and this is emitted to the outside together with dance.

Here, the processing contents of the action generating section 63 relating to the action generation of the robot 1 in the dance mode are functionally classified into a motion file reproducing section 70 and a music generating section 71, as shown in FIG. Can be divided. The processing of the motion file reproducing unit 70 and the music generating unit 71 in the dance mode will be described below.

(2-1) Motion file reproducing unit 70
When the process motion file reproducing unit 70 shifts to the dance mode, the posture of the robot 1 first transitions to a posture for "dancing" (for example, a "standing" posture), if necessary. Further, the motion file reproducing unit 70 thereafter reads the motion file 72 (FIG. 9) that defines the motion of the robot 1 for causing the robot 1 to “dance” from the external memory 58, reproduces it, and calls it “dance”. A control command COM for causing the robot 1 to express an action is generated and the corresponding sub-control unit 4
3A to 43D.

Specifically, in the case of this robot 1, FIG.
As shown in FIG. 4, the motion file 72 of such “dance” has a data structure in which motion numbers (motion 1, motion 2, ..., Motion m) of some primitive motions are arranged in time series. . The primitive motion is a detailed basic motion element such as "raising hands", "waving hips", "jumping", etc., which constitutes a complicated series of motions (for example, dance) of the robot 1. Say.

In addition, the external memory 58 stores a primitive motion file database 73,
The primitive motion file database 73
In order to express various primitive motions to the robot 1, which actuators A _{1 to} A ₁₄ (see FIG.
2) and the control parameters that define the time-series control contents of the actuators A _{1 to} A ₁₄ for each primitive motion, such as at what timing and at what angle the output shaft should be rotationally driven. , These are called primitive motion control parameters)
PM _{1 to} PM _n are the motion numbers (motion 1, motion 2, ..., Mo ₎ of the primitive motion.
(tion m) and stored.

Then, in the dance mode, the motion file reproducing section 70 (FIG. 8), based on the “dance” motion file 72 reproduced from the external memory 58, each of the motion files 72 is specified in time series. The primitive motion control parameters PM _{1 to} PM _n corresponding to the primitive motion are read from the primitive motion file database 73 in that order, and the control commands COM are sequentially generated based on the read primitive motion control parameters PM _{1 to} PM _n. , The control command CO
The M is sequentially sent to the corresponding sub control units 43A to 43D (FIG. 5).

As a result, the corresponding actuators A _{1 to} A ₁₄ are sequentially driven by the sub-control units 43A to 43D based on the control command COM,
"Raise your hand", "shake your hips", "jump"
The primitive motion is continuously expressed by the robot 1, and by the expression of the series of primitive motions, the robot 1 operates as if "dancing".

In the case of this embodiment, the motion file 72 of “dance” has a control parameter indicating how fast each primitive motion forming the motion file 72 is reproduced. Is the tempo (Fig. 9
Tempo1 and tempo2) are stored in association with the primitive motion.

This tempo (tempo1, tempo2, ... temp
On) is usually given by the following formula in music theory: Tempo = (60 (seconds) × numerator of time signature × total number of measures) ÷ total playing time (seconds).

[0083]

[Equation 4]

The term "fourth note" means that the quarter note is played 100 times in one minute.

Then, the motion file reproducing unit 70 (FIG. 8) generates a control command while taking into consideration the tempo data when generating a control command based on each primitive motion file read from the primitive motion file database 73. .

As a result, the corresponding actuators A _{1 to} A ₁₄ are sequentially driven by the sub-control units 43A to 43D based on the control command COM,
Each primitive motion is expressed by the robot at a speed according to the tempo.

(2-2) Processing of Music Generating Unit 71 On the other hand, the music generating unit 71, in parallel with the processing of the motion file reproducing unit 70, performs each process in the emotion / instinct model unit 61 (FIG. 6) at that time. Music is generated at a rhythm corresponding to the speed of movement of the robot 1 based on the emotional parameter value.

In practice, the music generation section 71, as shown in FIG. 10, rhythms based on the parameter values of each emotion in the emotion / instinct model section 61 at that time and the tempo of each primitive motion forming the dance. Music is generated by determining each of the three elements of the pattern, chord progression, and melody scale.

As a means for determining the rhythm pattern, chord progression and melody scale, a music generation rule database 80 as shown in FIG. 11 is stored in the external memory 58.

In this case, the music generation rule database 80 comprises an emotion value table 81, a rhythm pattern data file 82, a chord progression table data file 83 and a melody scale data file 84, of which the rhythm pattern data file 82. Has
Rock, Rhythm & Blues (R &
B), Dance style (Dance), Classic style (Classica)
A plurality of types of template data for various rhythm patterns such as l) are stored for each rhythm pattern.

In the emotion value table 81, "joy",
For each emotion of “sadness”, “surprise”, “fear”, “disgust”, and “anger”, the parameter value of the emotion in the emotion / instinct model unit 61 is the highest compared to the parameter values of other emotions. Several rhythm pattern options to be selected when the rhythm pattern is large are defined in association with the probability that the rhythm pattern should be selected.

For example, in the example shown in FIG. 11, when the parameter value of “joy” in the emotion / instinct model portion 61 is the largest, the rock tone (Rock), dance tone (Dance), ... It is stipulated that one rhythm pattern should be selected from the rhythm patterns with a probability of 50%, 20%, and so on.
s) ”has the largest parameter value, blues (Blues), rhythm and blues (R &
It is stipulated that one rhythm pattern should be selected from the rhythm patterns B), ... With a probability of 40%, 30% ,.

Thus, the music generation section 71 refers to the parameter value of each emotion in the emotion / instinct model section 61 immediately after the transition to the dance mode, detects the emotion having the largest parameter value among these, and Based on the detection result, one rhythm pattern is stochastically selected using the emotion value table 81 in the music generation rule database 80, and a plurality of rhythm patterns are selected as the rhythm pattern in the rhythm pattern data file 82. The rhythm pattern is determined by selecting one from the templates prepared for each type.

Further, the music generating section 71 obtains the tempo data of the primitive motion which the robot 1 is expressing at that time from the motion file reproducing section 70,
The tempo of the determined rhythm pattern is determined based on the tempo data.

On the other hand, the chord progression table data file 83 of the music generation rule database 80 stores chord progression table data defining various chord progressions. In this case, as the chord progression, each key (C, D, E,
For each code), there are multiple types of major scale and minor scale. For example, FIG. 11 exemplifies a part of chord progression with "C" as a key.

When determining the chord progression, the music generation section 71 first determines the key of the song (first chord) and the emotion of the maximum parameter value in the emotion / instinct model section 61 immediately after the transition to the dance mode. Based on the type, the major scale or the minor scale is determined. Specifically, the music generator 71
When the parameter value of the positive emotion such as "joy" in the emotion / instinct model unit 61 is the largest, the major scale is selected, and "sadness", "surprise", "fear", "disgust", and "anger" are selected. If the negative emotional parameter value is the largest, select the minor scale.

Further, the music generating section 71 thereafter determines, as described above, among the chord progressions defined in the chord progression table stored in the chord progression table data file 83, based on the above-mentioned determination result. One chord progression is selected from the ones corresponding to the keys and corresponding to the major scale or the minor scale determined as described above, and this is determined as the chord progression of the music to be output at that time.

Next, regarding the melody, there are three determinants for determining one note: length (note length), pitch (note scale), and velocity (note strength).

Therefore, as shown in FIG. 12, the music generating section 71 defines a melody space with three axes of length, pitch and velocity, and selects one point in this melody space as follows. A melody is generated by deciding one sound and repeating this in sequence. In FIG. 12, each axis is represented by 8 bits, and the length, pitch, and velocity are 0 to 2 respectively.
This is an example of a case where a value of 56 is taken.

Specifically, when determining one sound, the music generating section 71 first randomly determines the length so that the sound is always output in the “beat”. In other words, the timing for producing the sound is predetermined, and the length is determined by selecting where to produce the sound after that, including the timing of the “beat”. FIG. 12 shows a case where “100” is determined as the length value.

In order to reflect the emotion of the robot 1, the pitch is the pitch of the immediately preceding sound P (t), the pitch of the next sound P (t + 1), and the emotion / instinct model part immediately after the transition to the dance mode. From the parameter values of each emotion in 61, the maximum parameter value is Pr, and

[0102]

[Equation 5]

The pitch P (t + 1) of the next note is randomly determined within the range given by. Accordingly, when the emotion of the robot 1 is intense (when the maximum parameter value in the emotion / instinct model 61 is large), the pitch fluctuation range becomes wide, and conversely when the emotion is calm (the maximum parameter value is When it is small), the pitch fluctuation range becomes narrow.

As for the velocity, the velocity of the immediately preceding sound is V (t) and the velocity of the next sound is V (t +) in order to reflect the emotion of the robot 1 similarly to the pitch.
1) The value of the maximum parameter value among the parameter values of each emotion in the emotion / instinct model unit 61 immediately after the transition to the dance mode is Vr, and

[0105]

[Equation 6]

The velocity V (t + 1) of the next note is randomly determined within the range given by.

However, even if the length, pitch, and velocity are randomly determined as described above, it is necessary to consider the balance with the chord at the time of producing the note so as not to cause dissonance.

Therefore, in the case of this embodiment, the melody scale file 8 in the music generation rule database 80 is
In FIG. 4, for each chord, a plurality of sound candidates that are not dissonant with the chord (hereinafter referred to as candidate sounds) are stored in association with each other. In FIG. 11, “A1”, “E2”, “D1”, and the like are associated as candidate sounds when the chord at that time is “A”, and “No” as the candidate sound when the chord is “C”. It indicates that "C1", "E2", "G1", etc. are associated.

Then, the music generation section 71 considers the chord at that time when determining the next note, and selects from among the candidate notes associated with the chord stored in the melody scale file 84. Try to select one candidate note that satisfies all the length, pitch, and velocity conditions.

In practice, as a method of selecting such candidate sounds, the music generator 71 uses the melody scale file 84.
Among the candidate tones associated with the chord at that time stored in the above, the length determined as described above is L, and the pitch and velocity of the immediately preceding tone are P (t) and V, respectively.
As (t), the following equation in the melody space shown in FIG.

[0111]

[Equation 7]

And the following equation

[0113]

[Equation 8]

And the length-pitch-velocity space region 90 (FIG. 1) that satisfies all of the above equations (6) and (7).
2) spatial coordinates (L, P (t + 1), V (t
The next sound is determined by randomly selecting one candidate sound from among several candidate sounds having +1)). Note that r _{1 in the} equation (7) and r ₂ in the equation (8) are
Each of the following sounds is an empirical value preset in consideration of the average human sensitivity so as to match the chord at that time. Thus, the music generation section 71 repeats such processing to sequentially determine the next sound, thereby generating a melody.

Then, the music generation section 71 sequentially determines the rhythm pattern, chord progression and melody based on the parameter values of each emotion in the emotion / instinct model section 61 (FIG. 6) at that time as described above. While generating music, the voice data S3 of the music thus obtained
Are sequentially output to the speaker 54 (FIG. 5) to output the music to the outside.

In this way, in the robot 1, music is generated at a rhythm according to the speed of movement of the robot 1 based on the parameter value of each emotion in the emotion / instinct model unit 61 (FIG. 6) at that time. Then, this is output to the outside.

(3) Operations and Effects of this Embodiment In the above configuration, the robot 1 is
Depending on the emotions at that time, dance while playing bright music when you are happy, and dark music when you are otherwise sad or angry. Further, the music played at this time becomes a violent melody when the emotion is strong and a gentle melody when the emotion is weak, depending on the strength of the emotion at that time.

Therefore, the robot 1 cannot predict in advance what kind of music will be output from the movement of the robot, and the emotion of the robot 1 at that time is reflected in such music. Therefore, it is possible to effectively prevent the user from getting tired of the robot.

Further, since the robot 1 self-generates such music each time, it does not require a memory or the like for storing the music, and accordingly, when such a function is mounted on the robot, It is possible to prevent the configuration of the robot 1 from becoming complicated and upsizing.

According to the above configuration, when the user dances, music is generated based on the emotion of the robot 1 at that time and the sound is emitted, so that the user is effectively prevented from getting bored with the robot 1. Thus, it is possible to realize a robot capable of remarkably improving the entertainment property while preventing the configuration from becoming complicated and upsizing.

(4) Other Embodiments In the above-mentioned embodiments, the case where the present invention is applied to the bipedal walking robot 1 configured as shown in FIGS. 1 and 2 will be described. However, the present invention is not limited to this, and can be widely applied to, for example, a four-legged walking type pet robot and various other robot devices.

Further, in the above-mentioned embodiment, the case where the music is generated based on the emotion of the robot 1 has been described, but the present invention is not limited to this. For example, a part of the emotion (“joy”, "Sadness,""Surprise,""Fear,"
The music is generated based on "dislike" and "anger" or a part of instinct ("love desire", "search desire", "exercise desire", "charge desire" and "sleep desire". )) Or music may be generated according to various other internal states such as the internal temperature and combinations thereof.

Further, in the above-described embodiment, the case where the music generation process is performed only in the dance mode has been described. However, the present invention is not limited to this, and the music generated in the normal mode such as the autonomous mode is also described. You may make it perform a generation process.

Further, in the above-described embodiment, the motion expressing unit for expressing the motion is composed of the body unit 2, head unit 3, arm units 4A and 4B, and leg units 5A and 5B of the robot 1. Although the case of the whole body has been described, the present invention is not limited to this, and various other configurations can be widely applied depending on the form of the robot device to which the present invention is applied.

Further, in the above-described embodiment, the main control unit 40 which controls the operation of the entire robot 1 is applied as the music generating means for generating music according to the motion expressed by the robot and its own emotions. However, the present invention is not limited to this, and a music generation unit having such a function may be provided separately from the main control unit 40.

[0126]

As described above, according to the present invention, the robot apparatus is provided with the music generating means for generating the music according to the motion expressed by the motion expressing section and the internal state of the robot. Since a memory for preliminarily storing data of music to be emitted when an action is expressed is unnecessary, and various music is generated and emitted according to the internal state of the robot device, the user It is possible to effectively prevent the robot from getting tired of the robot device, and thus it is possible to realize a robot device that can significantly improve the entertainment property while preventing the configuration from becoming complicated and upsizing.

Further, according to the present invention, in the control method of the robot apparatus, while causing the robot apparatus to perform an operation,
By generating music according to the operation and the internal state of the self and outputting the music to the outside, the data of the music to be emitted when the robot device performs the operation is stored in advance. A memory for storing is unnecessary, and various kinds of music are generated and emitted according to the internal state of the robot apparatus, which can effectively prevent the user from getting tired of the robot apparatus, and thus It is possible to realize a control method of a robot apparatus that can significantly improve entertainment while preventing the configuration from becoming complicated and upsizing.

[Brief description of drawings]

FIG. 1 is a perspective view showing an external configuration of a robot according to this embodiment.

FIG. 2 is a perspective view showing an external configuration of a robot.

FIG. 3 is a schematic diagram used to describe an external configuration of a robot.

FIG. 4 is a block diagram for explaining the internal configuration of the robot.

FIG. 5 is a block diagram for explaining the internal configuration of the robot.

FIG. 6 is a block diagram for explaining processing contents of a main control unit.

FIG. 7 is a conceptual diagram for explaining a probability automaton.

FIG. 8 is a block diagram for explaining processing contents of an action generation unit in a dance mode.

FIG. 9 is a schematic diagram for explaining the processing content of a motion file reproducing unit.

FIG. 10 is a conceptual diagram for explaining the processing content of a music generation unit.

FIG. 11 is a conceptual diagram for explaining a music generation rule database.

FIG. 12 is a conceptual diagram for explaining a melody generation process.

[Explanation of symbols]

1 ... Robot, 2 ... Body unit, 3 ... Head unit, 4A, 4B ... Arm unit, 5A, 5B ...
Leg unit, 40 ... Main control unit, 50 ... CC
D, 51 ... Microphone, 52 ... Touch sensor, 5
4 ... Speaker, 55 ... Battery sensor, 56 ... Acceleration sensor, 58 ... External memory, 63 ... Behavior generation unit, 70 ... Motion file playback unit, 71 ... Music generation unit, 73 ... Motion database , 80 ... Music generation rule database, 81 ... Emotion value table,
81 ...... rhythm pattern data file, 82 ...... chord progression table data file, 84 ...... melody scale data files, _{A 1} to A 14 _...... actuator, primitive motion control parameters ...... _PM 1 to P
M _n .

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI Theme Coat (reference) B25J 13/00 B25J 13/00 Z G10K 15/04 302 G10K 15/04 302F (72) Inventor Hanagatari Tokyo 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Masaki Fukuchi 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo 6-35 Sony, Inc. (72) Inventor Keisuke Kato Kita, Shinagawa-ku, Tokyo Shinagawa 6-35 Sony F-Term in Sony Corporation (reference) 2C150 BA11 CA01 DA04 DA24 DA26 DA27 DA28 DF03 DF06 DF08 DF33 ED10 ED42 ED52 EF07 EF16 EF17 EF22 EF23 EF29 EF33 EF36 3C007 AS36 CS08 KS10 WA03 KS01 KS23 WA31 KT23 WB16 WB19 WB23 WB25 WC06 5D108 CA04 CA08 CA13 CA15 CA25 5D378 MM98

Claims (6)

[Claims] 1. A robot apparatus comprising: a motion expression unit that expresses a motion; and music generation means that generates music according to the motion expressed by the motion expression unit and its own internal state. . 2. A sensor means for detecting an external and / or internal situation, and an emotion generating means for generating an emotion according to an output of the sensor means, wherein the internal state is generated by the emotion generating means. The robot apparatus according to claim 1, wherein the robot apparatus is in the emotional state. 3. The emotion comprises a plurality of digitized emotions, and the state of the emotion is the emotion having the largest numerical value among the emotions and the numerical value of the emotion. The robot apparatus according to claim 2. 4. A method for controlling a robot apparatus, which is characterized by generating music according to the operation and an internal state of the robot apparatus while causing the robot apparatus to generate an operation and outputting the music to the outside. 5. The method for controlling a robot apparatus according to claim 4, wherein the internal state is an emotional state of the robot apparatus generated according to an external and / or internal situation. 6. The emotion is composed of a plurality of digitized emotions, and the state of the emotion is the emotion having the largest numerical value among the emotions and the numerical value of the emotion. The control method of the robot apparatus according to claim 5.