JP4039595B2 - Robot system - Google Patents

Description

動作終了条件句：実行コマンドがｇｏｇｇｌｅの時、眼の往復回数指示が可能。（例：ＲＩＧＨＴＥＹＥ＜ｄｏ：＝ｇｏｇｇｌｅ＞＜ｔｉｌｌ：＝ｒｅｐｅａｔ△＞）
位置決め命令：ｓｅｔ ＥＹＥ＜ｓｐｅｅｄ：＝（速度指定情報）＞＜ｒｉｇｈｔ：＝（右眼差し位置座標）＞＜ｌｅｆｔ：＝（左眼差し位置座標）＞（部位名は、例えば左右眼共通であるが、左右別々に設定するようにしてもよい）
ｓｐｅｅｄ ：眼の移動スピードを指示
ｒｉｇｈｔ ：右眼差し位置を指示
ｌｅｆｔ ：左眼差し位置を指示
・「右眼差し」は位置決め命令のｒｉｇｈｔで設定される位置まで移動
・「左眼差し」は位置決め命令ｌｅｆｔで設定される位置まで移動
・「キョロキョロ」は位置決め命令のｒｉｇｈｔとｌｅｆｔで設定される座標間を、往復回数指定（＜ｔｉｌｌ：＝ｒｅｐｅａｔ ｎ＞）。往復回数が指定されていない時は一往復のみとなる。そして最終の眼の位置は左設定位置で停止する。
なお眼は最初右側へ動かす。
【００７３】
部位名：右瞼（左瞼）（部位指定情報：ＲＩＧＨＴＥＹＥＬＩＤ（ＬＥＦＴＥＹＥＬＩＤ）
アクチュエータ：モータにより上下に駆動される。
書式：ＲＩＧＨＴＥＹＥＬＩＤ＜ｄｏ：＝実行コマンド＞＜ｔｉｌｌ：＝条件式＞
ＬＥＦＴＥＹＥＬＩＤ＜ｄｏ：＝実行コマンド＞＜ｔｉｌｌ：＝条件式＞

動作終了条件句 実行コマンドがｂｌｉｎｋの時、瞬き回数指示が可能。（例：ＲＩＧＨＴＥＹＥＬＩＤ＜ｄｏ：＝ｂｌｉｎｋ＞＜ｔｉｌｌ：＝ｒｅｐｅａｔ ｎ＞）
位置決め命令 ｓｅｔ ＥＹＥＬＩＤ＜ｏｐｅｎ：＝（時間設定情報）＞（左右瞼共通）
ｏｐｅｎ ：瞼を開けている時間を指定
【００７４】
部位名：口（部位指定情報：ＭＯＵＴＨ）
アクチュエータ：モータにより開閉駆動される。
書式：ＭＯＵＴＨ＜ｄｏ：＝実行コマンド＞＜ｔｉｌｌ：＝条件式＞

動作終了条件句 実行コマンドがｐａｒｒｏｔの時、回数指示が可能。（例：ＭＯＵＴＨ＜ｄｏ：＝ｐａｒｒｏｔ＞＜ｔｉｌｌ：＝ｒｅｐｅａｔ ｎ＞）
位置決め命令 ｓｅｔ ＭＯＵＴＨ＜ｏｐｅｎ：＝（時間設定情報）＞＜ｃｌｏｓｅ：＝（時間設定情報）＞
ｏｐｅｎ ：口を開けている時間を指示
ｃｌｏｓｅ ：口を閉じている時間を指示
【００７５】
部位名：舌（部位指定情報：ＴＯＮＧＵＳ）
アクチュエータ：モータにより出入れ駆動される。
書式：ＴＯＮＧＵＳ＜ｄｏ：＝実行コマンド＞＜ｔｉｌｌ：＝条件式＞

動作終了条件句 実行コマンドがｌｉｃｋの時、回数指示が可能。
（例：ＴＯＮＧＵＳ＜ｄｏ：＝ｌｉｃｋ＞＜ｔｉｌｌ：＝ｒｅｐｅａｔ ｎ＞）位置決め命令 ｓｅｔ ＴＯＮＧＵＳ＜ｏｕｔ：＝（時間設定情報）＞＜ｉｎ：＝（時間設定情報）＞
ｏｕｔ ：舌を出している時間を指示
ｉｎ ：舌を入れている時間を指示
【００７６】
部位名：手（部位指定情報：ＲＩＧＨＴＦＩＮＧＥＲ（ＬＥＦＴＦＩＮＧＥＲ））
アクチュエータ：シリンダにより、各指が独立に曲げ伸ばし駆動される。

実行コマンド 指パターン指示（０〜３２のいずれかを選択することにより、各種指曲げ状態を設定する。例えば、０：パー／１：親指曲／２：人差指曲／４：中指曲／８：薬指曲／１６：小指曲／３１：グー）
位置決め命令 なし
【図面の簡単な説明】
【図１】本発明のロボットシステムの全体構成を示すブロック図。
【図２】ロボット骨格の模式図。
【図３】ロボット頭部に設けられた口及び舌の動作説明図。
【図４】同じく眼及び瞼の動作説明図。
【図５】ロボット制御部の記憶装置の内容を示す説明図。
【図６】その部位動作モジュール記憶部の内容を示す説明図。
【図７】ロボット制御部のＲＡＭの内容を示す説明図。
【図８】算出・駆動制御部の構成を示すブロック図。
【図９】駆動データ作成装置の一構成例を示すブロック図。
【図１０】 脚部骨格に対する拘束条件の説明図。
【図１１】上位動作プログラムによる処理の流れを示すフローチャート。
【図１２】中位動作ルーチンの処理の流れを示すフローチャート。
【図１３】正対モデルに対する骨格画像の寸法合わせ処理の説明図。
【図１４】アクチュエータの駆動量を算出する方法の説明図。
【図１５】ロボットの動作の流れをプログラム階層と対応付けて説明する図。
【図１６】歩行に関する基本動作の例を示す説明図。
【図１７】図１６に続く説明図。
【図１８】上位動作プログラムの記述例を示す説明図。
【図１９】その標準変更型中位動作ルーチンの定義ルーチンの一例を示す説明図。
【図２０】部位動作モジュールの削除、追加を行う場合の動作命令文の記述例を示す説明図。
【図２１】重ね合わせ型中位動作ルーチンを用いた上位動作プログラムの記述例を示す説明図。
【図２２】その重ね合わせ型中位動作ルーチンの定義ルーチンの記述例を示す説明図。
【図２３】組合わせ型中位動作ルーチンの適用に好適なロボット動作例を示す説明図。
【図２４】図２３の動作のタイミングチャートの一例を示す図。
【図２５】それに使用する組合わせ型中位動作ルーチンの定義ルーチンの記述例を示す説明図。
【図２６】駆動データ作成装置の全体の処理の流れを示すフローチャート。
【図２７】その入力処理のフローチャート。
【図２８】その入力処理において、骨格単位長さを定める処理の流れを示すフローチャート。
【図２９】各骨格単位の長さをモデル画像の対応部位に合わせ込む処理の流れを示すフローチャート。
【図３０】同じく、各骨格単位の端点の位置合わせ処理の流れを示すフローチャート。
【図３１】同じく、骨格の端点の座標を算出する処理の流れを示すフローチャート。
【図３２】同じく、駆動データ算出の流れを示すフローチャート。
【図３３】同じく、時間入力処理の流れを示すフローチャート。
【図３４】同じく、駆動データ保存処理の流れを示すフローチャート。
【図３５】コマの時間軸上における配列の一例を示す模式図。
【図３６】骨格画像のモデル画像に対する位置合わせ処理の説明図
【符号の説明】
１ 駆動データ作成装置
５ 装置制御部（駆動データ演算手段）
２００ ロボットシステム
２０１ ロボット
２０２ ロボット制御部
２０４ ＣＰＵ（プログラム読出手段、プログラム展開手段、非教示部位駆動データ生成手段）
２０５ ＲＡＭ
２０５ａ 上位動作プログラム格納部
２０５ｂ 中位動作コマンド解析プログラム格納部
２０５ｃ 中位動作ルーチン格納部(I)
２０５ｄ 中位動作ルーチン格納部(II)
２０５ｇ 中位動作ルーチン展開メモリ領域
２０７ 記憶装置
２１２ 算出・駆動制御部
２１３ アクチュエータ
３００ 上位動作プログラム記憶部
３０２ 中位動作ルーチン記憶部
３０４ 部位動作モジュール記憶部
３０６ 中位動作コマンド解析プログラム記憶部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a robot system.
[0002]
[Prior art]
When programming various articulated robots, such as robots that replicate human movements, in order to program the movement, conventionally, the drive amount of each actuator that drives the robot is individually set according to the time series of the operation. The method of setting is taken. In this case, in the operation of a certain robot, when movements of a plurality of robot parts occur simultaneously or in combination, the driving amount of the actuator must be designated for each robot part. For example, taking the action of “walking” as an example, in addition to the movements of both feet, the movements of the arms and the movement of the shoulders are accompanied, and the programs of the operations of these parts are individually assembled. Then, when programming a more complicated high-order operation such as “go forward one step, thank you and go back to the original place”, the high-order operation “go forward one step” , Programming work is performed by breaking it down into subordinate actions of “thank you” and “go down one step”.
[0003]
The above-mentioned conventional method has to set the driving amount and operation sequence of a large number of actuators for each robot part, so it takes time for programming work and is suitable for efficiently creating complicated programs for higher-level operations. It's hard to say. Further, when it is desired to change the contents of the upper operation, it is necessary to rewrite the program or data at the drive amount level of each actuator, which takes time. That is, since it is not so easy to change the operation content of the robot, there is a difficulty in lacking flexibility in various operations of the robot.
[0004]
In order to solve this problem, Japanese Patent Laid-Open No. 8-314524 proposes a control method using the following hierarchical control program. In this control program, a middle-order operation routine that forms lower-order motion data in units of drive data related to driving of actuators provided in a plurality of robot parts, and integrates a plurality of lower-order motion data to define the basic motion of the robot. Build up. Further, a middle-level operation command is associated with each of these middle-level operation routines, and a higher-level operation program is defined by a combination of the middle-level operation commands. Then, while driving the higher-order operation program in units of the middle-order operation commands, the drive of each actuator is controlled based on the combination of the middle-order operation routines corresponding to the read middle-order operation commands.
[0005]
[Problems to be solved by the invention]
By the way, in the above method, if you want to increase the number of intermediate motion routines that define the basic motion of the robot, create the intermediate motion routine for the basic motion you want to add newly, going back to the unit of the lower motion data each time. There must be. This intermediate operation routine can be created only at the lower operation data level, which is difficult to grasp intuitively. Therefore, it requires specialized knowledge and a high degree of skill in programming, and is required for general users. The content is difficult to handle. In particular, setting of operation conditions such as timing is particularly troublesome in the case of a complex operation in which different part operations are linked. In this case, not only does programming take time, but the prospect of the program and overall grasping are not easy, so that it is more difficult to realize the intended robot operation. Therefore, it is good that simple programming is realized by using intermediate operation commands. However, due to the above circumstances, a person other than an expert can take a frame of the intermediate operation commands (basic operations) prepared in advance. There is a disadvantage that it cannot be exceeded and is not flexible.
[0006]
It is an object of the present invention to create a program or change a program easily even when a robot performs a complex operation in which a plurality of part operations are linked, and as a result, a robot system capable of realizing a desired robot operation as desired. Is to provide.
[0007]
[Means for solving the problems and actions / effects]
  In order to solve the above problems, the robot system of the present invention
  A part motion program module (hereinafter referred to as a part motion module) that grasps the upper robot motion as a combination of several basic actions and regulates the motion of each robot part (hereinafter also referred to simply as a part) required for the basic motion.ShapeA high-order operation program that defines a high-order robot action by combining a plurality of the part-motion modules and that defines a basic action action that defines a basic action.Is the floorAs a layered programWriteIs described,
During ~Position operation routineIs the medium operation commandAnd mapThe upper operating program isUse intermediate operation commands and combine these intermediate operation commands.The upper robot movementRegulationAnd
A medium operation routine storage unit for storing a medium operation routine;
It is a combination of intermediate operation commandsAn upper operation program storage unit for storing the upper operation program;
  Program reading means for reading the upper operation program stored in the upper operation program storage unit in units of intermediate operation commands;With,
  By the program reading meansRead outInsideBased on the combination of the intermediate action routines corresponding to the order action commands,
The part operation module is formed in units of drive data related to driving of an actuator provided in the robot part.ActuatorBy its driving dataControl the driveforActuator operation controllerWhen,CorrespondingRobot partOperation condition specification command including operation start timing specification command that specifies the operation start timing ofAnd consists ofThe actuator operation control unitThatControls actuator drive related to basic motion or part motion according to the motion conditions defined by the motion condition definition command.With
  The motion start timing regulation command included in the motion condition regulation command of the part motion module is based on the motion of the robot part defined by a series of model images of each frame in which the motion of the model corresponding to the robot motion is recorded over time. The operation start timing is given based on either the start time, end time or specific frame position passing time of the operation of the robot part when used asIt is characterized by that.
[0008]
In this robot system, the intermediate motion routine is configured as a routine for defining the operation for each robot part, that is, a collection of part action modules, and further, the condition is defined for each part action by the operation condition defining command. I have to. Since this operation condition defining command includes an operation start timing defining command for defining the operation start timing of the basic motion or the region motion, when combining a plurality of region motions in a simultaneous linkage, the operation start timing defining command is used to start each operation. By specifying the timing, the operation of a certain part is started later than the other part, or the operation of a specific part is started after a certain operation is completed, and further, the operation of specific parts is started simultaneously. It is possible to easily specify complicated operations such as
[0009]
As a result, it becomes easier to intuitively understand the creation of the intermediate operation routine, so that even general users who do not have specialized knowledge and advanced skills in programming work can easily handle it. In addition, the range of programming can be expanded. On the other hand, since the intermediate operation routine is configured in units of operation modules for each part, it is desired to change only the operation of a specific part to another action, or to prevent the part from being operated, or even newly It is possible to easily respond to a request such as adding a part action module by replacing, deleting, or adding a part action module. In any case, even when the robot performs complex motions that combine multiple part motions, it is extremely easy to create or change a program, and it is easy to see the program and understand the entire program. It can be realized as desired.
[0010]
The operation start timing defining command can give an operation start timing based on an elapsed time from a predetermined time origin. In this case, for example, the operation timing can be easily defined by grasping the time by timer measurement or the like.
[0011]
In the above robot system, the robot can be configured such that at least a part of the plurality of robot parts includes a skeleton having a plurality of skeleton units and actuators arranged between the skeleton units. In this case, at least a part of the plurality of part motion modules is a drive data creation device having the following requirements, that is,
A series of model images in which the motion of the model corresponding to the robot motion is recorded over time, and image display means for displaying a skeleton image;
Skeleton positioning means for aligning the skeleton image with the model image in the display means;
Created in units of frame-by-frame motion by a drive data creation device including data calculation means for calculating drive data based on displacements of the skeleton images respectively aligned with the model images that are aligned with each other on the time axis. The driving data can be a teaching operation routine part created by editing each part of the robot.
[0012]
According to this method, for example, a human model is actually performed on a human model, and this is recorded as a series of images in frame units. The skeleton image is positioned with respect to the model image at each specific time of the model operation that changes with time, and drive data is calculated based on the skeleton displacement between adjacent images. In other words, intuitive driving data can be generated by an image that directly teaches the movement of the model to the robot. Therefore, it is possible to obtain the lower-order motion data for each part included in the middle-level motion in a lump, and if the middle-level motion command is assigned and stored in the storage unit, it can be used as it is as a middle-level motion routine. Become. The operation for creating the drive data is centered on the operation of aligning the skeleton image with the model image on the screen of the image display means, so that the content is easy to understand. Since it is not necessary to set the drive amount one by one, a large amount of drive data necessary for creating the upper operation program can be obtained reasonably and simply.
[0013]
In this case, the motion of a certain part defined by the teaching motion routine part is used as a reference, and the motion start timing defining command is based on either the start time, end time or specific frame position passage time of the part motion. Operation start timing can be given. This is convenient when it is desired to set the timing on the basis of the operation of a specific part. Particularly, since the linked operation timing setting between the parts becomes easy, it is possible to more easily program a complicated operation.
[0014]
Next, when the teaching operation routine part is created by the drive data creation apparatus described above, for example, there may be a robot part where it is difficult or impossible to calculate the drive data from the skeleton image aligned with the model image. (For example, movement of wrists, fingers, eyes, mouth, etc., lighting of lamps, etc.). If the intermediate motion routine includes a part motion module of a robot part (non-teach part) where the teaching operation routine part is not created by the drive data creation device due to the above factors, the part motion module The non-teaching part positioning instruction routine corresponding to can be incorporated in the program. Further, it is possible to provide non-teaching part drive data generating means for generating drive data for the actuator of the corresponding non-teaching part based on the contents of the positioning command routine part.
[0015]
In the above configuration, when incorporating the operation of the non-teaching part where data cannot be created by the drive data creation device, the driving data of the actuator of the corresponding non-teaching part is displayed during the program execution based on the contents of the positioning command routine part. It will be automatically generated. This eliminates the need to prepare drive data for various non-teaching parts in advance and also makes it easy to incorporate the part movements of the non-teaching parts, so that the programming effort can be saved. In this case, the positioning command routine portion can include positioning information for designating at least one of the movement start position, movement end position, and movement speed of the corresponding non-teaching part. For example, when a non-teaching part moves only between two predetermined positions with respect to a specific part of the robot, such as rotation of the eyes, opening / closing of the mouth, and insertion / extraction of the tongue, or only simple rotational movement In the case where it is not performed, the drive data can be generated without any problem by such simple positioning information.
[0016]
  The operation start timing regulation command isIn the operation condition specification command of the part action module that constitutes the middle action routineIf it is not given to the action start timing of the corresponding basic action or part actionIs the middle-level routine executionSet to start timeRukoYou can. For example, in the combination type intermediate operation routine described above, the default start time (for example,Of the intermediate operation routineIf you want to start the operation of several parts at the same time in the program execution start), by providing the means as described above, the expected action can be executed without specifying the timing for those parts. Therefore, the program can be simplified.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows an overall configuration of a robot system 200 according to an embodiment of the present invention, which includes a robot 201, a robot control unit 202, and a drive data creation device 1, which are connected via communication interfaces 203 and 152, respectively. The communication lines 150 are connected to each other.
[0018]
The robot control unit 202 includes a CPU 204, a RAM 205 and a ROM 206 connected thereto, a storage device 207 including a hard disk device, an input device 208 such as a keyboard, a display control unit 210, and the like. A display device 211 such as a CRT is connected to the display control unit 210, and a communication interface 203 is connected to the CPU 204.
[0019]
The storage device 207 functions as a standard operation routine storage unit, a middle operation routine storage unit, and a part operation module storage unit. The CPU 204 functions as a program reading unit, a program development unit, and a non-teaching part drive data generation unit. Further, the RAM 205 functions as a work area and development storage means for the CPU 204.
[0020]
The robot 201 is configured, for example, as a robot that replicates human movements, and includes a skeleton 19 in which skeleton units 20 to 30 are connected to each other by joint mechanisms 31 to 37, as shown in FIG. The skeleton is divided into, for example, the following robot parts (hereinafter referred to as parts A, B, C...).
-Head (20): provided with an eye 20a and a mouth 20c. As shown in FIG. 3, the opening 20c can be opened and closed by an actuator (not shown). Furthermore, a tongue portion 20d is provided inside the mouth portion 20c so as to be able to enter and exit by an actuator (not shown). Further, as shown in FIG. 4, the eye 20a is provided so as to be rotationally driven left and right by an actuator (not shown). Further, a collar portion 20e that covers or opens the eye portion 20a is provided so as to be driven by an actuator (not shown). Further, a speaker 192 (voice output unit) shown in FIG. 1 is also built in, and voice such as a voice synthesized by the voice synthesis unit 191 can be output by a command from the CPU 204. Note that in this specification, the audio output unit is also regarded as one of the parts, and the sound output by that is also regarded as one of the operations.
-Torso: including shoulder 21, spine 22, and hip bone 23.
Left and right arm parts: each has a humerus 24, a humerus 25, and a hand part 29. In addition, each finger of the hand portion 29 can be independently driven to bend and stretch, and for example, various hand shapes such as “goo”, “choki”, “par” of janken can be made. .
-Left and right leg portions: each includes an upper limb bone 26, a lower limb bone 27, and a foot portion 30.
[0021]
As shown in FIG. 1, the robot 201 includes a calculation / drive control unit (calculation / drive control) corresponding to each of the above parts (in the drawing, simply indicated by A, B, C, etc.). Means) 212 is provided and is connected to the CPU 204 of the robot controller 202. The calculation / drive control unit 212 is connected to an actuator 213 included in each part. The types of actuators (pulse motor, servo motor, piston cylinder, etc.) and the number of shafts, the degree of freedom of shaft rotation, and the range of shaft rotation or piston slide provided in each joint mechanism 31 to 37 in FIG. Etc. are set individually for each skeleton unit.
[0022]
Next, as shown in FIG. 5, the storage device 207 of the robot control unit 202 (FIG. 1) includes a higher-order operation program storage unit 300, a middle-order operation routine storage unit 302, a site action module storage unit 304, and a middle-order operation. A command analysis program storage unit 306, a non-teaching part drive data generation program storage unit 308, and a system program storage unit 312 are formed, and each stores a corresponding program or routine (or data).
[0023]
The operation of each program is as follows.
(1) Higher-order operation program: A desired higher-order operation is described by selecting and arranging appropriate ones from intermediate operation commands prepared in advance. For example, if you want the robot 201 to perform a series of high-order actions such as "turn right and walk four steps, turn right and step one and a half steps, and pose karate chops" This is understood as a combination of several basic actions such as “right leg half step” and “left leg one step”. In the higher-order operation program, operation instruction statements 300a including intermediate operation commands that respectively define basic operations constituting the higher-order operation are arranged in a predetermined order reflecting the execution order, for example, the execution order. . In this embodiment, the intermediate operation command included in the operation command statement is described in a form including the definition name of the basic operation.
[0024]
(2) Intermediate operation routine: Basic as a combination of module names (motion specifying information) PM11, PM12,... Of the site motion program module (site motion module) that defines the motion of each robot site required for the corresponding basic motion. Stored in association with the action definition names 1, 2, 3,.
(3) Part motion module: As shown in FIG. 6, it is configured to include a group of actuator drive data included in parts A, B, C... Of the robot 201 in FIG. Information) 1, 2,... And module names (operation specifying information) PM11, PM12,. Each drive data includes displacement data for each time at both ends of the skeleton unit driven by each actuator, but may be angle data between the skeleton units. On the other hand, the drive data can be a drive amount of each actuator for each time.
[0025]
(4) Intermediate operation command analysis program: The upper operation program is read in units of intermediate operation commands, and corresponding intermediate operation routines are sequentially started.
(5) System program: Provides an operating environment for the upper operation program on the CPU 204 and an environment for rewriting or newly creating the upper operation program. Specifically, a program creation screen is opened on the display device 211 (FIG. 1), and a higher-level operation command is input using the input device 208 in a format (described later) as shown in FIG. A name is assigned and stored in the higher-order operation program storage unit 300 of the storage device 207. The stored upper operation program is displayed on the program creation screen, and the upper operation command is added, deleted, inserted, etc. based on the input from the input device 208, and the program is corrected / edited. On the other hand, the system program also performs a process of expanding a standard change type intermediate operation routine or a composite type intermediate operation routine, which will be described later, into a part operation module. These types of intermediate operation routines are stored in the upper operation program storage unit 300 in the form of a definition routine 300b and incorporated in the upper operation program. That is, a part of the storage unit 300 also functions as a middle-level operation routine storage unit.
(6) Non-teached part drive data generation program: will be described later.
[0026]
On the other hand, as shown in FIG. 7, the RAM 205 has an upper operation program storage unit 205a, a middle operation command analysis program storage unit 205b, a middle operation routine storage unit (I) 205c, and a middle operation routine storage unit (II). 205d and the like are formed, and corresponding programs and data read from the storage device 207 are stored. The storage unit 205c, 205d is loaded with each part operation module included in the intermediate operation routine related to execution (for the standard change type intermediate operation routine or the composite type intermediate operation routine, the intermediate operation routine). After being expanded in the form of a part action module in the expansion memory area 205g, it is loaded). Note that the two intermediate operation routine storage units (I) and (II) (or three or more) may be provided because the intermediate operation routine to be executed next is read and accumulated in advance. This is to reduce a delay in response based on reading of data from the storage device 207 when the robot 201 moves to the next operation.
[0027]
Further, as shown in FIG. 8, the calculation / drive control unit 212 includes a CPU 214, a RAM 215, a ROM 216, and the like, and a drive amount calculation program 216a is stored in the ROM 216. Then, the calculation / drive control unit 212 executes each of the actuators 213 connected to the CPU 214 based on the drive data DA1, DA2,... The driving amount of 1) is calculated in time series, and each actuator 213 is driven based on the calculation result. The calculation method will be described later.
[0028]
FIG. 9 is a block diagram illustrating a configuration example of the drive data creation device 1. The drive data creation device 1 includes a device control unit 5 including a CPU 2, a ROM 3, and a RAM 4 that constitute drive data calculation means, and includes an image capture control unit 6, a display control unit 7, an input device interface 8, and the entire device. A program storage device 9, a drive data storage device 10, and a moving image data storage device 11 storing a control program for controlling the control are connected. The input device interface 8 is connected with a mouse 12 and a keyboard 13 as a skeleton positioning means having a click button 12a (input unit). A CRT 14 as an image display means is connected to the display control unit 7. On the other hand, the image capture control unit 6 is connected to a video playback device (VTR) 15 and includes a VRAM 16 and a data transfer processor 17, and the image display data stored in the VRAM 16 is displayed via the bus 18. Direct transfer to the control unit 7 is possible.
[0029]
Next, some of the skeleton units forming the skeleton 19 of the robot 201 shown in FIG. 2 form a set, and the relative positional relationship between the skeleton units included in the set is predetermined. The restricted conditions are set. For example, taking a skeleton unit constituting both legs as an example, the lower limb bone 27 moves only in a plane P formed by a straight line along the upper limb bone 26 and a straight line perpendicular to the hip bone 23 as shown in FIG. Restriction conditions are set (in FIG. 10, the joint mechanisms 35 and 36 (FIG. 2) are not drawn). Therefore, if the upper end position of the upper limb bone 26 and the lower end position of the lower limb bone 27 are designated, the position of the coupling point between them (the portion corresponding to the knee) is automatically determined according to the constraint conditions.
[0030]
Next, the operation of the robot system 1 of this embodiment will be described in detail.
When the high-order operation program name is designated and started by the input device 208 or the like in FIG. 1, the middle-level operation command analysis program is started, and as shown in the flowchart of FIG. The intermediate operation command is read, and the corresponding intermediate operation routine is stored in the intermediate operation routine storage unit (I) 205c of the RAM 205 in M2. Next, a signal is issued to the stored intermediate operation routine to activate it (M3). During the execution of the intermediate operation routine, the next intermediate operation command is read by M4, and the corresponding intermediate operation routine is read by M5 in the intermediate operation routine storage section (in this case, the storage section ( II)). Further, if a middle-level process completion signal is transmitted from the middle-level operation routine (storage unit (I) side) that is being activated / executed in M6, the middle-level operation routine stored in the storage unit (II) 205d. An activation signal is transmitted to (M7). If the next intermediate operation command is present in M8, the process returns to M4, the next command is read and the corresponding intermediate operation routine is stored in an empty storage unit (in this case, the storage unit (I)), Thereafter, by repeating the same processing, the higher-order operation program is executed in units of intermediate operation commands. If the next middle operation command does not exist or is an end command, the process ends (M9).
[0031]
Next, as shown in the flowchart of FIG. 12, the middle operation routine side reads the drive data of the actuators at each part by receiving the activation signal from the higher operation program side (B1, B2). Then, the drive data described as displacement data for each time of the actuator is transmitted to the calculation / drive control unit 212 of each part in chronological order (B3). Then, the calculation / drive control unit 212 side calculates the drive amount of each actuator based on the displacement data based on the drive amount calculation program 216a (FIG. 8), and drives these actuators based on the calculation result. .
[0032]
As an example of the driving amount calculation method, a method based on the following principle can be employed.
First, in the skeleton 19 shown in FIG. 2, for example, indexes are given to both end positions of each skeleton unit as shown in FIG. 13 (W1).
-Head 20: H, f1.
-Shoulder 21: A1, H and H, A2.
-Spine 22: G and H. Here, G is used as a reference position when the driving amount of the actuator is calculated from the displacement of each skeleton unit.
-Hip bone 23: L1 and L2. G is positioned at the midpoint of the line segment connecting L1 and L2.
The humerus 24 and the humerus 25: A1, Ae1, Ah1 and A2, Ae2, Ah2.
Upper limb bone 26, lower limb bone 27: L1, Lk1, Lf1 and L2, Lk2, Lf2.
The displacement data is calculated as the displacement of both ends of each skeleton unit.
[0033]
For example, for the left leg in FIG. 13, as shown schematically in FIG. 14 (a), the position of the point G is unchanged, and the positions of L1 and Lk1 at a certain time T1 are at the next time T2. When moving to L1 ′ and Lk1 ′, the line segment GL1 and the line segment L1Lk1 are rotated, and the line segment GL1 is superimposed on the line segment GL1 ′ as shown in FIG. Then, by calculating the angle θ between the line segment L1Lk1 at time T1 and the line segment L1′Lk1 ′ at time T2 from the spatial coordinates of each point after the rotational movement, both skeletal units (that is, the upper arm portion 26). And the rotation angle of each axis of the actuator disposed between the lower leg portions 27). If the processing for obtaining the rotational movement of the skeleton unit and the rotation angle of the second skeleton unit is sequentially performed in the direction toward the end of the skeleton, the driving amount of the actuator included in each joint mechanism is sequentially calculated. It becomes. In the above description, the actuator is composed of a motor, and the first and second skeleton units perform only relative rotational motion by rotation of the motor shaft. In the case of an actuator, relative translational movement may be performed between the skeleton units due to expansion and contraction of the piston. In such a case, the translational movement distance is calculated in addition to the rotation angle of the second skeleton unit.
[0034]
When the driving amount of the actuator is calculated in this way and a series of driving of the actuator based on the driving data of the intermediate operation routine is completed, the lower-level processing completion signal is sent from the driving amount calculation program 216a to the intermediate operation routine side. Sent. The middle operation routine side receives this in B4 of FIG. 12, and transmits a middle processing completion signal to the upper operation program (B5). When the actuator drive amount itself is transmitted as drive data, the above calculation process on the calculation / drive control unit side is not necessary.
[0035]
FIG. 15 shows a specific execution example of the upper level operation of the robot 201 based on the above flow. That is, as shown in FIG. 16 and FIG. 17, the high-order operation of “walking 4 steps” is “right leg half step” (definition name “WALK2”), “left leg 1 step” (definition name “WALK3”). ), “One step on the right leg” (definition name “WALK4”), and “half step on the left leg” (definition name “WALK5”). It is assembled by using an operation command statement using a position operation command (including the corresponding definition name). Then, for example, the part movements constituting the basic action of the first “right leg half step” are “half stepping” and “front descending” with respect to each part of the right leg part and the left arm part, respectively. The drive data of the related actuator included in the operation module is transmitted to the calculation / drive control unit 212 in time series order (transmission control is performed by the above-described system program). The calculation / drive control unit 212 calculates the drive amount based on the transmitted drive data, drives the corresponding actuator, and proceeds to the execution of “the left leg 1 step” which is the content of the next middle operation command. .
[0036]
This will be described in more detail below with reference to specific program examples. FIG. 18 shows an example of a program for performing a high-order operation such as “turn right, walk four steps, thank you, turn right, step one and a half steps, and pose karate chop”. As described above, the program includes an operation instruction routine and a definition routine.
[0037]
In this embodiment, the operation instruction routine is an arrangement of one or more operation instruction statements in the order of execution. In a certain operation instruction statement, a command indicating that it is a middle operation routine of a defined basic operation is "Motion" command,
motion <do: = “definition name”>
The intermediate operation command for instructing the execution of the intermediate operation routine indicated by the definition name is defined. Then, the operation command statements by such middle operation commands are arranged in the order of execution (note that each sentence is not given a line number, but the execution order may be specified by giving a line number) . Note that the definition name of the basic action that is defined in each intermediate action command is indicated in parentheses at the end of each sentence.
[0038]
Here, among the intermediate operation routines, the combination type intermediate operation routine or the standard change type intermediate operation routine adopts the definition name of the corresponding format, and in this embodiment, at the head of the definition name. Specific character information, for example, “@” is given to distinguish it from other types of intermediate operation routines. For such a middle-level operation routine with a definition name, a definition routine for defining the contents of the middle-level operation routine is written in the program (if not written, it is determined that an error has occurred). . The definition name of the basic action is identified in a predetermined display format (in this embodiment, the definition name is enclosed in quotations ("")) to identify that it is the definition name of the basic action. Yes.
[0039]
In this embodiment, the basic action (definition name “@KARATE”) of “performing karate chopping while taking out the right leg half step forward” is the basic action (definition name) of “right half leg before step”. "WALK2") is set as a standard change type intermediate operation routine with standard operation. FIG. 19 shows an example of the definition routine. First, at the top of the definition routine
"Joint motion to @KARATE"
Is a statement for specifying the type and definition name of the intermediate operation routine, and “jointmotion to” indicates that it is a standard modified intermediate operation routine (the same is true for the combined intermediate operation routine) ing. Then the second line
“Base <do: =“ WALK2 ”>”
Is a statement instructing to select a middle-level operation routine having the definition name “WALK2” as a standard operation routine.
[0040]
Then on the third line
item RIGHTTARM <do: = CHOP>
Is an action command statement for instructing a part action to cause the right arm of the robot to perform a karate chop. “Item OO” is part specifying information and has a meaning of “specify XXX as a part”. Then, the right arm part for performing the karate chop is designated as the motion part by “item RHITARM”. Further, “<do: = CHOP>” is action specifying information, which means “can perform a part action specified by a module name“ CHOP ””. In this embodiment, the module name of the part action is distinguished from the definition name of the intermediate action routine by not providing a quotation. Note that “END” in the last line indicates the end of the definition routine.
[0041]
Now, the flow of processing of the system program when executing a higher-level operation program including such a definition routine is as follows. First, the program of FIG. 18 is read from the storage unit 300 of the storage device 207 of FIG. 5 and loaded into the storage unit 205a of the RAM 205 of FIG. The program is read in units of lines (that is, intermediate operation commands), and if this is a normal intermediate operation command (without “@” added to the beginning of the definition name), the corresponding intermediate The unit operation routine is read from the middle operation routine storage unit 302 in FIG. Then, referring to the module name included therein, the corresponding part action module is read from the part action module storage unit 304 in FIG. 6 and loaded into the intermediate action routine storage unit (I) 205c of the RAM 205. In the loaded intermediate operation routine, the drive data of the actuator of each part operation module included therein is transmitted to the calculation / drive control unit 212 (FIG. 1) of the corresponding part to drive the actuator of each part. Basic operations are performed. During this execution process, similar processing is executed for the next program line, and the intermediate operation routine is loaded into the intermediate operation routine storage unit (II) 205d (FIG. 7).
[0042]
Here, if the intermediate operation command is one that specifies a standard modified intermediate operation routine (that is, one that has “@” added to the beginning of the definition name), the corresponding definition routine is Read and move to the following expansion process. First, “base <do: =“ WALK2 ”>” reads the intermediate operation routine corresponding to the definition name (in this case “WALK2”) as the standard operation routine by recognizing the “base” command, Is stored in the middle-level operation routine expansion memory area 205g of FIG. Next, in item RIIGHTTARM <do: = CHOP>, it is confirmed whether or not there is an operation of the same part in the standard operation routine by recognizing the part specifying information (item RHIGHTARM). If it exists, the part action module specified by the action specifying information (<do: = CHOP>) is replaced with the module of the corresponding part in the standard action routine stored in the development memory area 205g. In this case, “WALK2” is the “right leg half-walk” walking motion, and the motion of the right arm is included as the region motion, which is replaced with the right arm motion of the karate chop.
[0043]
Although not shown in the above example of the definition routine, the following processing is performed by using the operation command statement shown in FIG. First, if “FAULT” (that is, <do: = FAULT>) is described as a specific definition name, a process of deleting a module of the corresponding part from the standard operation routine in the expanded memory area 205g is performed. (I.e., the operation of the part is prohibited). In addition, when an operation of a part that does not exist in the standard operation routine is designated, a corresponding part operation module is added to the standard operation routine. In the operation statement shown in the figure,
item RIGHTWLIST <do: = turn>
This shows the part movement of the right wrist (RIGHTWLIST).
[0044]
In addition, in an operation command statement for specifying replacement or addition of a part motion, it is possible to specify an action using a middle-level operation routine in addition to an operation designation by a module name of a part motion module. In this case, the intermediate operation routine is designated by the definition name, and the part is designated by the part name. As a result, the module of the designated part of the intermediate operation routine is replaced or added to the standard operation routine in the development memory area 205g. In the operation statement shown in the figure,
item RIGHTTARM <do: = “WALK3”>
However, this indicates that only the swinging motion of the right arm of the basic motion “one step before the left leg” is incorporated.
[0045]
In this way, when the execution of all the operation statements in the definition routine is completed, the standard operation routine in which the module replacement, addition and deletion processes have been completed in the expanded memory area 205g, that is, the expanded standard change It is a form that stores the middle-level operation routine. If the expansion process such as module replacement is performed using only the module name, it is not necessary to read the module from the storage unit 304 in FIG. In this case, after the expansion process is completed, the module name in the expansion memory area 205g is referred to read the corresponding module from the storage unit 304 and store it in the intermediate operation routine storage unit 205c or 205d. .
[0046]
By the way, in the high-order operation shown by the program in FIG. 18, the user walks 4 steps after turning to the right. The routine portion that walks 4 steps is integrated as a superposition type intermediate operation routine described below. It is also possible. An example thereof is shown in FIG. The difference between this program and the program of FIG. 18 is that an operation command statement group of 2 to 5 lines corresponding to the portion describing the “walk four steps” operation of FIG.
motion <do: = ”@@ ×××× 1”>
And a definition routine corresponding to the definition name “@@ xxxx × 1” is added. In other words, the action of “walking four steps” is assembled as a superposition-type intermediate action routine (definition name “@@ xxx × 1”) in which four walking actions that do not overlap in time series are superimposed. ing. In this embodiment, specific character information such as “@@” is added to the head of the definition name to distinguish the superposition type routine from other types of intermediate operation routines.
[0047]
FIG. 22 shows an example of the definition routine. First, at the top of the definition routine
“Pileup motion to @@ ×××× 1”
Is a sentence for specifying the type and definition name of the middle-level operation routine, and “pileup motion to” indicates that it is a superposition type middle-level operation routine. Thereafter, the instruction statements of the basic operations to be superimposed are arranged in the order of execution, and “end” is added to the last line to close the definition routine. In this embodiment, a command for repeatedly executing a specific basic operation sequence is prepared. In this embodiment, “repeat while ○”... “End” corresponds to this, and means that an operation sequence sandwiched between them is repeated “O” times. In FIG. 22, the program describes the contents of walking for four consecutive steps by repeating the operation of alternately performing “one step on the left leg” and “one step on the right leg” twice.
[0048]
Next, a description will be given of a program example of the combination-type intermediate operation routine and the flow of the processing. The combination type intermediate operation routine is executed in a state in which the basic operation and / or the region operation related to the intermediate operation routine and / or the region operation module to be combined with each other occurs at least partially in time series. High-order middle movement. An example will be described with reference to FIG. Here, the robot junken operation is taken as an example.
[0049]
For example, consider the action of putting out a par while uttering "Jan Kempon". This can be considered by breaking it down into the following part movements. First, as shown in (1), with the right hand in the “goo” state, the right arm is raised, and “Jean” is pronounced during the raising. In addition, an operation of opening and closing the mouth for a short time (hereinafter referred to as a mouth-to-mouth operation) is performed once according to the pronunciation. Next, in (2), the right arm raised up is stopped and pronounced “ken”. In (3), while swinging down the right arm, the right hand keeps the state of “goo” halfway, and in (4), while the right hand is in the “par” state and pronounced “pon”, Complete the lowering. In this example, in (5), after the arm swing is completed, an operation of the robot opening its mouth and sticking out its tongue is added.
[0050]
In the above operation, the right arm, the right hand, the mouth, the tongue, and the voice are in a form of performing linked operations at appropriate timing. An example of the operation timing is shown in FIG. The arrow in the figure represents the operation duration. In this example, the right arm swings up, stops, and swings down in one second each, with the mouth closed and the right hand goo by default. In addition, the swinging motion of the right arm is incorporated in basic motions that include part motions such as the back and forth movement of the hips, the vertical movement of the shoulders, and the movement of the head in order to show the gesture of an exaggerated robot. The teaching operation is, for example, 10 frames (frame feed speed: 0.1 second / frame) created by the creating device. The definition name of the intermediate operation routine is “RLIFUPUP”. On the other hand, the swing-down of the right arm is also incorporated in the basic operation, and the definition name of the intermediate operation routine is “RSHAKEDN”. Then, “Jan” is uttered in the middle of the right arm swing cycle, and “Ken” is uttered in the middle of the right arm stationary cycle. When the right arm is swung down, the right hand moves to “Par” from the seventh frame. At the same time, utter “Pong”. Then, as soon as the swinging of the right arm is completed, the tongue is pushed out.
[0051]
FIG. 25 is an example of a definition routine of a combination-type intermediate operation routine that defines the operation as described above. First, at the top of the definition routine is “join motion to @JANKEN”, which is the same as the standard change type intermediate operation routine. Subsequent (a) to (l) are operation command statement groups for designating a part operation or basic operation to be combined, and command each operation (a) to (l) in the timing chart of FIG. Is. Among these, (a) to (e) are in the operation process of (1) in FIG. 23, (f) and (g) are in the operation process of (2), (h) is in the operation process of (3), (I) to (k) correspond to the operation process 4, and (l) corresponds to the operation process (5).
[0052]
In the combination type intermediate operation routine, for example, the execution start time of the intermediate operation routine is set as the time origin, and unless the execution start timing is specified by a command described later, the time origin is set as the default start time, Each part operation or the incorporated basic operation starts simultaneously (time measurement is performed by the timer 190 in FIG. 1). That is, in (a) to (c), since the start timing is not specified, the right arm swinging motion ((a)), the right hand “goo” motion ((b)), and mouth At the same time, the closing operation of is started. Note that “RIGHTFINGER” is the right hand (finger), “MOUTH” is part designation information representing each part of the mouth, <do: = 31> is finger bending for making “Goo”, <do: = “close>” is operation designation information for designating each mouth closing.
[0053]
The next (d) is an operation command statement for instructing the utterance of “Jean”. In this embodiment, the name of the part corresponding to the voice, that is, “VOICE” is specified, followed by <do: = ”◇◇◇◇”> ◇◇◇◇, and the utterance content directly into characters (or character codes). The content of the utterance is specified by writing. The following <from: = XXX> is one of the operation condition defining commands, and is an operation start timing defining command for designating the operation start timing. <From: = timer ΔΔ> means “start operation after ΔΔ times the timer measurement unit (0.1 second in this embodiment) from the time origin”. That is, (d) the whole meaning is “speak“ Jan ”after 0.5 seconds from the time origin (that is, from the start of swinging the right arm)”.
[0054]
In (e), the mouth is selected as a part, and the action of the mouth is specified by <do: = parrot>. Since the operation start timing is <from: = timer 5>, it is the same as the utterance timing of “Jan”. Furthermore, <til: (condition name)> is a command that defines the condition for ending the operation. For example, <til: = repeat Δ> means that the operation is ended if the part operation is repeated Δ times. For example, by <til: = repeat 1>, the mouth pack is performed only once. If <til: = repeat Δ> is omitted, a default number of repetitions (for example, 1 time) may be automatically specified. In this way, the operations (a) to (e) in the timing chart of FIG. 24 are executed in combination with the operation command statements (a) to (e), and the operation (1) of FIG. 23 is realized. I understand.
[0055]
Next, the operation when the arm is lowered will be described. First, the swing-down of the right arm is started after 2 seconds from the time origin, and the start timing is given by <from: = timer 20> in the corresponding operation command statement of (h). As shown in FIG. 24, the time from the start to the end of arm swing is 1 second in terms of time, and 10 frames in terms of the frame feed number. Then, in the seventh frame (2.7 seconds in terms of time from the time origin), the right-hand “goo” operation, “goo” utterance operation, and mouth-packing operation start simultaneously.
[0056]
Corresponding to these are the three operation command statements (i) to (k) in FIG. Here, the operation start timing is defined by the frame advance number of the right arm operation. The format of <from: = item (part name) [△]> of the action start timing specification command written in each action command statement is “△ frame of the part action specified by item (part name)”. To start the operation ". Therefore, the start time of each operation is set at the seventh frame from the start of swinging down of the right arm by <from: = item RIGHTTARM [7]>.
[0057]
The last is the action of sticking out the tongue, but the corresponding action command sentence in FIG. 25 is (l).
The drive data as described above can be created by the drive data creation device 1. item TONGUS designates the tongue as a part name, and <do: = out> designates an “out” operation. Then, the operation start timing defining command <from: = item RIHTTARM [back]> specifies the part movement of the right arm as a reference part for timing setting, similar to the above-described <from: = item RIHTTARM [7]>. However, [back] means “start the operation when the operation of the part (right arm) is completed”. Therefore, when the lowering of the right arm is completed, the robot performs an operation of sticking out the tongue.
[0058]
The main part of the robot drive data as described above is taught and created by the drive data creation device 1 of FIG. 9, and each part motion module is also created as a teaching motion routine part based thereon. Hereinafter, the operation flow of the drive data creation device 1 will be described with reference to flowcharts. FIG. 26 shows a rough flow of the entire process. First, in the input process of S1100, a skeleton image is superimposed on the model image to generate drive data (in other words, the operation of the robot 201). Teach). This process is performed in units of basic operations. Here, prior to the input, the basic operation is performed on the human model, and the model image 38 (FIGS. 16 and 17) is shot with a video camera at predetermined frame intervals. Then, the moving image reproduced by the VTR 15 of FIG. 9 is captured as moving image data via the image capturing control unit 6 and stored in the moving image data storage device 11. Here, as shown in FIG. 35, the moving image data stored in the moving image data storage device 11 is arranged so that the intervals of the frames 80 are close when the model moves relatively complicatedly. On the contrary, when the motion is relatively monotonous, the frame of the moving image is appropriately thinned out when capturing the moving image data or editing the data after the capturing so that the interval between the frames 80 becomes coarse.
[0059]
FIG. 27 is a flowchart showing the details of the input process in S1100. First, after setting a part that does not require calculation of drive data (S1101), windows W1 and W2 as shown in FIG. 13 are displayed, respectively. (S1102, S1103). Among these, the model image 38 is displayed in one window W1, and the skeleton image 39 is displayed in the other window W2. The skeleton image 39 is displayed as a simplified diagram of the robot skeleton 19 shown in FIG.
[0060]
Returning to FIG. 27, in S1104, processing is performed to match the dimensions of the skeleton image 39 with the model image 38 (FIG. 13) captured with the model facing the video camera. Details of the processing are shown in FIGS. That is, in FIG. 13, the model image 38 in the face-to-face pose is displayed in the window W1 and the skeleton image 39 is displayed in the window W2 (C101 to 103), and the length of each skeleton unit is matched with the corresponding part of the model image (C104). ). Here, standard dimensions are set in advance for each skeleton unit of the skeleton image 39. 29. As shown in FIG. 29, first, the position of the reference point G is aligned (C201), and then the end points of each skeleton unit are sequentially aligned so as to gradually move away from the reference point G. Go out.
[0061]
Here, the input method is to match the end points of the skeleton image 39 to the model image 38. As shown in FIG. 13, each end point is displayed as a mark point on the window W2, and the window W1. A pointer 40 that moves on the screen as the mouse 12 (FIG. 9) moves is displayed above. The alignment of the mark points is performed in a preset order in the direction starting from the G point and gradually toward the end of the skeleton. Next, the alignment is performed on the skeleton image 39 displayed in the window W2. The end point that should be included is blinking to notify this. Then, the pointer 40 is moved to the position to be specified on the model image 38 on the window W1 and the click button of the mouse 12 is operated, so that the end points blinked on the window W2 are superimposed on the model image 38. The position is determined, and the length of each skeletal unit is sequentially calculated based on the position of the end point already determined (C202 to C209). Further, the skeletal unit having a predetermined length is sequentially drawn on the model image 38 of the window W1.
[0062]
Here, since the model image 38 in the facing pose is substantially bilaterally symmetric, each of the end points of the humerus 24 and the lower humerus 25 and the upper limb bone 26 and the lower limb bone 27 (FIG. 2) is either left or right. If is positioned, processing is performed so that the other is automatically positioned. In addition, the dimension of the head 20 (distance between H and f1, FIG. 13) compares the standard dimension of the skeleton unit other than the head 20 with the calculated dimension, and the ratio of the dimensions and the standard dimension of the head 20 (C210). Alternatively, a process may be performed in which a skeleton image having a standard size set in advance in the window W1 is displayed together with the model image 38, and the end points of each skeleton unit of the skeleton image are aligned with the model image 38 on the window W1. In this case, the window W2 can be omitted. Next, returning to FIG. 28, the skeleton image 39 of the facing pose is redrawn in the window W1 in accordance with the calculated size of the skeleton unit (C105), and the process proceeds to S1105 in FIG. Here, for the skeleton image 39 corresponding to the facing pose, the driving position of each actuator is set in advance as reference data, and the setting content is stored in the reference data storage unit created in the program storage device 9. Stored.
[0063]
Subsequently, in S1105 to S1109, by specifying the frame number of the moving image for aligning the skeleton image 39, the model image data of the frame is captured, and the process proceeds to the skeleton image alignment process (skeleton image input). . 30 and 31 are flowcharts showing details of the contents of the alignment processing (S1109). That is, as indicated by C301 to C305 in FIG. 30, the position of each end point of the skeleton image 39 is aligned with the model image 38 in the direction starting from the point G and gradually moving away from the captured frame model image 38. . An example of the input is shown in FIG. 36, but the procedure is almost the same as in the case of the dimension adjustment process described above, while the alignment position in the window W1 of the end point blinking displayed on the window W2 is designated with the mouse 12. Then, alignment input for the model image 38 is performed. Also in this case, the skeleton image 39 is drawn over the model image 38 on the window W1 in order from the portion where the alignment is completed. Here, the left and right leg portions and arm portions are individually subjected to input processing, unlike the above-described dimension matching processing. Note that the alignment input is skipped for the teaching unnecessary portion set in S1101 (FIG. 27).
[0064]
FIG. 31 shows an example of the flow of the alignment process of the parts corresponding to both legs, and the alignment of the upper end point L1 (L2) of the upper limb bone 26 is performed for each of the right leg and the left leg. Thereafter, the lower end point Lf1 (Lf2) of the lower limb bone 27 is aligned (C401 to C403). Here, if L1 (L2) and Lf1 (Lf2) are specified according to the constraint condition shown in FIG. 10, the position of Lk1 (Lk2) corresponding to the knee is automatically determined, and no input is performed. In this way, the coordinates on the screen of each end point are calculated, and the three-dimensional coordinates of the end points in the actual space are calculated based on the screen coordinates and the standard body size data of each skeleton unit stored in advance. .
[0065]
Next, returning to FIG. 27 and proceeding to S1110, auxiliary input processing of those spatial positions is performed on the skeleton units or parts for which it is difficult or impossible to generate drive data by the alignment processing, and the mouse 12 or keyboard 13 is used. Do by using. In this input process, a window different from the alignment process is opened, an image of the corresponding skeleton unit or region is displayed on the window, and various processing commands (for example, skeleton unit) prepared in advance for position setting are displayed. Are input individually for each skeletal unit or region, while selecting and executing the translation, rotation, etc.) by input from the mouse 12 or the keyboard 13.
[0066]
This completes the input process for one frame, but when the process proceeds to the next frame, the frame number is updated, and the skeleton image 39 and model image 38 of the immediately preceding frame are redrawn (S1111 to S1113). , Return to S1107 and repeat the process. On the other hand, when the input operation to the frame is finished (S1114 to S1116), the windows W1 and W2 are closed, and the process proceeds to S1200 in FIG.
[0067]
Here, when the displacement data of each end point obtained as described above is used as drive data as it is, the process proceeds to S1300. However, when the displacement data is used to calculate the drive amount of each actuator, The process of S1200, that is, the process of calculating the drive amount of the actuator that drives the skeleton unit is inserted based on the spatial position coordinates of the end points of the skeleton unit for each frame obtained by the above input. That is, as shown in FIG. 32, the number of two adjacent frames is designated, and the spatial position coordinates of the end points of each skeleton unit are read, and provided between the skeleton units based on the spatial position coordinates. The driving amount of the actuator is calculated (C501 to C503). The content of the calculation process is almost the same as the process of calculating the drive amount of each actuator by the drive amount calculation program 216a described above, and detailed description thereof is omitted. If the skeleton image of the first frame is not a facing pose, the driving position of each actuator relative to the first frame may be calculated based on the actuator reference position stored in advance.
[0068]
Next, the process proceeds to S1300 in FIG. 26, and the time on the real time axis of the robot operation is set for each frame for which input has been completed. Details of the processing are shown in the flowchart of FIG. That is, the time setting window is opened, the skeleton image data is read in the frame number order, and the time is input while the skeleton images are sequentially displayed in the window (C601 to C605). In C6051, the drive speed of each actuator is calculated from the time interval with the immediately preceding frame, and in C606, it is determined whether or not all actuators can be driven at that time interval. If it can be completed, the input time and actuator drive speed are stored in C607, the frame number is incremented in C608, the process returns to C603, and the process is repeated. If the driving cannot be completed, the process returns to C605 and the time is input again. When the time input process ends, the window is closed in C612 and the process proceeds to S1400 in FIG.
[0069]
In S1400, the finally obtained drive data is stored / saved in the drive data storage device 10. The processing flow is as shown in FIG. 34. First, a middle-level operation command name is input in C701. Then, the drive data is divided according to the part (C702), and a sub-file of lower-order motion data, that is, a part motion module is formed (C702). Then, these part motion modules are integrated with the inputted middle motion command name and stored in the storage device 10 as a middle motion routine (C703). Each part operation module can be exchanged or replaced between different middle operation routines.
[0070]
As described above, the part motion module of each part is taught and created by the drive data creation device 1, but the robot part that is difficult or impossible to calculate drive data from the skeleton image aligned with the model image, that is, There is a non-teaching site. Examples are the wrist, fingers, eyes, mouth, tongue, and the like. In this case, in the operation program, such a non-teaching part positioning instruction routine part can be incorporated as one part of a definition routine, for example. In this example,
set (part name) <setting factor: = setting condition value>
The positioning command routine part is described in the format.
[0071]
Specifically, by selecting the movement start position, movement end position, and movement speed of the corresponding non-teaching part as setting factors and entering numerical values or selecting from preset candidate values, the desired setting condition value Set. In response to this, based on the non-teaching part drive data generation program stored in the storage unit 308 of FIG. 5, the CPU 204 of FIG. For example, when the actuator is a servo motor or a stepping motor, the rotation start position and the rotation end position of the motor are calculated based on the values of the movement start position and the movement end position, and the motor rotation is received based on the movement speed value. Calculate the number. Hereinafter, some examples of the non-teaching part used in the robot of the present embodiment, its operation type, the format of the positioning command routine part, etc. will be shown.
[0072]
Part name: right eye (left eye) (part designation information: RIGHTEYE (LEFTEYE)). Actuator: Driven to the left and right by a motor.
Format: RIGHTEYE <do: = execution command> <til: = conditional expression>
LEFTEYE <do: = execution command> <til: = conditional expression>

Operation end condition phrase: When the execution command is goggles, it is possible to specify the number of eye reciprocations. (Example: RIGHTEYE <do: = goggle> <til: = repeatΔ>)
Positioning command: set EYE <speed: = (speed designation information)> <right: = (right eye position coordinate)> <left: = (left eye position coordinate)> (A part name is common to the left and right eyes, for example. (It may be set separately for left and right)
speed: indicates the speed of eye movement
right: indicates the right eye position
left: indicates the left eye position
・ "Right eye" moves to the position set by positioning command "right"
・ "Left eye" moves to the position set by positioning command left
“Kyo Kyoro” designates the number of round trips between coordinates set by right and left of the positioning command (<til: = repeat n>). When the number of round trips is not specified, only one round trip is possible. Then, the final eye position stops at the left set position.
First, move your eyes to the right.
[0073]
Site name: starboard (left side) (site designation information: RIGHTEYELID (LEFTEYELID)
Actuator: Driven up and down by a motor.
Format: RIGHTEYELID <do: = execution command> <til: = conditional expression>
LEFTEYELID <do: = execution command> <til: = conditional expression>

Operation end condition phrase When the execution command is blink, the number of blinks can be specified. (Example: RIGHTEYELID <do: = blink> <til: = repeat n>)
Positioning command set EYELID <open: = (time setting information)> (common to left and right eyelids)
open: Specify the time to open the casket
[0074]
Part name: mouth (part designation information: MOUTH)
Actuator: Driven by a motor.
Format: MOUTH <do: = execution command> <til: = conditional expression>

Operation end condition phrase When the execution command is parrot, the number of times can be specified. (Example: MOUTH <do: = parrot> <til: = repeat n>)
Positioning command set MOUTH <open: = (time setting information)> <close: = (time setting information)>
open: Indicates the time when the mouth is open
close: Indicates the time when the mouth is closed
[0075]
Part name: tongue (part specifying information: TONGUS)
Actuator: Driven in and out by a motor.
Format: TONGUS <do: = execution command> <til: = conditional expression>

Operation end condition phrase When the execution command is “lick”, the number of operations can be specified.
(Example: TONGUS <do: = lick> <til: = repeat n>) Positioning command set TONGUS <out: = (time setting information)> <in: = (time setting information)>
out: Indicates the time when the tongue is sticking out
in: Indicates the time when the tongue is put
[0076]
Part name: hand (part specifying information: RIGHTFINGER (LEFTFINGER))
Actuator: Each finger is independently bent and stretched by a cylinder.

Execution command Finger pattern instruction (by selecting any of 0 to 32, various finger bending states are set. For example, 0: par / 1: thumb song / 2: index finger song / 4: middle finger song / 8: ring finger (Song / 16: Pinkie / 31: Goo)
Positioning command None
[Brief description of the drawings]
FIG. 1 is a block diagram showing the overall configuration of a robot system according to the present invention.
FIG. 2 is a schematic diagram of a robot skeleton.
FIG. 3 is a diagram illustrating the operation of the mouth and tongue provided on the robot head.
FIG. 4 is an explanatory diagram of eye and eyelid movements.
FIG. 5 is an explanatory diagram showing the contents of a storage device of a robot control unit.
FIG. 6 is an explanatory diagram showing the contents of the part motion module storage unit.
FIG. 7 is an explanatory diagram showing the contents of a RAM of the robot control unit.
FIG. 8 is a block diagram showing a configuration of a calculation / drive control unit.
FIG. 9 is a block diagram showing a configuration example of a drive data creation device.
FIG. 10 is an explanatory diagram of a constraint condition for a leg skeleton.
FIG. 11 is a flowchart showing a flow of processing by a higher-level operation program.
FIG. 12 is a flowchart showing the flow of processing of a middle-level operation routine.
FIG. 13 is an explanatory diagram of a skeleton image size matching process for a directly facing model.
FIG. 14 is an explanatory diagram of a method for calculating the drive amount of an actuator.
FIG. 15 is a diagram for explaining a flow of robot operation in association with a program hierarchy;
FIG. 16 is an explanatory diagram showing an example of basic motion related to walking.
FIG. 17 is an explanatory diagram following FIG. 16;
FIG. 18 is an explanatory diagram showing a description example of a higher-level operation program.
FIG. 19 is an explanatory diagram showing an example of a definition routine of the standard change type intermediate operation routine.
FIG. 20 is an explanatory diagram showing a description example of an operation command statement when deleting or adding a part operation module.
FIG. 21 is an explanatory diagram showing a description example of a higher-level operation program using a superposition type intermediate operation routine.
FIG. 22 is an explanatory diagram showing a description example of a definition routine of the superposition type intermediate operation routine.
FIG. 23 is an explanatory diagram showing a robot operation example suitable for application of the combination type intermediate operation routine.
24 is a diagram showing an example of a timing chart of the operation of FIG.
FIG. 25 is an explanatory diagram showing a description example of a definition routine of a combination-type intermediate operation routine used for it.
FIG. 26 is a flowchart showing the overall processing flow of the drive data creation device.
FIG. 27 is a flowchart of the input process.
FIG. 28 is a flowchart showing a flow of processing for determining a skeleton unit length in the input processing.
FIG. 29 is a flowchart showing the flow of processing for matching the length of each skeleton unit to the corresponding part of the model image.
FIG. 30 is a flowchart showing the end point alignment processing flow of each skeleton unit.
FIG. 31 is a flowchart showing the flow of processing for calculating the coordinates of the end points of a skeleton.
FIG. 32 is a flowchart showing the flow of driving data calculation in the same manner.
FIG. 33 is a flowchart showing the flow of time input processing in the same manner.
FIG. 34 is a flowchart showing the flow of drive data storage processing.
FIG. 35 is a schematic diagram illustrating an example of an arrangement of frames on a time axis.
FIG. 36 is an explanatory diagram of a positioning process for a model image of a skeleton image.
[Explanation of symbols]
1 Drive data creation device
5 Device controller (drive data calculation means)
200 Robot system
201 Robot
202 Robot controller
204 CPU (program reading means, program expanding means, non-teaching part drive data generating means)
205 RAM
205a Host operation program storage unit
205b Middle operation command analysis program storage unit
205c Middle operation routine storage section (I)
205d Middle operation routine storage section (II)
205g Intermediate operation routine expansion memory area
207 storage device
212 Calculation / drive control unit
213 Actuator
300 Host operation program storage unit
302 Middle operation routine storage unit
304 Site action module storage unit
306 Middle operation command analysis program storage unit

Claims (6)

A part action program module (hereinafter referred to as a part action module) that grasps the upper robot action as a combination of several basic actions and defines the action of each robot part (hereinafter also referred to simply as part) necessary for the basic action. There made form a median operation routine is formed to define the basic operation by combining a plurality of the site operation module, higher operating program which defines the robot operation of the upper by a combination of the middle position operation routine of that is floor It has been describe as a layer type program,
The intermediate operation routine is associated with the intermediate operation command , the upper operation program defines the upper robot operation by a combination of the intermediate operation commands ,
A middle level operation routine storage unit for storing the middle level operation routine;
An upper operation program storage unit for storing the upper operation program which is a combination of the intermediate operation commands ;
The upper operation program stored in the upper operation program storage unit, and a program reading means for reading a unit of said intermediate operation command,
Based on a combination of the median operation routine corresponding to the position operation command in read by the program readout means,
The part motion module is formed in units of drive data related to driving of an actuator provided in the robot part, and an actuator operation control unit for controlling the drive of the actuator according to the drive data, and the operation of the corresponding robot part It consists of an operation condition specification command including an operation start timing specification command that specifies the start timing,
The actuator operation control unit in accordance with the operating conditions prescribed by the operating conditions specified command controls the driving of the actuator according to the basic operation or site operation,
The motion start timing regulation command included in the motion condition regulation command of the part motion module is a command for the robot site defined by a series of model images of each frame in which the motion of the model corresponding to the robot motion is recorded over time. A robot system characterized in that the operation start timing is given based on either the start time, end time or specific frame position passage time of the operation of the robot part when using the operation as a reference . When the operation start timing defining command according to claim 1 is not given in the operation condition defining command of the part operation module constituting the intermediate operation routine , the corresponding basic action or part action The robot system according to claim 1, wherein the operation start timing is set to an execution start time of the intermediate operation routine .   The robot system according to claim 1, wherein the operation start timing defining command gives the operation start timing based on an elapsed time from a predetermined time origin. To at least one of said plurality of robot sites, it is intended to comprise the backbone with said actuator disposed between the backbone units and their backbone units of multiple,
Among the plurality of sites operating modules, at least part of what is based on a series of model images recorded over time the operation of the model corresponding to the robot operation, the drive created by the frame feed unit of the robot operation The teaching operation routine part created by the drive data creation device that edits the data for each part of the robot is incorporated,
The drive data creation device is
Image display means for displaying the model image and a skeleton image composed of a plurality of skeleton units;
Skeleton positioning means for aligning the skeleton image with the model image in the image display means;
Data calculation means for calculating the drive data based on the displacement of the skeleton image respectively aligned with the model images that are mutually before and after on the time axis;
The robot system according to claim 1 , comprising: When the intermediate motion routine includes a part motion module of a robot part (hereinafter referred to as a non-teaching part) for which the teaching operation routine part is not created by the drive data creation device, it corresponds to the part motion module. The non-teaching part positioning command routine part is incorporated,
5. The robot system according to claim 4 , further comprising non-teaching part drive data generating means for generating drive data for the actuator of the corresponding non-teaching part based on the contents of the positioning command routine part.   The robot system according to claim 5, wherein the positioning command routine part includes positioning information for designating at least one of a movement start position, a movement end position, and a movement speed of a corresponding non-teaching part.

Images