JP2003266349A - Position recognition method, device therefor, program therefor, recording medium therefor, and robot device equipped with position recognition device - Google Patents

Abstract

(57) Abstract: An environment recognition method, an apparatus, a program, a recording medium, and a method for reducing the cost of maintaining and updating an environment map in consideration of limited computational resources and memory resources of a robot apparatus and the like. Provide a robot device equipped with an environment recognition device. An environment map includes a plurality of grids.
1 and a plurality of grids 402 that the robot apparatus 403 recognizes as obstacles.
It is map information of the surroundings centering on 3. Robot device 4
When 03 moves, the difference movement amount is input. When the difference movement amount is larger than the size of the grid 401, the map information is updated. Also, the robot device 403 is obtained from the posture information indicating the direction in which the robot device faces from a predetermined direction.
Is moved, the difference movement angle is input and the posture information is updated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a position recognition method for recognizing a surrounding environment and avoiding obstacles, an apparatus therefor, a program therefor, a recording medium therefor, and a robot apparatus equipped with a position recognition apparatus.

[0002]

2. Description of the Related Art A mechanical device that makes a movement similar to that of a human being (organism) using electric or magnetic action is called a "robot". Robots started to spread in Japan from the end of the 1960s, but most of them are industrial robots such as manipulators and transfer robots (Industrial Robots) for the purpose of automating and unmanning production work in factories.
rialRobot).

Recently, practical robots have been developed to support life as a human partner, that is, to support human activities in various situations such as living environment and other daily life. Unlike an industrial robot, such a practical robot is used in various aspects of the human living environment.
It has the ability to self-learn how to adapt to individuals with different personalities or various environments. For example, it is designed by using a “pet type” robot that imitates the body mechanism and movement of a quadruped animal such as a dog or a cat, or a body mechanism and movement of a human walking two legs upright. Robot devices such as “humanoid” or “humanoid” robots (Humanoid Robots) are already in practical use.

These robot devices are sometimes called entertainment robots because they can perform various operations, etc., with emphasis on entertainment, as compared with industrial robots. Further, some of such robot devices operate autonomously in accordance with information from the outside and internal states.

By the way, in the autonomous robot apparatus,
Needless to say, the ability to recognize the environment around oneself, plan a route, and move accordingly. Conventionally,
In the mobile robot device, a map of the entire environment is held, or the environment map is updated when the robot device is slightly moved or the posture is changed.

[0006]

However, in the conventional robot apparatus, the amount of data to be stored increases and the map update cost and the memory copy cost also increase, which is not suitable for a robot apparatus having limited resources. There is a problem.

The present invention has been proposed in view of such a conventional situation, and in view of the limited calculation resources and memory resources of a robot device or the like, the cost for holding and updating the environment map has been reduced. An object of the present invention is to provide a position recognition method, its device, its program, its recording medium, and a position recognition device-mounted robot device.

[0008]

In order to achieve the above-mentioned object, a position recognition method according to the present invention is arranged such that the position of a moving body moving in a predetermined environment is displayed on an environment map corresponding to the predetermined environment. A position recognition method for recognizing as a position, wherein the environment map is partitioned by a grid of a predetermined size,
With the movement of the moving body, when the movement range is within the grid, the environment map is retained, and when the movement range exceeds the grid, the environment map is moved by the grid unit and updated. To do.

In the present invention, since the environment map is divided into grids and the environment map is moved by the grid unit to update the environment map only when the movement range exceeds the size of the grid, the data to be held is retained. When the amount of movement can be reduced and the movement range does not exceed the grid, it is not necessary to update the environment map, and the calculation cost and the information copying cost to the memory can be reduced.

Further, since the direction of the moving body is recognized as the direction on the environment map by fixing the environment map, information on the direction of the moving body (posture information) is stored separately from the environment map, and the moving body moves. If the orientation is not changed due to, there is no need to update the posture information.

Further, since there is a step of changing the size of the environment map and / or the grid, for example, when the accuracy of the environment map is not required so much, the apparatus is stopped even during the operation of the apparatus. Without making the map smaller or increasing the size (resolution) of the grid, the calculation cost and the memory copy cost at the time of updating the environment map can be further reduced.

Furthermore, since the difference movement amount and the difference movement angle before and after the movement are input as the moving body moves, it is possible to obtain an environment map and posture information with a small amount of calculation and high accuracy.

A position recognition device according to the present invention is a position recognition device for recognizing a position of a moving body moving in a predetermined environment as a position on an environment map corresponding to the predetermined environment. It has storage means for storing and map updating means for updating the environment map with the movement of the mobile body, the environment map being partitioned by a grid of a predetermined size, and the map updating means for moving the movement. When the movement range is within the grid due to the movement of the body, the environment map is retained, and when the movement range exceeds the grid, the environment map is moved by the grid unit and updated. And

A robot device according to the present invention is a robot device which operates based on supplied input information,
A position recognizing device for recognizing the position of a moving body moving in a predetermined environment as a position on the environment map corresponding to the predetermined environment is mounted, and the position recognizing device includes a storage unit for storing the environment map, With the movement of the mobile body, it has a map update means for updating the environment map, the environment map is partitioned by a grid of a predetermined size, the map update means, with the movement of the mobile body, When the movement range is within the grid, the environment map is retained, and when the movement range exceeds the grid, the environment map is moved and updated in units of the grid.

Further, since the operation panel has output means for outputting the map information and the attitude information, for example, when the user remotely operates, the operation panel is used to display the environment map and the attitude information. This makes it possible to easily grasp information about obstacles around the moving body, that is, the robot device, and improve operability.

Further, the map updating means has a movable leg unit, and the movement amount is input each time the movable leg unit moves, or the map updating means has a movable leg unit composed of two left and right leg units. The map updating means inputs the movement amount each time any one of the leg units moves, so that the movement amount can be easily calculated from the number of revolutions of a robot device having wheels, for example. ,Also,
With a two- or four-legged robot device, the amount of movement can be easily calculated from the number of steps, and when it is not moving, updating work is unnecessary, so useless calculation when updating the environment map. The cost and the memory copy cost can be reduced.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Specific embodiments to which the present invention is applied will be described below in detail with reference to the drawings. This embodiment is applied to a bipedal robot device equipped with a position recognition device.

If the robot unit wants to move freely in the environment, it is necessary to create robot center map information (environment map) centered on the robot unit (origin). , Need to be updated from time to time. In a device having limited resources such as a robot device, it is preferable to make the memory as small as possible when updating the map. Therefore, in the present embodiment, the environment map is divided into grids of a predetermined size, and when the movement is small (when the movement range is within the grid), the environment map is not updated and memory copy is unnecessary. It is what The differential movement amount (movement distance) before and after the movement of the robot device can be easily calculated from the rotation of the wheel in the case of a robot device having wheels, for example. Further, as in the present embodiment, in a bipedal robot device, the moving distance can be easily calculated from the number of steps.

FIG. 1 is a block diagram showing an outline of a robot apparatus according to an embodiment of the present invention. As shown in FIG. 1, the head unit 250 of the robot apparatus 1 is provided with two CCD cameras 200R and 200L.
A stereo image processing device 210 is provided behind the CD cameras 200R and 200L. The right eye image 201R and the left eye image 201L captured by two CCD cameras (hereinafter, referred to as right eye 200R and left eye 200L) are
It is input to the stereo image processing device 210. The stereo image processing apparatus 210 calculates disparity information (disparity data) (distance information) of each of the images 201R and 201L, and a color image (YUV: luminance Y, UV color difference) 202 and a parallax image (YDR: luminance Y, parallax D). , Reliability R) 203 is calculated alternately for each frame. Here, the parallax indicates the difference between the points at which a certain point in space is mapped to the left eye and the right eye,
It changes according to the distance from the camera.

This color image 202 and parallax image 203
Is a CPU built in the trunk 260 of the robot apparatus 1.
(Control unit) 220 is input. Also, the robot device 1
An actuator 230 is provided at each joint of
A control signal 231 serving as a command from the CPU 220 is supplied, and the motor is driven according to the command value. A potentiometer is attached to each joint (actuator), and the rotation angle of the motor at that time is sent to the CPU. Potentiometer attached to this actuator,
Each sensor 240 such as a touch sensor attached to the sole of the foot and a gyro sensor attached to the torso measures the current state of the robot device such as the current joint angle, installation information, and posture information, and the sensor data CP as 241
Output to U220. The CPU 220 includes the color image 202 and the parallax image 20 from the stereo image processing device 210.
3 and the sensor data 241 such as all joint angles of the actuator are input to realize the software configuration described later.

The software of the present embodiment is configured in units of objects, recognizes the position, movement amount, obstacles around the robot apparatus, environment map, etc. of the robot apparatus, and takes action about the action that the robot apparatus should finally take. It is for performing various recognition processes for outputting a column. As the coordinates indicating the position of the robot device, for example, a camera coordinate system of a world reference system (hereinafter also referred to as absolute coordinates) having a predetermined position based on a specific object such as a landmark described later as an origin of the coordinates. , And a robot center coordinate system (hereinafter, also referred to as relative coordinates) centered on the robot apparatus itself (origin of coordinates).

FIG. 2 is a schematic diagram showing a software configuration of the robot apparatus according to this embodiment. In the figure, the circles represent entities called objects or processes. The entire system operates by communicating objects asynchronously.
Each object transfers data and activates a program (Invoke) by an inter-object communication method using message communication and shared memory.

As shown in FIG. 2, software 300
Kinematics Odometr
y) KINE310, Plane Extractor P
LEX320, Obstacle grid calculator (Occupancy Gri
d) OG330, landmark position detector (Landmark Se
nsor) CLS340, absolute coordinate calculation unit (Localization)
LZ350 and Situated behavior Laye
r) It is composed of the SBL 360, and processing is performed for each object. The detailed configuration and operation of these software will be described later. The detailed configuration and operation of the robot device will also be described later.

First, the position recognizing device according to the embodiment of the present invention mounted on the robot device 1 will be described. The position recognition device according to the present embodiment has an obstacle grid calculation unit (Occupancy Grid) OG330. That is, the position recognition device of the present embodiment has the same configuration and operation as the obstacle grid calculation unit OG330 of the robot device 1.

As described above, the CCD cameras 200L, 2
The video captured by 00R is input to the stereo image processing device 210, and the color image (YUV) 202 is obtained from the parallax information (distance information) of the left and right images 201R and 201L.
And the parallax image (YDR) 203 is calculated and the CPU 22
Input to 0. In addition, sensor data 240 from each of the plurality of sensors provided in the robot apparatus 1 is input.
As shown in FIG. 2, the image data 301 including the parallax information and the parallax image and the sensor data 302 are input to the kinematic odometry KINE 310.

Kinematic odometry KINE310
Calculates the movement amount (odometry) of the robot central coordinate system based on the input data composed of the image data 301 and the sensor data 302. The odometry 301 is input to the absolute coordinate calculation unit LZ350 and the obstacle grid calculation unit OG330.

FIG. 3 is a schematic diagram showing an environment map recognized by the position recognition device (robot device) of the present embodiment. As shown in FIG. 3, the environment map 400 is an environment map (map information in the robot center coordinate system) that is surrounding map information centered around the robot device (moving body) 403.
It is divided into a grid 410 of a predetermined size. Then, when the robot device 403 moves, the robot device 403 is constantly updated so as to be positioned within the central grid 401 located at the center of the environment map 400. The sizes of the environment map 400 and the grid 410 shown in FIG. 3 can be appropriately changed as needed. That is,
For example, when a fine environment map is unnecessary, the environment map 400 may be made smaller, or the grid 410 may be made larger to reduce the resolution, thereby reducing the amount of calculation and the memory copy cost. it can.

Further, in FIG. 3, a hatched grid 402 shows a minimum unit area (obstacle occupied area) recognized as an obstacle. As a method of recognizing an obstacle in the position recognizing device, for example, a method of obtaining from a parallax image by a stereo camera, a method of using rebound of radio waves such as a range finder, and a method of using a distance measuring sensor such as an ultrasonic sensor . Then, the position recognition device recognizes (displays) the position of the obstacle recognized by these methods as an attribute of the grid of the environment map. In the present embodiment, the obstacle information is input from the plane extraction unit PLEX320 described later.

Position recognition device (obstacle grid calculation unit OG
330) is the kinematic odometry KINE3 to which the data image and the sensor data are input, as described above.
When the odometry 314 is input from 10, the environment map previously stored in the memory (storage unit) is updated.
In the memory, together with the environment map 400, which is map information having information about obstacles around the robot device centering on the robot device 403, and posture information indicating the orientation of the robot device (moving body) on this map. 404. The posture information 404 indicates a direction in which the robot device 403 faces from a predetermined direction, for example, the x-axis direction or the y-axis direction. Further, it has an updating means for updating the previously recognized environment map and the information on the posture direction shown in FIG. 3 according to the change and movement of the posture of the robot apparatus. The updating means retains the environment map when the moving range is within the grid due to the movement of the moving body such as the robot device, and moves and updates the environment map in grid units when the moving range exceeds the grid. is there.

First, a method for updating the environment map will be described. In the position recognition device of the present invention, as described above, the environment map is not updated depending on the amount of movement of the robot device or the like.

FIGS. 4 (a), (b), (c), and (d) are diagrams showing a method of updating the environment map according to the movement amount.
It is a schematic diagram which shows an environment map when not updating an environment map and when updating an environment map, respectively. As shown in FIG. 4A, the robot device 503 is located in the center grid 501 in the environment map 500. A hatched grid 502 indicates an obstacle area. When the movement amount 505 of the robot device 503 is small and the position of the robot device 503 is in the center grid 501 even in the robot device 506 after the movement, as shown in FIG. 4B. , The environment map 500 is not updated. In this case, only the movement position information of the robot device in the central grid 501 is updated.

On the other hand, similarly to FIG. 4A, as shown in FIG. 4C, the central grid 50 in the environment map 500 is shown.
When the movement amount 515 of the robot device 503 located in the position 1 greatly exceeds the central grid 501, the robot device 513 after the movement is positioned on the central grid 511 of the environment map as shown in FIG. 4D. , The environment map 500 is shifted and updated by the number of moved grids to obtain the updated environment map 510.

That is, the size of the grid, which is the minimum unit of the size recognized as the obstacle area, is CS (Cell Siz
e), the position of the robot device in the grid is (Bx, B
y), the movement amount of the robot device in the two-dimensional direction is (dx, d
y), the magnitude of the grid shift (Sx, S
y) is represented by the following mathematical expression (1).

[0034]

[Equation 1]

Here, in the above equation (1), []
Is a Gaussian symbol and represents the maximum integer value that does not exceed the value in []. The robot position (Rx, Ry) in the grid is represented by the following mathematical expression (2).

[0036]

[Equation 2]

The values of the above equations (1) and (2) are shown in FIG.
Shown in. As shown in FIG. 5A, the robot device 603
Are located in the central grid 601 in the environment map 600 divided into a plurality of grids having a grid size CS. The environment map 600 has a plurality of grids 602 that indicate obstacle areas shown by hatching. Here, for example, a point 607 at the lower left corner in the drawing in the central grid 601 can be set as the position origin of the robot device 603 in the grid 601. This robot device 603
Is the position (Bx, B) as the distance from the origin 607.
can be represented as y). This robot device 603
Moves the moving amount (dx, dy) 605 and moves to the grid 6
When moved to 08, the position of the robot apparatus 613 after the movement can be represented as (Bx + dx, By + dy).

The robot device 603 moves the moving amount (dx, d
y) When moving 605, the environmental map shift amount (S
x, Sy) can be calculated from the above equation (1). Here, in the updated environment map 610, the environment map 6
00 is updated by shifting the environment map by the shift amount Sx in the x-axis direction and the shift amount Sy in the y-axis direction, which are shifted in the opposite directions to the shift amount dx in the x-axis direction and the shift amount dy in the y-axis direction, respectively. be able to. In addition, here, the movement amount (dx, dy) of the robot device is positive in the right direction, and the shift amount (Sx, Sy) of the environment map is positive in the left direction which is the opposite direction to the movement direction.

In this way, the robot apparatus 613 after moving is
It is located on the central grid 611 of the updated environment map 610.
Here, the robot position (Ax, Ay) in the center grid 611 of the updated environment map 610 after shifting the environment map of the robot apparatus 613 after movement is the center grid 601 of the pre-shift environment map 600 of the robot apparatus 603 before movement.
From the robot position (Bx, By) and the in-grid movement amount (Rx, Ry) 615, the robot position (Ax, By)
Ay) = (Bx + Rx, By + Ry) = (Bx + dx−
It can be calculated as CS × Sx, By + dy−CS × Sy).

Next, updating of information on the posture direction of the robot device will be described. FIGS. 6A and 6B are schematic views showing the states before and after updating the posture direction. Regarding the posture information of the robot device, the environment map can be shown as an angle with respect to the fixed coordinate system, that is, when the robot device changes only the posture direction dα, only the information of the posture direction of the robot device is updated to change the environment. The map is not updated.

That is, as shown in FIG. 6A, the environment map 70 having a plurality of grids 702 which are obstacle regions.
In the robot apparatus 703 located on the central grid 701 at 0, the robot posture direction 704 before the movement is α
And The posture direction α can be shown as a tilt from the y-axis or a tilt from the x-axis, with the clockwise direction being positive, for example. Here, when the robot device 703 rotates at the posture rotation angle dα, the robot posture direction (posture information) 714 after movement can be represented as α + dα, as shown in FIG. 6B.

FIG. 7 is a diagram showing the effect of the present embodiment. In this embodiment, as shown in FIG. 7, the environment map and the information on the posture direction of the robot apparatus itself are updated, that is, the movement amount (odometry) 802 from the kinematic odometry KINE 801 to the obstacle grid calculation unit 803.
The delivery can be performed automatically every predetermined time, but is performed every time the robot apparatus walks one step. As a result, it is possible to reduce the load of the robot device, for example, the CPU, by not performing the update work when not moving and by eliminating the calculation cost and the memory copy cost due to the update work. Also, if the software configuration is such that processing is performed on an object-by-object basis, the data to be transferred is
Only the rotation difference amount (dx, dy, dα) is required, and the data amount is smaller than that when the absolute coordinate and the absolute angle of the robot device are transferred, and high accuracy can be obtained.

Further, when the entertainment robot apparatus performs processing other than position recognition, there are cases where the priority of position recognition may be lowered because it is not directly related to position recognition, such as voice recognition and face recognition. In such a case, if the position recognition device (object of position recognition) is stopped, when the priority of position recognition becomes high,
There is a problem that the position recognition result before the stop cannot be matched due to movement during the stop (Kid
Nap problem) occurs. Therefore, when the priority of the position recognition may be lowered, the position recognition object is stopped by increasing the size CS of the environment map grid or by decreasing the size of the environment map size itself. It is possible to reduce the CPU resource without doing so.

Furthermore, the environment map and the posture information of the robot device obtained by the above-described processing can be displayed on the operation panel shown in FIG. FIG. 8 is a schematic diagram showing an operation panel used by a user for remote control of the robot apparatus, for example. As shown in FIG. 8, a camera image 901 of the robot apparatus and an environment map 910 are displayed on the operation panel 900. In addition, the operation panel 900 is provided with, for example, operation buttons 920 for moving the robot apparatus 913. In the figure, the camera image 901 schematically shows the image traced.

As an important usage method of the mobile robot apparatus, there is a usage method in which the user remotely operates in addition to the autonomous operation. 2. Description of the Related Art Conventionally, there is one that displays an image obtained by a camera mounted on a robot device on a control panel of the robot. However, in particular, it is difficult for the user to operate the robot apparatus by grasping the environment around the robot apparatus only with a camera having a limited field of view. However, as in the present embodiment, the CCD installed in the robot device 913
By displaying the environment map 910 on the operation panel 900 for remote control, the operability of robot control is dramatically improved, as compared with the conventional robot control in which remote control is performed using only the camera image 901 from the camera. .

Here, in the above-mentioned embodiment, it is explained that the environment map is not updated even if the posture information of the robot changes, but normally, in the robot apparatus, C is used.
An imaging device such as a CD camera is provided on the front of the head unit, and a camera image 901 displays an image in front of the robot device. Therefore, when the orientation information changes and the orientation of the robot device is rotated by 180 °, for example, the directions of the camera image and the environment map are reversed. Therefore, when the posture information is rotated by a predetermined angle such as 90 ° or more, the environment map is rotated in accordance with this rotation and the traveling direction is displayed so as to be substantially the upward direction in the figure, thereby further controlling the robot. Operability is improved.

Next, the robot device 1 shown in FIG. 2 described above.
The configuration and operation of the software will be described in detail. FIG. 9 is a flowchart showing the operation of the software 300 shown in FIG.

As described above, the image data 301 and the sensor data 302 are input to the kinematic odometry KINE 310 of the software 300 shown in FIG. This image data 301 is a color image and a parallax image from a stereo camera. The sensor data is data such as the joint angle of the robot device. The kinematic odometry KINE 310 receives these input data 301 and 302, and updates the image and sensor data stored so far in the memory (step S101).

Next, the image data 301 and the sensor data 302 are temporally correlated (step S102).
-1). That is, the joint angle of the sensor data 302 at the time when the image of the image data 301 is captured is calculated. Next, using this joint angle data, the robot central coordinate system in which the robot apparatus 1 is fixed to the center is converted into the coordinate system of the camera provided in the head unit (step S10).
2-2). In this case, in the present embodiment, the simultaneous conversion matrix and the like of the camera coordinate system are derived from the robot center coordinate system, and this simultaneous conversion matrix 311 and the image data corresponding thereto are transmitted to the object that performs image recognition. That is,
Simultaneous conversion matrix 311 and corresponding parallax image 312
To the plane extraction unit PLEX320, and the simultaneous conversion matrix 3
11 and the color image 313 in the landmark sensor section CL
Output to S340.

Further, the movement amount of the robot apparatus 1 is calculated from the walking parameters obtained from the sensor data 302 and the number of steps counted using the sole sensor, and the movement amount of the robot apparatus 1 in the robot apparatus central coordinate system is calculated. calculate. Hereinafter, the amount of movement of the robot device central coordinate system is also referred to as odometry. The odometry 314 is output to the obstacle grid calculation unit OG330 and the absolute coordinate calculation unit LZ350.

The plane extraction unit PLEX 320 receives the simultaneous conversion matrix 311 calculated by the kinematic odometry KINE 310 and the parallax image 312 obtained from the stereo camera corresponding thereto, and is stored in the memory until then. Also, these data are updated (step S103). Then, three-dimensional position data (range data) is calculated from the parallax image 312 using the calibration parameters of the stereo camera (step S).
104-1). Then, using the Hough transform or the like, the planes other than the planes such as the wall and the table are extracted as the planes from the range data. Further, by taking a correspondence from the coordinate transformation matrix 311 with a plane where the sole of the robot apparatus 1 is in contact with the ground, selecting a floor surface, a point that is not on the floor surface, for example, a position higher than a predetermined threshold value, etc. Is used as an obstacle to calculate the distance from the floor surface, and the obstacle information (obstacle) 321 is output to the obstacle grid calculation unit 330 (step S1).
04-2).

In the obstacle grid calculation unit OG330, as described above, the kinematic odometry KINE31.
Odometry 314 calculated at 0 and plane extraction unit PL
When the obstacle observation information (obstacle information) 321 calculated by the EX 320 is input, the data stored up to that point stored in the memory is updated (step S105). Then, the obstacle grid holding the probability of whether or not there is an obstacle on the floor is updated by the stochastic method (step S
106).

This obstacle grid calculation unit OG330
For example, information about an obstacle around the robot device 1 around 4 m, that is, the environment map described above, and posture information indicating the direction in which the robot device 1 faces are held, and the environment map is updated by the method described above. By outputting the updated recognition result (obstacle information 331), the upper layer, that is, the route plan determination unit SBL3 in the present embodiment.
At 60, a plan to avoid obstacles can be created.

When the simultaneous conversion matrix 311 and the color image 313 are input from the kinematic odometry KINE 310, the landmark sensor CLS 340 updates these data stored in advance in the memory (step S107). Then, the color image 313 is subjected to image processing to detect a color landmark recognized in advance. The position and size of this color landmark on the color image 313 are converted into the position on the camera coordinate system.
Further, using the simultaneous conversion matrix 311, the position of the color landmark in the camera coordinate system is converted into the position in the robot center position coordinate system, and the information of the color landmark position in the robot center position coordinate system (color landmark relative position The information) 341 is output to the absolute coordinate calculation unit LZ350 (step S108).

The absolute coordinate calculation unit LZ350 uses the odometry 314 from the kinematic odometry KINE310.
And the color landmark relative position information 341 from the landmark sensor unit CLS340 are input, these data stored in advance in the memory are updated (step S109). Then, the absolute coordinate calculation unit LZ3
50 uses the absolute coordinates (position in the world coordinate system) of the color landmarks, the relative position information 341 of the color landmarks, and the odometry 314 that the 50 recognizes in advance, and uses the probabilistic method to determine the absolute coordinates (world coordinate system) of the robot device. Position) is calculated. And this absolute coordinate position 35
1 is output to the route plan determination unit SBL360.

The route plan determining unit SBL360 receives the obstacle grid information 331 from the obstacle grid calculating unit OG330.
Is input and the absolute coordinate position 351 is input from the absolute coordinate calculation unit LZ350, these data stored in advance in the memory are updated (step S111). Then, the recognition result regarding the obstacle existing around the robot apparatus 1 is acquired from the obstacle information 331 from the route plan determination unit SBL360 obstacle grid calculation unit OG330, and the current coordinate of the robot apparatus 1 is acquired from the absolute coordinate calculation unit LZ350. By acquiring the absolute coordinates, a path that can be walked without colliding with an obstacle is generated for the target point given in the absolute coordinate system or the robot center coordinate system centered on the robot device, and the route is set according to the route. Issue an action command to perform. That is, from the input data, the robot device 1 can be used according to the situation.
Determines the action to be taken by the user and outputs the action sequence (step S112).

In the case of human navigation, the obstacle grid calculation unit OG330 shows the recognition result regarding the obstacles existing around the robot apparatus and the absolute coordinate of the current position of the robot apparatus from the absolute coordinate calculation unit LZ350. It is provided to the user and the operation command is issued in response to the input from the user.

FIG. 10 is a diagram schematically showing the flow of data input to the above software. Note that FIG.
2, the same components as those shown in FIGS. 1 and 2 are designated by the same reference numerals, and detailed description thereof will be omitted.

Face detection unit FDT (FaceDetecto)
r) 371 is an object that detects a face area in the image frame, receives the color image 202 from an image input device such as a camera, and reduces and converts the color image 202 into, for example, a nine-step scale image. A rectangular area corresponding to the face is searched from all the images. The face identification unit F outputs the information 372 such as the position, size, and feature amount regarding the area finally determined to be the face by reducing the overlapping candidate areas.
I (Face Identification) 377.

The face identifying section FI377 is an object for identifying the detected face image, receives information 372 consisting of a rectangular area image indicating the area of the face from the face detecting section FDT371, and stores this face image in the memory. The person is identified by comparing which person in the existing person dictionary corresponds to. Then, the ID information 378 of the person is output to the DIL 379 together with the position and size information of the face image area of the face image received from the face detection unit EDT371.

Color recognition unit MCT (MultiColorT)
The racker) 373 is an object for performing color recognition, receives the color image 202 from an image input device such as a camera, extracts a color area based on a plurality of color model information held in advance, and forms a continuous area. To divide.
The color recognition unit MCT 373 provides information 374 such as the position, size, and feature amount of each divided area to the distance information addition unit DIL.
(DistanceInformationLinke
r) Output to 379.

Motion detection unit MDT (MotionDete)
(ctor) 375 is an object for detecting a moving portion in the image, and information 37 of the detected moving area.
6 is output to the distance information adding unit DIL379.

The distance information addition unit DIL379 is an object that adds distance information to the input two-dimensional information and outputs three-dimensional information. The ID information 378 from the face detection unit FI377 and the color recognition unit MCT373. The distance information is added to the information 374 such as the position, size, and feature amount of each of the divided areas and the information 376 of the moving area from the motion detecting unit MDT375 to add the three-dimensional information 380 to the short-term storage unit ST.
The data is output to the M (Short Term Memory) 381.

The short-term storage unit STM381 is an object that holds information about the external environment of the robot apparatus 1 for a relatively short time, and is an ArthurDecoder.
The voice recognition result (word, sound source direction, certainty factor) is received from (not shown), the position and size of the flesh color area and the position and size of the face area are received from the color recognition unit MCT373, and the face recognition unit FI377 receives the person. Receives ID information and the like. Further, the direction (joint angle) of the neck of the robot apparatus is received from each sensor on the body of the robot apparatus 1. Then, by using these recognition results and sensor outputs in an integrated manner, it is possible to obtain information about which person is currently in which person, the spoken word belongs to which person, and what kind of dialogue has been performed with that person so far. save. The physical information about such an object, that is, the target, and the event (history) viewed in the time direction are output, and the route plan determination unit (situation-dependent action hierarchy) (SB
L) Pass it to a higher-level module such as 360.

The route plan determination unit SBL is an object that determines the action (action depending on the situation) of the robot apparatus 1 based on the information from the above-mentioned short-term storage unit STM381. Multiple actions can be evaluated and performed simultaneously. In addition, the action can be switched to put the aircraft in a sleep state, and another action can be activated.

The above objects will be described in more detail below. The plane extraction unit PLEX320 receives the simultaneous conversion matrix 311, and the parallax image 312 corresponding to the left-eye image 201L and the right-eye image 201R shown in FIG. 11 corresponding to the simultaneous transformation matrix 311, and removes an obstacle according to the processing procedure shown in FIG. 12 below. recognize.

First, the plane extraction unit PLEX 320 receives the simultaneous conversion matrix 311 and the parallax image 312 corresponding thereto (step S61). For example, a parallax image 312 (or a distance image) obtained by a method such as stereoscopic vision (stereo camera) or an operation of a range sensor or the like should be easily converted into three-dimensional data by conversion based on appropriate calibration. And an array of this three-dimensional data is generated. That is, three-dimensional position data (x, y, z) viewed from the camera coordinates for each pixel is generated as a distance image using a calibration parameter that absorbs lens distortion and stereo attachment error from the parallax image 312. (Step S62). At this time, each three-dimensional data individually has a reliability parameter obtained from the reliability or the like in an input image such as a parallax image or a distance image, and is selected and input by this reliability parameter.

Data is randomly sampled from the selected three-dimensional data group, and the plane is estimated by Hough transform. That is, when the direction of the normal vector is (θ, φ) and the distance from the origin is d, the plane parameter (θ, φ,
d) is calculated, and the plane parameters are converted into voting spaces (θ, Ψ,
Directly vote for d) = (θ, φ cos θ, d) to estimate the plane. Thereby, the parameters of the dominant plane in the image are detected (step S63). The parameters of this plane are detected by the histogram of the parameter space (θ, φ) (voting space) shown in FIG. A small parameter vote can be regarded as an obstacle, and a large parameter vote can be regarded as on a plane.

At the time of voting, according to the calculation method of the reliability parameter or the plane parameter associated with the three-dimensional data,
By assigning different weights to the votes of one vote to make the voting values different, and by estimating the peak value obtained from the distribution of the voting values, weighted averaging in the vicinity of the peak value, etc. It is possible to estimate highly accurate data. Then, by performing iteration using this plane parameter as an initial parameter to determine the plane, it is possible to determine the plane with higher reliability.
In addition, the reliability of the plane is calculated using the reliability parameter associated with the three-dimensional data for which the finally determined plane is calculated, the residual error in the iteration, etc. The output facilitates the subsequent processing. As described above, the plane extraction is performed by a probabilistic method of determining the dominant plane parameters included in the three-dimensional data by voting, that is, estimating the probability density function based on the histogram. By using the parameters of the plane thus obtained, it is possible to know how far from the plane the measurement point of the distance originally obtained from the image is.

Next, the simultaneous conversion matrix 311 of the camera coordinate system
To conversion of the robot to the ground contact surface of the sole of the foot as shown in FIG. 14 (step S64). As a result, the parameters of the ground plane expressed in the camera coordinate system are calculated. Then, based on the detection of the plane in the image in step S63 and the collation of the sole grounding plane in step S64, the one corresponding to the floor surface is selected from the plane parameters in the image (step S65).

Then, using the plane parameters selected in step S65, points on the plane are selected from the original range image (step S66). This is calculated by using the following formulas (3) and (4), and the distance d from the plane is
Is smaller than the threshold value D _th .

[0072]

[Equation 3]

[0073]

[Equation 4]

FIG. 15 shows the measurement points (marked with X) selected within the range where the threshold value D _th is 1 cm. In this FIG.
The black dots are points that were not judged to be flat.

Therefore, points (points not on the floor) other than the points lying on the plane (floor) in step S66 can be recognized as obstacles in step S67. These judgment results are expressed by a point (x, y) on the floor plane and its height z. For example, the height z <0 indicates a point that is recessed from the plane.

As a result, an obstacle point having a height higher than that of the robot can pass through it, so that it can be determined that the obstacle point is not an obstacle.

Further, the robot view ((a) of FIG. 16)
The height z of the floor surface extraction image ((b) of FIG. 16) obtained from FIG.
If the coordinates are converted so that (z = 0), it is possible to represent a floor or an obstacle at a two-dimensional position on a plane as shown in (c) of FIG.

In this way, the obstacle recognition device can detect a stable plane because it detects a plane using a large number of measurement points. In addition, the correct plane can be selected by comparing the plane candidates obtained from the image with the floor surface parameters obtained from the posture of the robot. Moreover, since the floor surface is substantially recognized instead of recognizing the obstacle, it is possible to recognize the obstacle irrespective of the shape or size of the obstacle. Further, since the obstacle is represented by the distance from the floor surface, it is possible to detect even a small step or a recess. In addition, it is easy to determine whether to cross over or pass through considering the size of the robot. Furthermore, since it represents an obstacle on a two-dimensional floor surface, the method used by existing mobile robots can be applied to route planning and the like.
It can be operated faster than dimensional representation.

The obstacle grid calculation unit OG330, as described above, is an environment map which is the map information of the robot central coordinate system divided into grids of a predetermined size and, for example, the x-axis direction or y on the environment map. It has attitude information indicating a direction in which the robot device faces from a predetermined direction such as an axial direction. In the environment map, the obstacle information is the plane extraction unit P.
It has a grid (obstacle occupied area) that is input from the LEX and is recognized as an obstacle. When the robot apparatus moves, that is, when the odometry 314 is input, the obstacle grid calculation unit OG330 uses the environment map and the attitude information stored in the memory in advance to change the attitude of the robot apparatus (differential movement angle). ) And the movement distance (difference movement amount), the previously recognized environment map and posture direction information are updated. Here, if the difference movement amount is smaller than the grid size, the environment map is not updated, and if the movement amount exceeds the grid size, the environment map is updated. Also,
By appropriately changing the size of the environment map and its grid as necessary, the amount of calculation, the memory copy cost, etc. can be reduced.

The landmark position detector CLS340 is
As shown in FIG. 17, for example, the green portion 1001,
In an environment including an artificial color landmark 1004 having a color such as a pink portion 1002 or a blue portion 1003, the robot device 1's self-position (position) is determined by the sensor information of the robot device 1 and the motion information performed by the robot device 1. And posture).

For example, in a two-dimensional work space, grids (x, y) are provided at substantially equal intervals, and the probability p that a robot device exists for each position l (localization) of each grid.
Manage (l). The existence probability p (l) is updated in response to the movement of the robot apparatus, that is, the internal observation information a, or the observation of the landmark, that is, the external observation information s.

The existence probability p (l) is the existence probability p (l ') at the previous state of the robot apparatus, that is, at the self position l'.
And the transition probability p (l | a, l ′) that the state becomes l when the movement a is performed in the previous state l ′. That is, the probability p (l ') of each state l'
And adding (or integrating) the products of transition probabilities p (l | a, l ′) that the state 1 is obtained when the movement a is performed in the state l ′. It converges to the existence probability p (l) that the position is 1. Therefore, when the movement “a” of the robot apparatus is observed as the external observation result, the existence probability p (l) of the robot apparatus can be updated in each grid according to the following mathematical expression (5).

[0083]

[Equation 5]

Further, the existence probability p (l) that the robot device exists in the state, that is, the self position l is
(L) and the transition probability p (s | l) of observing a landmark in this state l. Therefore, in the state 1, the landmark observation, that is, the external observation information s
When is input, the existence probability p (l) of the robot device can be updated according to the following equation. However, as shown in the following mathematical expression (6), the right side is normalized for normalization by dividing by the probability p (s) of observing the landmark.

[0085]

[Equation 6]

FIG. 18 shows a landmark sensor CLS (self-position identification system) 3 according to the present embodiment, which uses both Markov localization and extended Kalman filter.
The functional configuration of 40 is schematically shown. As shown in the figure, the landmark sensor CLS340 includes a Markov localization unit (ML) 342, an extended Kalman filter unit (EKL) 343, and an EKL control unit 344 that controls the operations of the extended Kalman filter unit 343. It consists of and.

Markov localization unit 342
Holds its own position in the work space as a self-position probability density distribution on a discrete grid, and inputs external observation information s regarding the observation of landmarks and internal observation information a regarding the movement of the robot device itself. Then, the self-position probability density distribution is updated. Then, at each time point, the grid having the highest value of the self-position probability density distribution is output to the EKL control unit 344 as the self-position estimation result.

FIG. 19 shows the self-position probability density distribution on each grid obtained by the Markov localization unit 342. In the figure, the probability density in each grid is represented by shading. Darkest color,
That is, the grid having the highest value of the self-position probability density distribution becomes the self-position estimation result.

The self-position identification by this Markov localization is robust against the noise of the sensor, and is characterized mainly in that the accuracy of the identification solution is coarse but the convergence speed of the solution is fast.

On the other hand, the extended Kalman filter unit 343 shown in FIG. 18 holds its own position as a measured value of the state variable [x, y, θ] and observes the color landmark 1004 installed in the environment, The self-position is estimated based on the relative position from the landmark. Further, when observing its own motion information, the state quantity is estimated based on the motion information.

The extended Kalman filter unit 343 shows the relationship between the motion information a of the robot apparatus itself and the state, that is, the state model defining the relationship between the self position l and the observation position s of the self position l and the landmark. It consists of a specified observation model.

The state model has a transition function F (l, a) that gives a theoretical state 1 when the robot apparatus performs the action a in the state (self position) l. In reality, since the noise component w is superimposed on the theoretical state 1, the state 1 of the robot apparatus converges according to the following equation (7) according to the state model.

[0093]

[Equation 7]

Further, the observation model is such that the robot device is in a state, that is, in the self position i, in a known environment Env.
It is provided with an observation function H (Env, l) that gives a theoretical value s of observation (for example, the position of a landmark).
Since the noise component v is actually superimposed on the theoretical value of the observation, the observed value s converges according to the following equation (8) according to the observation model.

[0095]

[Equation 8]

The noises w and v superimposed on the state 1 and the observation s are assumed to be Gaussian distributions with a median of zero.

An extended Kalman filter unit 343 having a state model defining the relationship between the motion information a of the robot apparatus itself and the self-position l and an observation model defining the relationship between the self-position l and the observation information s of the landmark. In the above, the operation information a is the observation information s of the landmark as the internal observation result.
Are known as external observation results. Therefore, the self-position identification of the robot device can be reduced to the problem of estimating the state 1 of the robot device from the motion information a and the observation information s of the robot device. Here, the operation a, the state l, and the observation s of the robot device can be expressed as a Gaussian distribution shown below.

[0098]

[Equation 9]

It is estimated that the state 1 of the robot apparatus at a certain time point has a Gaussian distribution having a certain median value and covariance. Then, when the motion a of the robot apparatus is observed, the median value and the covariance regarding the estimated value of the state 1 can be updated by the following mathematical expressions (10-1) and (10-2).

[0100]

[Equation 10]

Here, ∇F _l and ∇F _a are as follows.

[0102]

[Equation 11]

Similarly, the state 1 of the robot apparatus at a certain point of time is estimated as taking a Gaussian distribution having a certain median and covariance. And the landmark observation information s
When is observed, the median and covariance regarding the estimated value of the state 1 can be updated by the following mathematical formulas (12-1) and (12-2).

[0104]

[Equation 12]

Here, each parameter is as follows.

[0106]

[Equation 13]

Since the extended Kalman filter 343 is excellent in robustness to sensor information, the estimation result of the extended Kalman filter unit 343 is the output of the entire landmark sensor CLS340.

The EKL control section 344 controls the operation of the extended Kalman filter section 344 according to the output result of the Markov localization section 342. More specifically, the validity of the landmark observation information s is verified based on the self-position estimation result of the Markov localization unit 342. The validity of the observation information s is that the probability p (s | mlp) of observing a landmark at the grid position mlp, which is the maximum existence probability in the Markov localization unit 342, is a predetermined threshold parameter th.
It can be judged by whether or not it exceeds the resh _obs .

The probability p (s | mlp) of observing a landmark at the grid position mlp is the threshold parameter th.
When it is less than resh _obs , it is estimated that even in the Markov localization unit 342 that is robust against sensor noise, the identification solution does not sufficiently converge due to sensor noise. In such a case, even if the self-position is estimated in the extended Kalman filter unit 343, which has low robustness against sensor noise, a highly accurate solution cannot be obtained, and rather the calculation time is wasted. Therefore, when it is determined that the observation information s is not appropriate, the input of the external observation information s, that is, the observation information of the landmark, to the extended Kalman filter unit 343 is blocked using the switch 345, and the extension is performed. Kalman filter unit 34
The update of the self-position estimation value in 3 is stopped.

The EKL control section 344 also verifies the validity of the self-position estimation result of the extended Kalman filter section 343. The validity of the self-position estimation result can be determined by a distribution comparison test with the existence probability p (l) output from the Markov localization unit 342, using the estimated median and covariance of the state l. . An example of the distribution comparison test is the chi-square test chi-square.
-Test (ml, ekf).

By the distribution comparison test, the Markov localization unit 342 and the extended Kalman filter unit 343.
If the probability distributions of and are not similar, it may be determined that the self-position estimation value in the extended Kalman filter unit 343, which has low robustness to sensor noise, is not valid due to the influence of sensor noise. it can. In such a case, the EKL control unit 344 causes the extended Kalman filter unit 343 to be reinitialized. This is because the extended Kalman filter requires a great deal of time to restore again.

Next, the operation of the landmark sensor CLS340 will be described. FIG. 20 is a flowchart showing the operation of the landmark sensor CLS340. Figure 2
As shown in 0, when the inner-field observation information a relating to the movement of the robot apparatus 1 is input to the landmark sensor CLS340, the Markov localization unit 342 first updates the self-position estimation value using the above mathematical expression (5). Processing is performed (step S201). Then, in the extended Kalman filter unit 343, the above mathematical formula (10)
Is used to update the self-position estimation value (step S202).

When the outside world observation information s regarding the observation of the landmark is input to the landmark sensor CLS340, first, in the Markov localization unit 342, the self-position estimation value is updated by using the above equation (6). It is performed (step S211).

The output result of the Markov localization unit 342 is input to the EKL control unit 344, and the validity of the observation information s is verified (step S212). The validity of the observation information s is based on the Markov localization section 3
The grid position mlp having the maximum existence probability at 42
Probability of observing landmarks at p (s | mlp)
Can exceed the predetermined threshold parameter thresh _obs .

The probability p (s | mlp) of observing a landmark at the grid position mlp is the threshold parameter th.
When it is less than resh _obs , it is estimated that even in the Markov localization unit 342 that is robust against sensor noise, the identification solution does not sufficiently converge due to sensor noise. In such a case, even if the extended Kalman filter unit 343, which has low robustness to sensor noise, estimates the self-position, an accurate solution cannot be obtained, but rather the calculation time is wasted. Therefore, when it is determined that the observation information s is not appropriate, the input of the external observation information s, that is, the observation information of the landmark, to the extended Kalman filter unit 343 is blocked using the switch 345, and the extension is performed. Kalman filter unit 34
The update of the self-position estimation value in 3 is stopped.

On the other hand, as a result of verifying the observation information s, the probability p (s | mlp) that satisfies the validity, that is, the landmark is observed at the grid position mlp, is the threshold parameter t.
If it exceeds hresh _obs , the extended Kalman filter unit 343 further calculates the above-mentioned mathematical expression (12-1),
The update process is performed using (12-2) (step S
213).

The result of self-position estimation by the extended Kalman filter unit 343 is input to the EKL control unit 344 and its validity is verified (step S214). The validity of the self-position estimation result by the extended Kalman filter unit 343 is determined by comparing the distribution with the existence probability p (l) output from the Markov localization unit 342, using the estimated median and covariance of the state l. Can be judged by the test. An example of a distribution comparison test is a chi-square test chi-square-test (ml, ek
f).

By the distribution comparison test, the Markov localization unit 342 and the extended Kalman filter unit 343.
When the respective probability distributions are not similar, the self-position estimation value in the extended Kalman filter unit 343, which has low robustness to sensor noise, is the sensor position.
It can be judged to be invalid due to the influence of noise. In such a case, the EKL control unit 344 causes the extended Kalman filter unit 343 to be re-initialized (step S215). This is because the extended Kalman filter requires a great deal of time to restore again.

In this way, the landmark sensor CL
In S340, it is possible to perform high-accuracy, high-speed, and robust self-localization by using both the global search, which performs a search in a relatively short search time in a wide range, and the local search, which has a high accuracy but requires a search time. .

The route plan determination unit SBL360 acquires the recognition result regarding the obstacles existing around the robot apparatus 1 from the obstacle information 331 from the obstacle grid calculation unit OG330, and the current robot is calculated from the absolute coordinate calculation unit LZ350. By acquiring the absolute coordinates of the device 1, a path that can be walked without colliding with an obstacle to a target point given in the absolute coordinate system or the robot center coordinate system of the robot device center is generated, Issue an operation command to carry out the route. That is, the action to be taken by the robot apparatus 1 is determined from the input data according to the situation, and the action sequence is output. Here, in the obstacle grid calculation unit OG330,
The processing based on the obstacle information 331 will be described later.

The points on the obstacle map generated by the obstacle information 331 from the obstacle grid calculation unit OG330 are
As shown in FIG. 21, it is classified into the following three types.

The first point is a point where an obstacle exists (point shown in black in the figure). The second point is a point (point represented by white in the figure) on the free space (space where no obstacle exists). Then, the third point is a point on the unobserved region (point indicated by the diagonal line in the figure).

Next, the route planning algorithm adopted by the route plan determining unit SBL360 is shown in the flowchart of FIG. 22, and the details will be described below.

First, the line of sight is directed toward the destination so that an obstacle map around a straight route connecting the current position to the target position is created (step S71). Then, the distance image is observed, the distance is measured, and the obstacle map is created (updated) (step S72).

Next, in the generated obstacle map, the unobserved area and the free space area are regarded as the movable area and the route is planned (step S73).

[0126] As the path planning, for example, A ^* search (A ^* search) that minimizes the cost of the entire route using the method referred to. In this A ^* search, f is used as the evaluation function, and h
It is a best-first search in which the function is admissible. We use the point of efficiency optimization for any heuristic function.

In step S73, it is checked in step S74 whether or not the route generated by applying the A ^* search is movable, and the route that can avoid the obstacle cannot be planned. In the case of (NO), there is no possibility that a movable route can be obtained even if the observation is continued any more, so that is notified and the route planning ends (step S75).

In step S73, if the movable route can be planned by applying, for example, the A ^* search (YES), the process proceeds to step S76 and whether or not an unobserved region is included in the output route. To search. This step S76
If no unobserved area is included in the route (NO)
, A movable route is output as a route plan to the destination in step S77. If the unobserved region is included in step S76 (YES), the process proceeds to step S78, the step count from the current position to the unobserved region is calculated, and it is checked whether the step count exceeds a threshold value.

If the step count exceeds the threshold value in step S78 (YES), the movable route to the unknown area is output in step S79, and the process returns to step S71. On the other hand, when the number of steps to the unknown observation region is less than the threshold value in step S78 (NO), the line-of-sight direction is controlled so that the unobserved region is observed by distance (step S80), and the obstacle is detected again. Update the physical map.

The route plan determination unit SBL360 which adopts the route planning algorithm as described above, considers the unobserved region and the free space region as the movable region, performs the route planning, and outputs the unobserved region included on the route. By re-observing only a part, it is possible to generate a moving route plan efficiently and in a short time without performing unnecessary observation and distance image calculation processing when moving to a destination.

Hereinafter, a bipedal walking type robot device equipped with the above-described position recognition device according to the embodiment of the present invention will be described in detail. This humanoid robot device is a practical robot that supports human activities in various situations in the living environment and other daily life, and can act according to internal conditions (anger, sadness, joy, enjoyment, etc.), It is an entertainment robot that can express the basic actions that humans perform.

As shown in FIG. 23, the robot apparatus 1 has
The head unit 3 is connected to a predetermined position of the trunk unit 2, and two left and right arm units 4R / L and two left and right leg units 5R / L are connected (however, provided. Each of R and L is a suffix indicating each of right and left. The same applies below.).

FIG. 24 schematically shows the joint degree-of-freedom structure of the robot apparatus 1. The neck joint supporting the head unit 3 includes a neck joint yaw axis 101 and a neck joint pitch axis 1.
02 and the neck joint roll shaft 103 have three degrees of freedom.

Further, each arm unit 4R / L constituting the upper limb has a shoulder joint pitch axis 107, a shoulder joint roll axis 108, an upper arm yaw axis 109, and an elbow joint pitch axis 110.
, Forearm yaw axis 111, wrist joint pitch axis 112,
It is composed of a wrist joint roll shaft 113 and a hand portion 114. The hand portion 114 is actually a multi-joint / multi-degree-of-freedom structure including a plurality of fingers. However, since the motion of the hand portion 114 has little contribution or influence to the posture control and the walking control of the robot apparatus 1, it is assumed that the degree of freedom is zero in this specification. Therefore, each arm has seven degrees of freedom.

The trunk unit 2 has three degrees of freedom: the trunk pitch axis 104, the trunk roll axis 105, and the trunk yaw axis 106.

Further, each leg unit 5R / L constituting the lower limb has a hip joint yaw axis 115 and a hip joint pitch axis 1
16, a hip joint roll shaft 117, and a knee joint pitch shaft 11
8, ankle joint pitch axis 119, and ankle joint roll axis 1
20 and a foot 121. In this specification,
The intersection of the hip joint pitch axis 116 and the hip joint roll axis 117 defines the hip joint position of the robot apparatus 1. The foot 121 of the human body is actually a structure including a multi-joint, multi-degree-of-freedom foot, but the foot of the robot apparatus 1 has zero degrees of freedom. Therefore, each leg has 6 degrees of freedom.

In summary, the robot apparatus 1 as a whole has a total of 3 + 7 × 2 + 3 + 6 × 2 = 32 degrees of freedom. However, the robot device 1 for entertainment is not necessarily limited to 32 degrees of freedom. It is needless to say that the degree of freedom, that is, the number of joints, can be appropriately increased or decreased in accordance with design / production constraints, required specifications, and the like.

Each degree of freedom of the robot apparatus 1 as described above is actually implemented by using an actuator.
It is preferable that the actuator be small and lightweight in view of demands such as eliminating extra bulges in appearance and approximating the shape of a natural human body, and performing posture control for an unstable structure such as bipedal walking. .

FIG. 25 schematically shows the control system configuration of the robot apparatus 1. As shown in the figure, the robot device 1 performs a coordinated operation between the trunk unit 2, the head unit 3, the arm unit 4R / L, the leg unit 5R / L, which represent human limbs, and the respective units. And a control unit 10 that performs adaptive control for realizing the above.

The overall operation of the robot apparatus 1 is controlled by the control unit 10. Control unit 10
Is a CPU (Central Processing Unit) or DRA
An interface (not shown) for exchanging data and commands with the main control unit 11 including main circuit components (not shown) such as M and flash ROM, and the power supply circuit and each component of the robot apparatus 1. And the peripheral circuit 12 including.

In implementing the present invention, the installation place of the control unit 10 is not particularly limited. Although it is mounted on the trunk unit 2 in FIG. 25, it may be mounted on the head unit 3. Alternatively, the control unit 10 may be provided outside the robot apparatus 1 to communicate with the body of the robot apparatus 1 in a wired or wireless manner.

The degree of freedom of each joint in the robot apparatus 1 shown in FIG. 24 is realized by an actuator corresponding to each joint. That is, the head unit 3 includes a neck joint yaw axis 101, a neck joint pitch axis 102, and a neck joint roll axis 103.
, A neck joint yaw axis actuator A ₂ , a neck joint pitch axis actuator A ₃ , and a neck joint roll axis actuator A ₄ are provided.

Further, the head unit 3 is provided with a CCD (Charge Coupled Device) camera for picking up an image of an external situation, and a distance sensor for measuring a distance to an object located in front, A microphone for collecting external sound, a speaker for outputting voice, a touch sensor for detecting the pressure received by the physical action of the user such as "stroking" or "striking" are provided. ing.

The trunk unit 2 includes a trunk pitch axis actuator A ₅ , a trunk roll axis actuator A ₆ , which expresses the trunk pitch axis 104, trunk roll axis 105, and trunk yaw axis 106, respectively. A trunk yaw axis actuator A ₇ is provided. Also, in the trunk unit 2,
The robot apparatus 1 is provided with a battery as a power source. This battery is composed of a chargeable / dischargeable battery.

The arm unit 4R / L is subdivided into an upper arm unit 4 ₁ R / L, an elbow joint unit 4 ₂ R / L, and a forearm unit 4 ₃ R / L. 107, shoulder joint roll axis 108, upper arm yaw axis 109, elbow joint pitch axis 110, forearm yaw axis 111, wrist joint pitch axis 112, wrist joint roll axis 113, respectively, shoulder joint pitch axis actuator A ₈ , shoulder joint roll An axis actuator A ₉ , an upper arm yaw axis actuator A ₁₀ , an elbow joint pitch axis actuator A ₁₁ , an elbow joint roll axis actuator A ₁₂ , a wrist joint pitch axis actuator A ₁₃ , and a wrist joint roll axis actuator A ₁₄ are provided.

Also, the leg unit 5R / L is used for the thigh unit.
Knit 5₁R / L and knee unit 5 _TwoR / L and shin
Knit 5_ThreeAlthough it is subdivided into R / L, the hip joint yaw axis 11
5, hip pitch axis 116, hip roll axis 117, knee
Joint pitch axis 118, ankle joint pitch axis 119, ankle function
Hip joint yaw axis actuation expressing each of the knot roll axes 120
Player A₁₆, Hip joint pitch axis actuator
A₁₇, Hip joint roll axis actuator A₁₈,Knee joint
Pitch axis actuator A₁₉, Ankle joint pitch axis
Cheetah A₂₀, Ankle joint roll axis actuator A
₂₁Has been deployed. Actuator used for each joint
Data A_Two, A_ThreeIs more preferably a direct gear type
Motor unit with integrated servo control system in one chip
Small type AC servo actuator mounted inside
Can be composed of

For each mechanical unit such as the trunk unit 2, the head unit 3, each arm unit 4R / L, each leg unit 5R / L, etc., the sub-control units 20, 21, 22R / of the actuator drive control unit are provided. L, 23R / L are deployed. Furthermore, a ground contact confirmation sensor 30R / L that detects whether or not the sole of each leg unit 5R / L has landed is attached, and a posture sensor 31 that measures a posture is provided in the trunk unit 2. is doing.

The ground contact confirmation sensor 30R / L is composed of, for example, a proximity sensor or a micro switch installed on the sole of the foot. Further, the posture sensor 31 is composed of, for example, a combination of an acceleration sensor and a gyro sensor.

By the output of the ground contact confirmation sensor 30R / L, it is possible to determine whether each of the left and right legs is currently standing or free leg during an operation period such as walking or running. Further, the output of the posture sensor 31 can detect the inclination and posture of the trunk.

The main control section 11 uses the sensors 30R / L, 3
The control target can be dynamically corrected in response to the output of 1. More specifically, the sub control units 20, 21,
By performing adaptive control on each of 22R / L and 23R / L, it is possible to realize a whole-body movement pattern in which the upper limbs, the trunk, and the lower limbs of the robot apparatus 1 are cooperatively driven.

The whole body motion of the robot apparatus 1 on the body is
Foot control, ZMP (Zero Moment Point) trajectory, trunk movement, upper limb movement, waist height, etc. are set, and commands for instructing movements according to these setting contents are issued to the sub-control units 20, 21, 22R. / L, 23R / L. The sub-control units 20, 21, ... Interpret the received commands from the main control unit 11 and output drive control signals to the actuators A ₂ , A ₃ ,. . The “ZMP” mentioned here is a point on the floor surface where the moment due to the floor reaction force during walking becomes zero, and the “ZMP trajectory” is, for example, the robot device 1
Means a locus of movement of the ZMP during the walking motion period. Regarding the concept of ZMP and the application of ZMP to the stability criterion of walking robots, see Miomir Vukobratovi.
c "LEGGED LOCOMOTION ROBOTS" (Ichiro Kato, "Walking Robot and Artificial Feet" (Nikkan Kogyo Shimbun)).

As described above, in the robot apparatus 1, each sub-control unit 20, 21, ... Interprets the received command from the main control unit 11 and each actuator A ₂ , A ₃
A drive control signal is output to ... to control the drive of each unit. As a result, the robot apparatus 1 makes a stable transition to the target posture and can walk in a stable posture.

Further, in the control unit 10 of the robot apparatus 1, in addition to the posture control as described above, various sensors such as an acceleration sensor, a touch sensor, a ground contact confirmation sensor, image information from a CCD camera, and sound from a microphone. Information is handled in a centralized manner. In the control unit 10, although not shown, various sensors such as an acceleration sensor, a gyro sensor, a touch sensor, a distance sensor, a microphone, a speaker, actuators, CCD cameras, and batteries are connected to the main control unit 11 via corresponding hubs. Has been done.

The main controller 11 sequentially takes in sensor data, image data, and audio data supplied from the above-mentioned sensors, and sequentially stores them in a predetermined position in the DRAM through the internal interface. Further, the main control unit 11 sequentially takes in the battery remaining amount data representing the battery remaining amount supplied from the battery, and stores this in the DRAM.
It is stored in a predetermined position inside. Each sensor data, image data, voice data, and battery remaining amount data stored in the DRAM are used when the main control unit 11 controls the operation of the robot apparatus 1.

The main control section 11 reads the control program and stores it in the DRAM at the initial stage when the robot apparatus 1 is powered on. In addition, the main control unit 11 uses the sensor data, the image data, the audio data, and the battery remaining amount data that are sequentially stored in the DRAM from the main control unit 11 as described above, based on the self and surrounding conditions and the user. Judging whether or not there are instructions and how to work.

Further, the main control section 11 determines an action according to its own situation based on this determination result and the control program stored in the DRAM, and drives a necessary actuator based on the determination result. The robot apparatus 1 is caused to take actions such as so-called “gesture” and “hand gesture”.

In this way, the robot apparatus 1 can judge the self and surrounding conditions based on the control program, and can act autonomously in accordance with instructions and actions from the user.

By the way, the robot apparatus 1 can act autonomously according to the internal state. Therefore, a software configuration example of the control program in the robot apparatus 1 will be described with reference to FIGS. 26 to 31.

In FIG. 26, the device driver layer 40 is located in the lowest layer of the control program and is composed of a device driver set 41 consisting of a plurality of device drivers. In this case, each device
The driver is an object that is allowed to directly access hardware used in a normal computer such as a CCD camera and a timer, and receives an interrupt from the corresponding hardware to perform processing.

The robotic server object 42 is located in the lowest layer of the device driver layer 40, and for example, the various sensors and actuators A described above are used.
₂ , a virtual robot 43 that is a software group that provides an interface for accessing hardware such as A _3, ..., a power manager 44 that is a software group that manages switching of power supplies, and various other types. The device driver manager 45 is a software group that manages the device driver, and the designed robot 46 is a software group that manages the mechanism of the robot apparatus 1.

The manager object 47 includes an object manager 48 and a service manager 49.
It consists of Object manager 48
Is a software group that manages activation and termination of each software group included in the robotic server object 42, the middleware layer 50, and the application layer 51, and the service manager 49.
Is a software group that manages the connection of each object based on the connection information between each object described in the connection file stored in the memory card.

The middle wear layer 50 is located in the upper layer of the robotic server object 42 and is composed of a software group that provides basic functions of the robot apparatus 1 such as image processing and voice processing. There is. Further, the application layer 51 is located above the middleware layer 50, and the middleware layer 50
It is composed of a software group for determining the action of the robot apparatus 1 based on the processing result processed by each software group forming the layer 50.

FIG. 27 shows specific software configurations of the middleware layer 50 and the application layer 51, respectively.

The middleware layer 50, as shown in FIG. 27, is for noise detection, temperature detection, brightness detection,
Each signal processing module 6 for scale recognition, distance detection, posture detection, touch sensor, motion detection, and color recognition
A recognition system 70 having 0 to 68 and an input semantics converter module 69, an output semantics converter module 78, and posture management, tracking, motion reproduction, walking, fall recovery, LE
Signal processing modules 71 to 7 for D lighting and sound reproduction
And an output system 79 having 7 or the like.

Each signal processing module 60 of the recognition system 70
68 captures corresponding data of each sensor data, image data, and audio data read from the DRAM by the virtual robot 43 of the robotic server object 42, performs a predetermined process based on the data, The processing result is given to the input semantics converter module 69. Here, for example, the virtual robot 43 is configured as a portion that exchanges or converts a signal according to a predetermined communication protocol.

The input semantics converter module 69 detects "noisy", "hot", "bright", "ball detected", and "fall" based on the processing results given from the respective signal processing modules 60 to 68. The user and surroundings, such as "Yes", "Stabbed", "Struck", "I heard Domiso scale", "A moving object was detected", or "An obstacle was detected", and the user. It recognizes the command and the action from, and outputs the recognition result to the application layer 41.

The application layer 51 is shown in FIG.
As shown in FIG. 5, the action model library 80, the action switching module 81, the learning module 82, the emotion model 83, and the instinct model 84 are composed of five modules.

In the behavior model library 80, as shown in FIG. 29, "when the battery level is low",
Independently corresponding to some preselected condition items such as "returning from a fall", "avoiding obstacles", "expressing emotions", "detecting a ball", etc. A behavior model is provided.

Each of these behavior models will be described later as needed when a recognition result is given from the input semantics converter module 69 or when a certain time has passed since the last recognition result was given. As described above, each subsequent action is determined with reference to the corresponding emotional parameter value held in the emotion model 83 and the corresponding desire parameter value held in the instinct model 84, and the decision result is determined by the action switching module 81. Output to.

In the case of this embodiment, each behavior model uses one node (state) NODE _{0 to} NODE _n as shown in FIG. 30 to determine any other node NODE ₀ as a method for determining the next behavior. ˜NODE _n , a finite probability that determines probabilistically based on the transition probabilities P _{1 to} P _n respectively set for the arcs ARC ₁ to ARC _n1 connecting the nodes NODE _{0 to} NODE _n. An algorithm called an automaton is used.

Specifically, each behavior model is
NODE that forms the behavior model of the child₀~ NODE_n
To correspond to each of these nodes NODE₀~ NO
DE _nEach has a state transition table 90 as shown in FIG.
There is.

In this state transition table 90, the node NO.
Input events (recognition results) that are transition conditions in DE _{0 to} NODE _n are listed in order of priority in the column of “input event name”, and further conditions regarding the transition conditions are listed in the columns of “data name” and “data range”. It is described in the corresponding line.

Therefore, in the node NODE ₁₀₀ represented by the state transition table 90 of FIG. 31, "the ball is detected (B
ALL) ”is given, the“ size (SIZ) of the ball given together with the recognition result is given.
"E)" is in the range of "0 to 1000" and the recognition result of "obstacle detection (OBSTACLE)" is given, the "distance (DISTANCE) to the obstacle given together with the recognition result is given. ) ”Is in the range of“ 0 to 100 ”is a condition for transition to another node.

Further, in this node NODE ₁₀₀ , even when there is no recognition result input, the parameter values of each emotion and each desire held in the emotion model 83 and the instinct model 84, which the behavior model periodically refers to, are stored. Among them, "Joy" held by emotion model 83,
When the parameter value of either "Surprise" or "Sadness" is in the range of "50 to 100", it is possible to transit to another node.

Further, in the state transition table 90, the node names that can transit from the nodes NODE _{0 to} NODE _n are listed in the row of “transition destination node” in the column of “transition probability to other node”, and “ Input event name ",
Each other node NOD that can transit when all the conditions described in the columns of "data name" and "data range" are met
The transition probabilities from E _{0 to} NODE _n are respectively described in the corresponding locations in the “transition probabilities to other nodes” section, and the action to be output when transitioning to the nodes NODE _{0 to} NODE _n is “other It is described in the row of “output action” in the column of “transition probability to node”. In addition, the sum of the probabilities of each row in the column of "probability of transition to other node" is 100.
It is [%].

Therefore, in the node NODE ₁₀₀ represented by the state transition table 90 of FIG. 31, for example, "ball is detected (BALL)" and the "SIZE" of the ball is in the range of "0 to 1000". If the recognition result that is, is given, there is a probability of "30 [%]"
ODE ₁₂₀ (node 120) ", at that time" A
The action of “CATION 1” will be output.

Each behavior model has nodes NODE ₀ to NO described as such a state transition table 90.
DE _n are configured to be connected to each other, and when a recognition result is given from the input semantics converter module 69, the corresponding nodes NODE ₀ to
The next action is stochastically determined using the state transition table of NODE _n , and the determination result is output to the action switching module 81.

The action switching module 81 shown in FIG.
Among the actions output from the action models of the action model library 80, the action output from the action model having a predetermined high priority is selected, and a command to execute the action (hereinafter, action command That.)
To the output semantics converter module 78 of the middleware layer 50. Note that, in this embodiment, the behavior model shown on the lower side in FIG. 29 has a higher priority.

Further, the action switching module 81 notifies the learning module 82, the emotion model 83, and the instinct model 84 that the action is completed based on the action completion information given from the output semantics converter module 78 after the action is completed. .

On the other hand, the learning module 82 inputs the recognition result of the teaching received as an action from the user such as “struck” or “stabbed” among the recognition results given from the input semantics converter module 69. To do.

Then, based on the recognition result and the notification from the action switching module 71, the learning module 82 reduces the probability of the action to be expressed when "struck (scared)" and "stabbed ( Praised) ”, the corresponding transition probability of the corresponding behavior model in the behavior model library 70 is changed so as to increase the occurrence probability of that behavior.

On the other hand, the emotion model 83 is "joy (Jo
y) ”,“ Sadness ”,“ Anger ”,
With respect to a total of 6 emotions of “Surprise”, “Disgust”, and “Fear”, a parameter indicating the strength of the emotion is held for each emotion. Then, the emotion model 83 gives specific recognition results such as “struck” and “stabbed” given from the input semantics converter module 69 to the parameter values of these emotions, the elapsed time and the action switching module 81. It is updated periodically based on notifications from etc.

Specifically, the emotion model 83 is determined based on the recognition result given from the input semantics converter module 69, the action of the robot apparatus 1 at that time, the elapsed time from the last update, and the like. The amount of change in emotion at that time calculated by the arithmetic expression is ΔE
[T], E [t] of the current parameter value of the emotion, the coefficient representing the sensitivity of the emotion as k _e, the following equation (1
The parameter value E [t + 1] of the emotion in the next cycle is calculated by 4), and the parameter value of the emotion is updated by replacing this with the current parameter value E [t] of the emotion. Further, the emotion model 83 updates the parameter values of all emotions in the same manner as above.

[0184]

[Equation 14]

The degree of influence of each recognition result and the notification from the output semantics converter module 78 on the variation amount ΔE [t] of the parameter value of each emotion is predetermined, and for example, “beating” is performed. The recognition result such as “ta” is the variation amount ΔE of the parameter value of the emotion of “anger”
[T] has a great influence, and the recognition result such as “struck” is the variation amount ΔE of the parameter value of the emotion of “joy”.
It has a great influence on [t].

Here, the notification from the output semantics converter module 78 is so-called action feedback information (action completion information), which is information about the appearance result of an action, and the emotion model 83 is also based on such information. Change emotions. This is, for example, that the behavior level of anger is lowered by the action of "screaming". The notification from the output semantics converter module 78 is sent to the learning module 8 described above.
2 is also input, and the learning module 82 changes the corresponding transition probability of the behavior model based on the notification.

The feedback of the action result may be performed by the output of the action switching module 81 (action added with emotion).

On the other hand, the instinct model 84 is "exercise desire (exer
cise), “affection”, “appetite”
e) ”and“ curiosity ”independent of each other 4
For each desire, a parameter indicating the strength of the desire is held for each of these desires. And the instinct model 84
Updates the parameter values of these desires periodically based on the recognition result provided from the input semantics converter module 69, the elapsed time, the notification from the action switching module 81, and the like.

Specifically, the instinct model 84 determines a predetermined "movement desire", "love desire", and "curiosity" based on the recognition result, the elapsed time, the notification from the output semantics converter module 78, and the like. The fluctuation amount of the desire at that time calculated by the arithmetic expression is ΔI [k],
Let I [k] be the current parameter value of the desire and a coefficient k _i representing the sensitivity of the desire, and the following mathematical expression (1
5) is used to calculate the parameter value I [k + 1] of the desire in the next cycle, and the calculation result is replaced with the current parameter value I [k] of the desire to update the parameter value of the desire. . Also, the instinct model 84
Updates the parameter values of each desire except "appetite" in the same manner.

[0190]

[Equation 15]

The degree of influence of the recognition result and the notification from the output semantics converter module 78 on the variation amount ΔI [k] of the parameter value of each desire is predetermined, and for example, the output semantics converter is used. The notification from the module 78 has a great influence on the fluctuation amount ΔI [k] of the “tiredness” parameter value.

In this embodiment, each emotional influence is
And each desire (instinct) parameter value is 0 to 10
It is regulated to fluctuate within the range of 0, and
A few k _e, K_iThe value of is also set individually for each emotion and each desire.
Has been.

On the other hand, the output semantics converter module 78 of the middleware layer 50, as shown in FIG. 27, is "forward" and "happy" given from the action switching module 81 of the application layer 51 as described above. , An abstract action command such as “squeal” or “tracking (chasing the ball)” is given to the corresponding signal processing modules 71 to 77 of the output system 79.

Then, these signal processing modules 71 to 7
When an action command is given, 7 generates a servo command value to be given to a corresponding actuator to take the action, sound data of sound output from a speaker, and / or drive data given to the LED, based on the action command. Then, these data are sequentially transmitted to the corresponding actuator or speaker or LED via the virtual robot 43 of the robotic server object 42 and the signal processing circuit.

In this way, the robot apparatus 1 can perform an autonomous action according to its own (internal) and surrounding (external) conditions, and instructions and actions from the user, based on the control program described above.

Such a control program is provided via a recording medium recorded in a format readable by the robot apparatus. As a recording medium for recording the control program,
Recording medium of magnetic reading system (for example, magnetic tape, flexible disk, magnetic card), recording medium of optical reading system (for example, CD-ROM, MO, CD-R, DVD)
Etc. are possible. The recording medium also includes a storage medium such as a semiconductor memory (so-called memory card (rectangular type, square type, or any shape), IC card) or the like. Also,
The control program may be provided via the so-called Internet or the like.

These control programs are reproduced via a dedicated read driver device, a personal computer or the like, transmitted to the robot device 1 by a wired or wireless connection, and read. Further, when the robot device 1 includes a drive device for a miniaturized storage medium such as a semiconductor memory or an IC card, the control program can be directly read from the storage medium.

[0198]

As described in detail above, the position recognition method according to the present invention recognizes the position of a moving body moving in a predetermined environment as a position on the environment map corresponding to the predetermined environment. In the method, the environment map is partitioned by a grid of a predetermined size, and when the moving body moves, if the moving range is within the grid, the environment map is retained and the moving range exceeds the grid. In that case, since the above environment map is moved and updated in the above grid units,
If the moving range of the moving body does not exceed the grid, it is not necessary to update the environment map, the calculation amount and memory copy cost at the time of update can be reduced, and the environment map is divided into a grid of a predetermined size. Therefore, the amount of data can be reduced.

The robot device according to the present invention is a robot device which operates based on the input information supplied,
A position recognizing device for recognizing the position of a moving body moving in a predetermined environment as a position on the environment map corresponding to the predetermined environment is mounted, and the position recognizing device includes a storage unit for storing the environment map, With the movement of the mobile body, it has a map update means for updating the environment map, the environment map is partitioned by a grid of a predetermined size, the map update means, with the movement of the mobile body, When the movement range is within the grid, the environment map is retained, and when the movement range exceeds the grid, the environment map is moved in units of the grid. Therefore, when updating the environment map when the robot moves, the robot Since the environment map is not updated when the movement range of the device does not exceed the grid, it is possible to eliminate unnecessary calculation amount in the robot device and reduce the load on the robot device. With wear, to store the information of the environment map grid units, it reduces the amount of data that the robot device's.

[Brief description of drawings]

FIG. 1 is a block diagram showing an outline of a robot apparatus according to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing a software configuration of the robot apparatus according to the embodiment of the present invention.

FIG. 3 is a schematic diagram showing an environment map displayed by the position recognition device (robot device) according to the embodiment of the present invention.

4 (a), (b), (c), and (d) are diagrams showing a method for updating an environment map according to a movement amount of a position recognition device according to an embodiment of the present invention. It is a schematic diagram which shows an environment map when not updating a map and when updating an environment map.

FIG. 5 is a schematic diagram showing each movement amount when updating the environment map in the position recognition device in the embodiment of the present invention.

6 (a) and 6 (b) are schematic views showing states before and after updating the posture direction in the position recognition device according to the embodiment of the present invention.

FIG. 7 is a schematic diagram illustrating effects of the position recognition device according to the embodiment of the present invention.

FIG. 8 is a schematic diagram showing an operation panel displaying an environment map and attitude information.

FIG. 9 is a flowchart showing an operation of software of the robot device according to the embodiment of the present invention.

FIG. 10 is a schematic diagram showing a flow of data input to the software.

[Fig. 11] Fig. 11 is a diagram for describing generation of a parallax image input to the plane extraction unit PLEX.

FIG. 12 is a flowchart showing a processing procedure in which the plane extraction unit PLEX recognizes an obstacle.

FIG. 13 is a diagram showing plane parameters detected by a plane detector PLEX.

FIG. 14 is a diagram for explaining a conversion process from a camera coordinate system to a foot sole contact plane coordinate system.

FIG. 15 is a diagram showing points on a plane extracted by a plane extraction unit PLEX.

FIG. 16 is a diagram showing that the floor surface is extracted from the robot view, and the coordinates are further converted to express an obstacle in a two-dimensional (floor plane) square.

FIG. 17 is a schematic diagram showing a color landmark in a landmark sensor CLS.

FIG. 18 is a block diagram schematically showing a configuration of a landmark sensor CLS.

FIG. 19 is a schematic diagram showing a self-position probability density distribution on each grid obtained by the Markov localization unit.

FIG. 20 is a flowchart showing an operation of the landmark sensor CLS340.

FIG. 21 is a schematic diagram showing a route from a current position to a destination position on an obstacle map generated by obstacle information.

FIG. 22 is a flowchart showing a route planning algorithm.

FIG. 23 is a perspective view showing an external configuration of a robot device according to an embodiment of the present invention.

FIG. 24 is a diagram schematically showing a degree-of-freedom configuration model of the robot apparatus.

FIG. 25 is a block diagram showing a circuit configuration of the robot apparatus.

FIG. 26 is a block diagram showing a software configuration of the robot apparatus.

FIG. 27 is a block diagram showing a configuration of a middle wear layer in the software configuration of the robot apparatus.

FIG. 28 is a block diagram showing a configuration of an application layer in the software configuration of the robot apparatus.

FIG. 29 is a block diagram showing a configuration of an application layer behavior model library.

FIG. 30 is a diagram illustrating a finite probability automaton that is information for determining the action of the robot apparatus.

FIG. 31 is a diagram showing a state transition table prepared in each node of the finite probability automaton.

[Explanation of symbols]

1 robot device, 250 head unit, 200R,
200L CCD camera, 201R right eye image, 201
L left eye image, 202 color image, 203 parallax image,
210 stereo image processing device, 220 CPU, 23
0 actuator, 231 control signal, 240 sensor, 241 sensor data, 260 trunk, 300 software, 310 kinematic odometry KIN
E310, 320 Plane extraction unit PLEX, 330 Obstacle grid calculation unit OG, 340 Landmark position detection unit CLS, 350 Absolute coordinate calculation unit LZ, 360 Action determination unit SBL, 400, 500, 510, 600, 61
0,700 environment map, 401,501,601,70
1 Center grid, 402, 502, 602 Obstacle grid, 403, 503, 506, 513, 603, 70
3 robot device, 410 grid, 404 information,
900 operation panel

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kenta Kawamoto             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Atsushi Okubo             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Masaki Fukuchi             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Stefan Goodman             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F-term (reference) 3C007 AS36 CS08 KS11 KS12 KS18                       KT03 KT11 MT14 WA03 WA13                       WB16 WB21                 5B050 AA10 BA07 BA17 EA05 EA07                       EA13                 5B057 AA05 CA12 CB12 CD02 CH01                       DA07 DB02 DC08                 5H301 AA01 AA10 BB14 CC03 CC06                       DD01 DD06 DD17 GG09 GG29

Claims (25)

[Claims] 1. A position recognition method for recognizing a position of a moving body moving in a predetermined environment as a position on an environment map corresponding to the predetermined environment, wherein the environment map is a grid of a predetermined size. If the moving area is partitioned and the moving range is within the grid due to the movement of the moving body, the environment map is retained, and if the moving range exceeds the grid, the environment map is moved by the grid unit and updated. A position recognition method characterized by. 2. The position recognition method according to claim 1, wherein the environment map is controlled so that the moving body is located within a central grid of the environment map. 3. The position recognizing method according to claim 1, wherein the direction of the moving body is fixed as the environment map and is recognized as a direction on the environment map. 4. The position recognition method according to claim 1, further comprising the step of changing the size of the environment map and / or the grid. 5. The position recognition method according to claim 1, wherein the movement range is a difference movement amount before and after movement. 6. The position recognition method according to claim 1, further comprising the step of outputting the environment map on an operation panel. 7. The method according to claim 3, wherein when the direction of the moving body is changed along with the movement of the moving body, the direction on the environment map is recognized based on a difference moving angle before and after the movement. Position recognition method. 8. A position recognition device for recognizing the position of a moving body moving in a predetermined environment as a position on the environment map corresponding to the predetermined environment, and a storage means for storing the environment map, A map updating means for updating the environment map according to the movement of the mobile body, wherein the environment map is partitioned by a grid of a predetermined size, and the map updating means moves along with the movement of the mobile body. A position recognition device, characterized in that when the range is within the grid, the environment map is retained, and when the movement range exceeds the grid, the environment map is moved and updated in units of the grid. 9. The position recognition device according to claim 8, wherein the map updating means updates the environment map so that the moving body is located within a central grid of the environment map. 10. The position recognition according to claim 8, wherein the storage unit stores posture information in which the orientation of the moving body is fixed as the environment map and is recognized as a direction on the environment map. apparatus. 11. The position recognition device according to claim 1, further comprising changing means for changing a size of the environment map and / or the grid. 12. The position recognition device according to claim 8, wherein the movement range is a difference movement amount before and after movement. 13. The position recognition device according to claim 8, further comprising output means for outputting the environment map on an operation panel. 14. When the direction of the moving body is changed as the moving body moves, a difference moving angle before and after the movement is input, and the posture information updating means updates the posture information based on the difference moving angle. The position recognition device according to claim 10, wherein 15. A robot device that operates based on supplied input information, and a position for recognizing the position of a moving body moving in a predetermined environment as a position on an environment map corresponding to the predetermined environment. A recognition device is mounted, and the position recognition device has a storage unit that stores the environment map, and a map updating unit that updates the environment map with the movement of the moving body. Partitioned by a grid of a size, the map updating means holds the environment map when the moving range is within the grid as the moving body moves, and the environment map when the moving range exceeds the grid. A robot apparatus for moving and updating in units of the grid. 16. The robot apparatus according to claim 15, wherein the map updating means updates the environment map so that the moving body is located within a central grid of the environment map. 17. The storage means stores posture information in which the direction of the moving body is recognized as a direction on the environment map by fixing the environment map.
Robot device described. 18. The robot apparatus according to claim 15, further comprising changing means for changing the size of the environment map and / or the grid. 19. The robot apparatus according to claim 15, wherein the movement range is a difference movement amount before and after movement. 20. The robot apparatus according to claim 15, further comprising output means for outputting the environment map on an operation panel. 21. The robot apparatus according to claim 15, further comprising a movable leg unit, wherein the map updating means inputs a movement amount each time the movable leg unit moves. 22. A movable leg unit having two left and right leg units is provided, and the map updating means inputs a movement amount each time any one of the leg units moves. The robot apparatus according to claim 15. 23. When the direction of the moving body is changed along with the movement of the moving body, a difference movement angle before and after the movement is input, and the posture information updating means updates the posture information based on the difference movement angle. The robot apparatus according to claim 17, wherein: 24. A program for causing a computer to execute an operation of recognizing a position of a moving body moving in a predetermined environment as a position on an environment map corresponding to the predetermined environment, wherein the environment map is a predetermined one. When the moving range is within the grid, the environment map is retained when the moving body moves, and when the moving range exceeds the grid, the environmental map is divided by the grid unit. A program characterized by moving and updating. 25. A computer-readable recording medium recording a program for causing a computer to execute an operation of recognizing a position of a moving body moving in a predetermined environment as a position on an environment map corresponding to the predetermined environment. The environment map is partitioned by a grid of a predetermined size, and when the moving body moves, the environment map is retained when the moving range is within the grid, and the moving range exceeds the grid. Is a recording medium having a program recorded thereon, which is characterized in that the environment map is moved and updated in units of the grid.