CN100484726C - Flexible and remote-controlled operation platform for robot based on virtual reality - Google Patents

Abstract

The invention relates to a dexterous hand teleoperation platform based on a virtual reality robot. It is composed of man-machine interface system and telerobot dexterous hand system. The man-machine interface system is a virtual environment operating platform composed of a computer, which forms the operator's operation interface. The remote robot dexterous hand system is controlled by a robot dexterous hand connected to a dexterous hand controller in a working scene, and the dexterous hand controller is connected to a host control computer; a CCD camera is aimed at the robot dexterous hand to capture the position of the dexterous hand and Attitude; the host computer and the CCD camera are all connected to the computer of the virtual environment operating platform by cables. The invention can not only realize remote operation, and make the operator escape from the dangerous operating environment, but also the man-machine interface system can be operated off-line, so as to realize the preview of the whole operation process and the simulation of various control algorithms. The present invention minimizes the control links in design, improves the control response speed, and reduces the hardware and software costs of the system at the same time.

Description

Smart hand teleoperation platform based on virtual reality robot

technical field

The invention relates to a remote operation platform based on a virtual reality robot dexterous hand, which combines remote control technology, virtual reality technology and robot dexterous hand technology to form a virtual reality robot dexterous hand remote operation platform. It can be applied to the field of robot teleoperation, robot teaching and other related fields.

Background technique

At present, there are no commercialized products based on the dexterous hand teleoperation platform for virtual reality robots. In the dexterous hand remote operating system using virtual reality technology, a main problem is involved, which is the way of controlling the dexterous hand. For the research on this problem, there are related research reports at home and abroad. The typical control method is to use data gloves to control the dexterous hand; Movement, measure the coordinates of human fingers and palms, and then map them to the fingers and palms of dexterous hands to achieve the purpose of controlling the robot dexterous hand to grab objects; the third method is to use dual-camera stereo vision technology to capture the operator's hand The gesture action of the robot is mapped to the motion parameters of the robot dexterous hand to control the dexterous hand.

There are many system links, large errors and poor operability in the above three methods. The method adopted in the present invention is to operate the three-dimensional graphics simulation hand in the virtual environment with a common keyboard and mouse, thereby synchronously driving the real dexterous hand to complete various operations. This method not only reduces the control links, improves the control response speed, but also reduces the hardware and software costs of the system.

Contents of the invention

The purpose of the present invention is to provide a dexterous hand remote operation platform based on virtual reality robot. The preview of the whole operation process of grabbing objects, and the simulation of various control algorithms.

In order to achieve the above object, design of the present invention is as follows:

In design, the present invention divides its system into two parts in space: the man-machine interface system and the telerobotic dexterous hand system. As shown in Figure 1.

In the human-machine interface system, the virtual environment operating platform is a computer, which contains the virtual simulation environment of the geometric model and the kinematics model of the dexterous hand, and also contains the image of the operating environment of the telerobot dexterous hand, which reflects the operating site. The real scene enables the operator to grasp the information of the scene and the dexterous hand in a timely and accurate manner, and compare the position and posture of the simulated hand in order to adjust the posture of the dexterous hand in time; the operator platform is the working interface of the operator, through which the dexterous hand can be observed Hand data, input various data of the dexterous hand to control the position and posture of the dexterous hand.

In the remote robot dexterous hand system, it includes a dexterous hand controller and a robot dexterous hand. The dexterous hand is the end manipulator of the robot and is the final object controlled by the system. It will move with the virtual hand at the man-machine interface; The camera is used as the eyes of the robot to capture images of the robot scene and the position and posture of the dexterous hand; the upper control computer is used to collect information about the robot dexterous hand and control information from the human-machine interface system. All kinds of information are fused, and then the grasping strategy of the dexterous hand is determined, and all kinds of information are fed back to the human-machine interface system.

According to above-mentioned inventive concept, the present invention adopts following technical scheme:

A robot dexterous hand teleoperation platform based on virtual realization, which is composed of a man-machine interface system and a telerobot dexterous hand system, and is characterized in that:

(1) The human-machine interface system is a virtual environment operating platform composed of a computer; it forms the operator's operating interface;

(2) The remote robot dexterous hand system is controlled by a robot dexterous hand connected to a dexterous hand controller in a virtual operation scene, and the dexterous hand controller is connected to a host control computer; a CCD camera is aimed at the robot dexterous hand to capture The position and posture of the dexterous hand; the host control computer and the CCD camera are all connected to the computer of the virtual environment operating platform by cables.

In the above-mentioned virtual operation scene, there are objects grasped by the robot dexterous hand and the workbench on which it is placed, as well as markers of the pose and size of the robot dexterous hand and the target object; the robot dexterous hand uses a BH-3 three-finger Nine degrees of freedom robot dexterous hand.

The above-mentioned operation interface always displays the status of each joint of the dexterous hand and the status of the force feedback of the fingertips; the control panel adopts the form of a Windows standard dialog box: the nine joint states of the virtual dexterous hand and the force feedback status editing box are arranged on the top of the control panel , the up and down pointed buttons are added next to the nine joints so that the angle of each joint can be adjusted independently; in the middle of the control panel, an interface for setting the grasped object is designed, and in this part of the interface there are three check boxes to be selected respectively There are three different objects to be grasped: cube, ball, or cylinder. There is a size and pose button. Pressing this button will pop up another interface for setting the position and size of the grasped object; it is designed at the bottom of the control panel A group of check boxes and a group of buttons, the group of check boxes is used to select the control mode and add control functions; this group of buttons is to operate the dexterous hand in the manual mode, and its operations include parallel grasping, centering Grab, tweezer and pre-grab.

Compared with the prior art, the present invention has the following obvious outstanding substantive features and advantages: In the present invention, a computer constitutes a virtual environment operation platform, which is composed of a robot dexterous hand, a dexterous hand controller, a host computer and a CCD camera, With the required software, it has the following functions:

1. The direct control method is used to realize the remote operation of the robot dexterous hand. The operator directly controls the dexterous hand through the mouse and keyboard, so that the dexterous hand can complete various grasping tasks.

2. Using the teaching method to remotely control the dexterous hand, first the operator "teaches" the dexterous hand to complete a certain grasping task, and then the dexterous hand automatically completes the same task.

3. Use an autonomous method to control the virtual dexterous hand, so that the virtual dexterous hand can automatically generate a grasping plan according to the geometric characteristics and pose of the grasped object, and perform a preview of the grasping process.

Therefore, the present invention can not only realize remote operation, so that the operator is out of the dangerous operating environment, but also the man-machine interface system can be operated offline, so as to realize the preview of the whole operation process and the simulation of various control algorithms. The invention minimizes the control links in design, improves the control response speed, and reduces the hardware and software costs of the system at the same time.

Description of drawings

Fig. 1 is a schematic diagram of the hardware structure of the present invention.

Fig. 2 is a schematic diagram of the hardware connection of the present invention.

Figure 3 The photo map of the virtual operation scene.

Figure 4 Photogram of the robotic dexterous hand.

Figure 5 Control Panel Diagram.

Figure 6 is the setting interface diagram of the grasped object.

Figure 7 (a~c) Photos of the capture method.

Detailed ways

A preferred embodiment of the present invention is described in detail as follows in conjunction with accompanying drawing:

The dexterous hand remote operation platform based on the virtual reality robot is shown in Figure 2: the virtual operation platform computer is used as the human-machine interface system. The remote robot dexterous hand system consists of a host control computer, a BH-3 three-finger nine-degree-of-freedom robot dexterous hand of Beihang University, its controller, and a camera. In the remote robot dexterous hand system, the mechanical part of the dexterous hand can be installed on any mechanical arm, and the electrical part is connected with the dexterous hand controller by a connection line, which is composed of three strands of 25-wire cables; The cable is connected with the upper control computer, and the connection is composed of four 25-wire cables; the cable of the camera CCD is directly connected to the USB interface of the virtual environment operation platform computer; the upper control in the virtual environment operation platform computer and the telerobot dexterous hand system The computer is inserted into the respective RS-232 interface through the connection line, and the remote control distance is within tens of meters.

In terms of software design, under the programming environment of VC++6.0, the software interface of OpenGL, a professional 3D graphics hardware, is used to complete the design of the following functions:

1. Design of virtual environment

The virtual work scene is mainly composed of a virtual robot dexterous hand and the grasped target. In order to make the whole virtual work scene more "real", elements such as the working platform and environmental background are added to the virtual work scene, as shown in Figure 3. The virtual dexterous hand is modeled on the BH-3 three-finger nine-degree-of-freedom robot dexterous hand (as shown in Figure 4) of Beihang University. The virtual dexterous hand consists of a palm rest and three fingers with three knuckles each. The operating platform provides a variety of grasping targets, as well as their size and pose.

In order to enable the operator to understand the virtual dexterous hand’s posture, collision and other information more effectively and intuitively during the operation, the operator can use the mouse and keyboard to rotate, translate and zoom the entire virtual operation scene, and the virtual dexterous hand nine The rotation angle of each joint will be displayed in the virtual operation scene in real time.

2. Design of operation interface

The operation interface is the human-computer interaction interface between the operator and the entire remote operation platform. In order to accommodate more operation items, the entire operation interface is divided into two levels; The status of each joint and the status of fingertip force feedback are always displayed on the operation interface.

The control panel adopts the form of Windows standard dialog box, as shown in Figure 5. On the top of the control panel, there are nine joint states and force feedback state editing boxes of the virtual dexterous hand, and up and down pointed buttons are added next to the nine joints so that the angle of each joint can be adjusted independently.

In the middle of the control panel, an interface for setting the grasped object is designed. In this part of the interface, there are three check boxes to select 3 different grasped objects (cube, ball, cylinder), and there is a size and pose button. Pressing this button will pop up another interface for setting the object to be grasped, as shown in Figure 6.

The lower part of the control panel is designed - a set of check boxes and a set of buttons. This group of check boxes is mainly used to select the control mode and add control functions, such as teaching function and online mode. A group of buttons is mainly for the operation of dexterous hands in manual mode, they have parallel grabbing, centering grabbing, tweezers grabbing and pre-grabbing and so on.

3. Design of operation function

The design of the operation function is mainly carried out around the content of the present invention, that is, the operation platform can not only simulate the intelligent grasping of objects by the dexterous hand in the offline state, but also control and operate the actual dexterous hand in the online state. , and the robotic dexterous hand after teaching completes the automatic grasping operation.

When the dexterous hand grasps an object, it must constantly change the grasping method and path according to the shape, size, and pose of the grasped object. Based on the experience of human grasping objects and a large number of experiments of BH-3 dexterous hand grasping objects, three basic grasping methods are summarized, namely parallel grasping, centered grasping and tweezers grasping. As shown in Figure 7a~c.

The simulation drill function of the dexterous hand intelligent grasping process is realized by using the intelligent control algorithm of fuzzy logic. According to the shape, size and posture of the target, the program selects the corresponding path and grasping method to complete the grasping of the target. The design of this function provides operators with the path and scheme of grabbing objects, highlighting that virtual reality technology provides flexibility, pre-operation and demonstration for the operation of the entire system, and can also provide experiments for various intelligent control algorithms platform.

The teaching and grasping function means that in the online state, the operator selects the corresponding path and grasping method according to the shape, size, and posture of the target object to complete the grasping of the target object. During the operator's operation, the program records each step of the operator's operation in order. Then return the dexterous hand to its original position, press the teaching and playback button, and the program will automatically complete the task of grabbing the target.

4. The force feedback of the dexterous hand grasping the object and the detection of the virtual hand colliding with the object

In the virtual reality robot dexterous hand teleoperation platform, how does the operator judge whether the dexterous hand has grasped the target object? Mainly through two ways, one is prompted by the collision detection between the virtual hand and the virtual object in the virtual environment. The second is to obtain the gripping force of the dexterous hand through the three-dimensional force sensor installed on the tip of the dexterous finger.

The collision detection in the virtual environment not only has a good effect in the rehearsal of the dexterous hand grasping the object, but also provides a good warning when the system is operated online.

Claims (2)

1. A robot dexterous hand teleoperation platform based on virtual reality, consisting of a man-machine interface system and a telerobotic dexterous hand system, is characterized in that: (1) The man-machine interface system is a virtual environment operating platform composed of a computer (1); it forms the operator's operating interface; (2) The remote robot dexterous hand system is controlled by a robot dexterous hand (4) connected to a dexterous hand controller (5) in a virtual operation scene, and the dexterous hand controller (5) is connected to a host control computer (2 ); a CCD camera (6) is arranged to aim at the robot dexterous hand (4) to take the position and attitude of the dexterous hand; the upper control computer (2) and the CCD camera (6) are all connected to the virtual environment operating platform by cables computer(1); (3) The state of each joint of the dexterous hand and the state of fingertip force feedback are always displayed on the operation interface; the control panel adopts the form of a Windows standard dialog box: the nine joint states and force of the virtual dexterous hand are arranged on the top of the control panel. Feedback state editing box, add up and down pointed buttons next to the nine joints so that the angle of each joint can be adjusted independently; in the middle of the control panel, an interface for setting the grasped object is designed, and there are three checks in this part of the interface The box selects three different objects to be grasped: cube, ball, or cylinder, and there is a button of size and pose. Pressing this button will pop up another interface for setting the position and size of the object to be grasped; A group of check boxes and a group of buttons are designed at the bottom of the panel. This group of check boxes is used to select the control mode and add control functions; Grab, Focus Grab, Tweezer Grab and Pre-Grip. 2. The teleoperation platform based on virtual reality robot dexterous hand according to claim 1, in the virtual operation scene, there is a target (7) grabbed by the robot dexterous hand and a workbench (8) for placing it, and a robot The dexterous hand (4) and the target object (7) pose size marker (9); the robot dexterous hand adopts the BH-3 three-finger nine-degree-of-freedom robot dexterous hand.

Images