CN111203880A - Data-driven image visual servo control system and method - Google Patents
Data-driven image visual servo control system and method Download PDFInfo
- Publication number
- CN111203880A CN111203880A CN202010045487.0A CN202010045487A CN111203880A CN 111203880 A CN111203880 A CN 111203880A CN 202010045487 A CN202010045487 A CN 202010045487A CN 111203880 A CN111203880 A CN 111203880A
- Authority
- CN
- China
- Prior art keywords
- data
- control
- robot
- historical
- rgb camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
- B25J19/021—Optical sensing devices
- B25J19/023—Optical sensing devices including video camera means
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Multimedia (AREA)
- Manipulator (AREA)
Abstract
The invention provides an image vision servo control system based on data driving. The monocular RGB camera is fixed on an end effector of the six-axis serial robot and is in communication connection with the control host; the six-axis series robot is fixed in a working space and receives an instruction of a robot control cabinet. The robot control cabinet is in communication connection with the control host; the data storage module acquires data from the control host for storage; the data calculation module is in data connection with the data storage module, acquires historical data from the data storage module, and provides the calculated data driving control compensation amount to the control host. The invention also provides a control method of the system, the data calculation amount is small, the calculation operation speed is high, the influence on the real-time performance of the system caused by the calculation amount can be effectively avoided, and the convergence performance of the traditional visual servo is improved by utilizing the optimal weight combination of the historical control data.
Description
Technical Field
The invention relates to the technical field of sensor robot control, in particular to an image vision servo control system and method based on data driving.
Background
At present, mechanical arms are applied to industrial automation. In general, the working mode of the robot arm is always fixed, in other words, the robot arm repeatedly runs pre-compiled instructions in a static structural environment and then realizes corresponding functions. And complex operation is completed in a dynamic uncertain environment, so that the mechanical arm is required to be provided with a corresponding sensor to obtain intelligent sensing capability. The traditional sensor technology generally has the problems of limitation of detection range, singleness of detection means and the like.
The vision sensor is used as a non-contact sensor of the robot, can sense the dynamic change of the external environment of the robot, provides richer information, and has more diversified detection ranges and modes. Therefore, the vision sensor has an important position and role in the field of robots. Visual servoing is the control of the motion of a robot using computer vision feedback data that can be obtained from cameras mounted on the robot or cameras mounted in the environment. Therefore, the visual servo design covers the fields of image processing, computer vision, control theory and the like.
In recent years, with the improvement of computer vision technology, robots are widely applied, and vision servo is rapidly developed. Currently, visual servoing is divided into position-based visual servoing and image-based visual servoing according to feedback information of computer vision in a control system. The mounting method of the camera sensor is divided into visual servo based on eyes on hands and visual servo based on eyes outside the hands. The system can be divided into a monocular vision servo system, a binocular vision servo system and a monocular vision servo system according to the number of cameras.
The position-based visual servo forms a closed-loop control system in a 3D Cartesian space coordinate system, then obtains the position and the posture of an observed object in the Cartesian space coordinate system by using visual information, forms a feedback error with an expected position and posture, designs a control rate according to the error, and drives the motion of the robot. The vision servo based on the image forms a closed-loop control system in a two-dimensional image space, feedback error characteristics are constructed by utilizing image characteristics obtained by image information and expected image characteristics, and a control law is designed according to the errors, so that the feedback motion of the robot is realized. As the name suggests, the visual servo of eyes on hands is to place a camera at the tail end of a robot actuator for observation; the visual servo of the eyes outside the hands is to fix the camera elsewhere and then observe the end effector position and attitude of the robot.
At present, the control method of visual servo is mainly divided into the following three categories, namely, an adaptive algorithm, a robust algorithm and an intelligent algorithm. The adaptive algorithm can be classified into a parameter adaptive method and an image jacobian matrix adaptive method. The two methods are respectively used for identifying and estimating parameters in the model or the image jacobian matrix. The robust visual servo is used for ensuring the stability of the controller under the condition of parameter perturbation, and the robust controller can be designed on the basis of the optimal parameter estimation, so that the stability is ensured in a certain range of parameter change. The intelligent algorithm designs the controller mainly through a neural network, fuzzy control and iterative learning control.
According to literature, a certain research result is found in the aspect of self-adaptive control of the current visual servo, and the method mainly comprises the steps of identifying an image Jacobian matrix or uncertain parameters in the image Jacobian matrix and then designing a feedback control law by using the identified image Jacobian matrix. However, when parameter identification is usually performed, errors still exist between the identified model and the real model to a certain extent, and the problem inhibits the performance of visual servo to a certain extent and reduces the convergence speed of the visual servo; and the identification process has large calculation amount, the running speed of the controller can be reduced, and the real-time performance of the system is reduced.
At present, no explanation or report of the similar technology of the invention is found, and similar data at home and abroad are not collected.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an image visual servo control system and method based on data driving.
The invention is realized by the following technical scheme.
According to an aspect of the present invention, there is provided a data-driven image vision-based servo control system, comprising: the robot comprises a six-axis series robot, a robot control cabinet, a control host, a monocular RGB (red, green and blue) camera, a data storage module, a data calculation module and a vision module. Wherein:
the monocular RGB camera is fixed on an end effector of the six-axis serial robot and is in communication connection with the control host;
the vision module processes image data acquired by the monocular RGB camera, extracts the pixel position of a target and provides feedback information for the control host;
the six-axis serial robot is fixed in a working space, the joint movement of the six-axis serial robot is controlled by a robot control cabinet, and the robot control cabinet is in communication connection with a control host;
the data storage module is in data connection with the control host and is used for acquiring and storing historical data from the control host in real time;
the data calculation module is in data connection with the data storage module and is used for acquiring historical data from the data storage module in real time to calculate, and providing the acquired data drive control compensation quantity to the control host for visual servo control.
Preferably, the data storage module stores the operation data in real time, the data calculation module performs optimization calculation on the acquired historical data, and the result obtained by calculation is used as a visual servo control compensation amount for compensating the visual servo control amount.
Preferably, the specific process of the data calculation module performing optimization calculation on the acquired historical data is as follows:
the data calculation module solves the optimal weight coefficient lambda of the data drive as follows:
in the formula,
for the historical error increment, e (k) is the error between the target pixel at time k and the desired pixel, and T is the matrix transpose operation.
Solving data driven control compensation
Comprises the following steps:
in the formula,
is a historical control quantity.
Preferably, the monocular RGB camera is in communication connection with a control host.
Preferably, the vision module processes image data collected by the camera and extracts pixel positions of the target, and provides feedback information to the control host.
Preferably, the six-axis series robot is fixed in a working space, the joint movement of the six-axis series robot is controlled by a robot control cabinet, and the robot control cabinet is in communication connection with the control host.
Preferably, the data storage module is in data connection with the control host and is used for acquiring and storing historical data from the control host in real time.
Preferably, the data calculation module is in data connection with the data storage module, and is configured to acquire historical data from the data storage module in real time to perform calculation, and provide the obtained data drive control compensation amount to the control host for visual servo control.
According to another aspect of the present invention, there is provided a control method of the data-driven-based image visual servo control system, comprising the steps of:
s1, initializing the system;
s2, at the moment k, the data calculation module acquires historical data from the data storage module and solves the optimal weight coefficient of data drive and the compensation amount of data drive control;
s3, acquiring the error between the current pixel and the expected pixel of the target from the vision module, acquiring the data driving control compensation amount from the data calculation module, adding the data driving control compensation amount into the traditional vision servo feedback control rate, and calculating to generate a new control amount; the control host converts the monocular RGB camera motion into six-axis serial robot joint motion, converts new control quantity into a robot instruction and sends the robot instruction to the robot control cabinet;
s4, sending all data generated at the moment k to a data storage module to wait for the next control; stopping the control if the current pixel of the target reaches the desired pixel; otherwise, return to S2.
Preferably, the initializing the system in S1 includes the following steps:
an initialization data storage module and a data calculation module, comprising: the length of a data rolling window is initialized to be l, and the optimal weight coefficient of the historical control quantity is initialized to be
Monocular RGB camera historical movement speed initialization under rolling window
Historical error delta initialization under rolling window
Initializing a visual servo feedback control rate h, a control period and an image processing period of a control host; acquiring a desired image of a designated tracking object of a system control target, wherein the control target is the designated tracking target in the working space.
Preferably, in S2, the method for solving the data-driven optimal weight coefficient includes:
data storage module provides monocular RGB camera historical movement speed
And historical error increments
Wherein:
in the formula, l is the length of a data rolling window;
the data calculation module calculates the optimal weight coefficient λ of the data drive as follows:
where e (k) is the error between the target pixel at time k and the desired pixel, and T is the matrix transposition operation.
Preferably, in S2, the data-driven control compensation amount is solved
The method comprises the following steps:
wherein h is the visual servo feedback control rate of the control host,
is an estimate of the inverse of the jacobian matrix for the image.
Preferably, in S3, the solving method for generating the new control amount is:
method for solving motion speed V of monocular RGB camera at next moment of current moment by vision servo controller based on data drivingc(k) Comprises the following steps:
in the formula,
for the conventional visual servo control rate, e (k) is the error between the target pixel at the time k and the expected pixel, h is the visual servo feedback control rate of the control host,
for the estimation of the inverse of the jacobian matrix of the image, λ is the optimal weight coefficient for the data drive,
the monocular RGB camera has a historical speed of motion,
is a historical error increment;
obtaining the monocular RGB camera movement speed V at the next moment of the current momentc(k) Then, the speed V is adjustedc(k) Sending the speed V into a data storage module for storage, and storing the speed Vc(k) And converting into the desired joint speed of the six-axis serial robot.
Preferably, in S3, the method for converting the monocular RGB camera motion into the six-axis tandem robot joint motion includes:
according to the positive kinematics of the mechanical arm and the current joint angle of the six-axis tandem robot, solving the coordinate conversion relation between the current base coordinate system and the terminal coordinate system into
The coordinate conversion relation between the coordinate system of the tail end of the six-axis serial robot and the coordinate system of the camera is obtained through hand-eye calibration
Finally, the transformation matrix of the six-axis serial robot base coordinate system and the monocular RGB camera coordinate system is obtained as follows:
in the formula,
in order to be a matrix of rotations,
is a translation vector;
calculating the relationship between the monocular RGB camera movement speed and the six-axis series robot joint movement speed as follows:
in the formula, JbIs a six-axis series robot kinematic jacobian matrix,
the joint speed of the six-axis series robot is shown, theta is the joint angle, delta theta is the joint increment, and I is a unit matrix;
wherein
In the formula, vcFor monocular RGB camera line speed, wcAngular velocity for a monocular RGB camera;
obtaining the six-axis serial robot joint speed corresponding to the monocular RGB camera motion speed
The joint speed is sent to the robot control cabinet.
Compared with the prior art, the invention has the following beneficial effects:
according to the image vision servo control system and method based on data driving, provided by the invention, model errors are eliminated by adding the data driving module, so that the calculated amount in the identification process is avoided, and the control performance of the system is improved. The data driving module adds the optimal weight combination of the historical output data into the traditional control quantity to inhibit the influence of the model error on the control performance, so that the convergence performance of the visual servo is improved, and the convergence speed of the system control is improved.
The image vision servo control system and method based on data driving provided by the invention are additionally provided with a data storage module and a data calculation module, and a data driving compensation module is designed; the data storage module provides effective historical data to the control host; the data calculation module calculates the data driving control rate according to the historical data, and the data driving control rate is used for compensating the traditional visual servo control rate in the control host, so that the data calculation amount is small, the calculation operation speed is high, the influence on the real-time performance of the system due to the calculation amount can be effectively avoided, and the convergence performance of the traditional visual servo is improved by using the optimal weight combination of the historical control data.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic diagram of a data-driven image vision servo control system according to an embodiment of the present invention;
FIG. 2 is a diagram of a checkerboard target provided as a control target in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a data-driven image vision-based servo control system according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating an operation process of a data-driven image vision-based servo control system according to an embodiment of the present invention;
FIG. 5 is a flowchart of a control method of a data-driven image vision-based servo control system according to an embodiment of the present invention;
in the figure, 1 is a monocular RGB camera, 2 is a six-axis tandem robot, 3 is a target feature point on a target object, 4 is a working coordinate system, and 5 is the target object.
Detailed Description
The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
The embodiment of the invention provides an image vision servo control system based on data driving, which comprises: the robot comprises a shaft series robot, a robot control cabinet, a control host, a monocular RGB (red, green and blue) camera, a data storage module, a data calculation module and a vision module. Wherein,
the monocular RGB camera is fixed on an end effector of the six-axis serial robot and is in communication connection with the control host;
the vision module processes image data acquired by the camera, extracts the pixel position of a target and provides feedback information for the control host;
the six-axis serial robot is fixed in a working space, the joint movement of the six-axis serial robot is controlled by a robot control cabinet, and the robot control cabinet is in communication connection with a control host;
the data storage module is in data connection with the control host and is used for acquiring and storing historical data from the control host in real time;
the data calculation module is in data connection with the data storage module and is used for acquiring historical data from the data storage module in real time to calculate, and providing the acquired data drive control compensation quantity to the control host for visual servo control.
Further, the data calculation module performs optimization calculation on the acquired historical data, and the calculated result is a visual servo control compensation amount, and the result is used for compensating the visual servo control amount.
Based on the image visual servo control system based on data driving provided by the embodiment of the invention, the embodiment of the invention also provides a control method of the system, which comprises the following steps:
s1, carrying out the following steps on the systemInitializing; an initialization data storage module and a data calculation module, comprising: the length of a data rolling window is initialized to be l, and the optimal weight coefficient of the historical control quantity is initialized to be
Monocular RGB camera historical movement speed initialization under rolling window
Historical error delta initialization under rolling window
Initializing a visual servo feedback control rate h, a control period and an image processing period of a control host; acquiring a desired image of a designated tracking object of a system control target, wherein the control target is the designated tracking target in the working space.
S2, at the moment k, the data storage module provides historical movement speed of the monocular RGB camera
And historical error increments
The data calculation module calculates the optimal weight coefficient λ of the data drive as follows:
solving data driven control compensation
Comprises the following steps:
s3, solving the list of the monocular RGB camera at the next moment of the current moment based on the visual servo controller driven by the dataMovement velocity V of eye RGB camerac(k) Comprises the following steps:
obtaining the monocular RGB camera movement speed V at the next moment of the current momentc(k) Then, the speed V is adjustedc(k) And sending the data to a data storage module for storage.
According to the positive kinematics of the mechanical arm and the current joint angle of the six-axis tandem robot, solving the conversion matrix of the six-axis tandem robot base coordinate system and the monocular RGB camera coordinate system as follows:
calculating the relationship between the monocular RGB camera movement speed and the six-axis series robot joint movement speed as follows:
obtaining the six-axis serial robot joint speed corresponding to the monocular RGB camera motion speed
The joint velocity is generated to the robot control cabinet.
S4, sending all data generated at the moment k to a data storage module to wait for the next control; stopping the control if the current pixel of the target reaches the desired pixel; otherwise, return to S2.
The technical solutions of the embodiments of the present invention are further described in detail below with reference to specific embodiments and accompanying drawings.
The image visual servo control system based on data driving provided by the embodiment adds the data driving module to eliminate model errors on the basis of previous visual servo work, avoids calculated amount in the identification process, and improves system control performance. The data driving module adds the optimal weight combination of the historical data into the traditional control quantity to inhibit the influence of model errors on the control performance, so that the convergence performance of visual servo is improved, and the calculation speed of the system is improved.
The image vision servo control system based on data driving provided by the specific embodiment has a schematic structural diagram shown in fig. 1, and is implemented by constructing a six-degree-of-freedom six-axis serial robot and a monocular camera, a tracking target is shown in fig. 2, a system structural diagram is shown in fig. 3, a whole working process is shown in fig. 4, and a control method flow chart of the system is shown in fig. 5.
In the image visual servo control system based on data driving provided in this embodiment, the specific implementation process of each module is as follows:
1. vision module
The vision module in this embodiment consists of an industrial monocular RGB camera and its accessories. The resolution is 960 × 1280, the frame number is 30fps/s, USB3.0 communication transmission map is used, and the internal parameters are as follows
The monocular RGB camera vision module detects the checkerboard in real time and feeds pixel positions of the feature points back to the vision servo controller module in real time, and the movement of the monocular RGB camera module is supported by the six-axis serial robot module.
The visual inspection specifically tasks are as follows: setting angular points on a given checkerboard as characteristic points, carrying out real-time information acquisition and tracking on the characteristic points, and feeding back position information of a target to a control host.
2. Six-axis series robot and robot control cabinet
The six-axis series robot is selected as a six-degree-of-freedom series six-axis series robot, and DH parameters of the six-axis series robot are shown in table 1.
The six-axis serial robot end effector is provided with a visual model, and a coordinate system conversion matrix between the visual model and the visual model is
TABLE 1
| i | αi-1 | ai-1 | θi-1 | di-1 |
| 1 | 0° | 0 | θ1 | 0.242m |
| 2 | -90° | 0 | θ2-90° | 0 |
| 3 | 0° | 0.225m | θ3+90° | 0 |
| 4 | 90° | 0 | θ4 | 0.229 |
| 5 | 90° | 0 | -θ5 | 0 |
| 6 | -90° | 0 | θ6 | 0.05m |
The control chip of the robot control cabinet is an embedded STM32, the positions of six-axis serial robot joints are controlled by adjusting PWM waves, and control commands and feedback robot state information are received through serial port communication.
3. Control host, data storage module and data calculation module
And the control host designs a control instruction of the six-axis series robot module based on a traditional visual servo control mode according to the coordinate information fed back by the visual module. In the control process, historical control instructions and error increments are stored in the data storage module for use by the data calculation module. And the data calculation module calls historical data to calculate the optimal weight combination coefficient of the historical control quantity and calculate the compensation quantity of the control rate of the visual servo. And finally, the control host obtains a final output quantity by combining the traditional visual servo control quantity and the data driving compensation quantity, the data driving control quantity is sent to the robot control cabinet, and the six-axis series robot completes corresponding operation according to a control instruction.
The image visual servo control system based on data driving provided by the embodiment of the invention comprises the following steps:
s1, initializing the system; an initialization data storage module and a data calculation module, comprising: the length of a data rolling window is initialized to be l, and the optimal weight coefficient of the historical control quantity is initialized to be
Monocular RGB camera historical movement speed initialization under rolling window
Historical error delta initialization under rolling window
Initializing a visual servo feedback control rate h, a control period and an image processing period of a control host; acquiring a desired image of a designated tracking object of a system control target, wherein the control target is the designated tracking target in the working space.
S2, at the moment k, the data storage module provides historical movement speed of the monocular RGB camera
And historical error increments
The data calculation module calculates the optimal weight coefficient λ of the data drive as follows:
solving data driven control compensation
Comprises the following steps:
s3, solving the movement speed V of the monocular RGB camera at the next moment of the current moment based on the vision servo controller driven by the datac(k) Comprises the following steps:
obtaining the monocular RGB camera movement speed V at the next moment of the current momentc(k) Then, the speed V is adjustedc(k) And sending the data to a data storage module for storage.
According to the positive kinematics of the mechanical arm and the current joint angle of the six-axis tandem robot, solving the conversion matrix of the six-axis tandem robot base coordinate system and the monocular RGB camera coordinate system as follows:
calculating the relationship between the monocular RGB camera movement speed and the six-axis series robot joint movement speed as follows:
obtaining the six-axis serial robot joint speed corresponding to the monocular RGB camera motion speed
The joint velocity is generated to the robot control cabinet.
S4, sending all data generated at the moment k to a data storage module to wait for the next control; stopping the control if the current pixel of the target reaches the desired pixel; otherwise, return to S2.
The image vision servo control system and method based on data driving provided by the above embodiment of the invention, wherein, the monocular RGB camera is fixed on the end effector of the six-axis serial robot and is in communication connection with the control host; the six-axis series robot is fixed in a working space and receives an instruction of a robot control cabinet. The robot control cabinet is in communication connection with the control host; the data storage module acquires data from the control host for storage; the data calculation module is in data connection with the data storage module, acquires historical data from the data storage module, and provides the calculated data driving control compensation amount to the control host. The system base and the method provided by the embodiment of the invention have the advantages of small data calculation amount and high calculation running speed, can effectively avoid the influence on the real-time performance of the system due to the calculation amount, and improve the convergence performance of the traditional visual servo by utilizing the optimal weight combination of the historical control data.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.
Claims (9)
1. A data-driven image vision-based servo control system, comprising: the robot comprises a six-axis series robot, a robot control cabinet, a control host, a monocular RGB (red, green and blue) camera, a data storage module, a data calculation module and a vision module. Wherein:
the monocular RGB camera is fixed on an end effector of the six-axis serial robot and is in communication connection with the control host;
the vision module processes image data acquired by the monocular RGB camera, extracts the pixel position of a target and provides feedback information for the control host;
the six-axis serial robot is fixed in a working space, the joint movement of the six-axis serial robot is controlled by a robot control cabinet, and the robot control cabinet is in communication connection with a control host;
the data storage module is in data connection with the control host and is used for acquiring and storing historical data from the control host in real time;
the data calculation module is in data connection with the data storage module and is used for acquiring historical data from the data storage module in real time to calculate, and providing the calculated data drive control compensation quantity to the control host for visual servo control.
2. The image visual servo control system based on data driving according to claim 1, wherein the data storage module stores the operation data in real time, the data calculation module performs optimization calculation on the acquired historical data, and the obtained result is calculated as a visual servo control compensation amount for compensating the visual servo control amount.
3. The image vision servo control system based on data driving according to claim 2, wherein the data calculation module performs optimization calculation on the acquired historical data by:
the data calculation module solves the optimal weight coefficient lambda of the data drive as follows:
in the formula,
e (k) is the error between the pixel of the target at the time k and the expected pixel, and T is the matrix transposition operation;
solving data driven control compensation
Comprises the following steps:
in the formula,
is a historical control quantity.
4. A control method of a data-driven-based image vision servo control system according to any one of claims 1 to 3, comprising the steps of:
s1, initializing the system;
s2, at the moment k, the data calculation module acquires historical data from the data storage module and solves the optimal weight coefficient of data drive and the compensation amount of data drive control;
s3, acquiring the error between the current pixel and the expected pixel of the target from the vision module, acquiring the data driving control compensation amount from the data calculation module, adding the data driving control compensation amount into the traditional vision servo feedback control rate, and calculating to generate a new control amount; the control host converts the monocular RGB camera motion into six-axis serial robot joint motion, converts new control quantity into a robot instruction and sends the robot instruction to the robot control cabinet;
s4, sending all data generated at the moment k to a data storage module to wait for the next control; stopping the control if the current pixel of the target reaches the desired pixel; otherwise, return to S2.
5. The control method of image visual servo control system based on data driving as claimed in claim 4, wherein the initialization of the system in S1 includes the following steps:
an initialization data storage module and a data calculation module, comprising: the length of a data rolling window is initialized to be l, and the optimal weight coefficient of the historical control quantity is initialized to be
Monocular RGB camera historical movement speed initialization under rolling window
Historical error delta initialization under rolling window
Initializing a visual servo feedback control rate h, a control period and an image processing period of a control host; acquiring a desired image of a designated tracking object of a system control target, wherein the control target is the designated tracking target in the working space.
6. The method for controlling a visual servo control system based on data driven image as claimed in claim 4, wherein in S2, the method for solving the optimal weight coefficient of data driving is:
data storage module provides monocular RGB camera historical movement speed
And historical error increments
Wherein:
in the formula, l is the length of a data rolling window;
the data calculation module calculates the optimal weight coefficient λ of the data drive as follows:
where e (k) is the error between the target pixel at time k and the desired pixel, and T is the matrix transposition operation.
7. The control method of image visual servo control system based on data driving as claimed in claim 4, whereinIn S2, the data-driven control compensation amount is solved
The method comprises the following steps:
wherein h is the visual servo feedback control rate of the control host,
is an estimate of the inverse of the jacobian matrix for the image.
8. The method for controlling a servo control system according to claim 4, wherein in step S3, the solution method for generating new control quantity is:
method for solving motion speed V of monocular RGB camera at next moment of current moment by vision servo controller based on data drivingc(k) Comprises the following steps:
in the formula,
for the conventional visual servo control rate, e (k) is the error between the target pixel at the time k and the expected pixel, h is the visual servo feedback control rate of the control host,
for the estimation of the inverse of the jacobian matrix of the image, λ is the optimal weight coefficient for the data drive,
for the monocular RGB camera historical motion speed,
is a historical error increment;
obtaining the monocular RGB camera movement speed V at the next moment of the current momentc(k) Then, the speed V is adjustedc(k) Sending the speed V into a data storage module for storage, and storing the speed Vc(k) And converting into the desired joint speed of the six-axis serial robot.
9. The control method of image vision servo control system based on data driving of claim 5, wherein in the step S3, the method for converting monocular RGB camera motion into six-axis serial robot joint motion is as follows:
according to the positive kinematics of the mechanical arm and the current joint angle of the six-axis tandem robot, solving the coordinate conversion relation between the current base coordinate system and the terminal coordinate system into
The coordinate conversion relation between the coordinate system of the tail end of the six-axis serial robot and the coordinate system of the camera is obtained through hand-eye calibration
Finally, the transformation matrix of the six-axis serial robot base coordinate system and the monocular RGB camera coordinate system is obtained as follows:
in the formula,
in order to be a matrix of rotations,
is a translation vector;
calculating the relationship between the monocular RGB camera movement speed and the six-axis series robot joint movement speed as follows:
in the formula, JbIs a six-axis series robot kinematic jacobian matrix,
the joint speed of the six-axis series robot is shown, theta is the joint angle, delta theta is the joint increment, and I is a unit matrix;
wherein
In the formula, vcFor monocular RGB camera line speed, wcAngular velocity for a monocular RGB camera;
obtaining the six-axis serial robot joint speed corresponding to the monocular RGB camera motion speed
The joint speed is sent to the robot control cabinet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010045487.0A CN111203880B (en) | 2020-01-16 | 2020-01-16 | Image visual servo control system and method based on data drive |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010045487.0A CN111203880B (en) | 2020-01-16 | 2020-01-16 | Image visual servo control system and method based on data drive |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111203880A true CN111203880A (en) | 2020-05-29 |
| CN111203880B CN111203880B (en) | 2022-12-06 |
Family
ID=70781081
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010045487.0A Active CN111203880B (en) | 2020-01-16 | 2020-01-16 | Image visual servo control system and method based on data drive |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111203880B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113057673A (en) * | 2021-03-11 | 2021-07-02 | 华中科技大学同济医学院附属协和医院 | Robot ultrasonic scanning control method, system, equipment and storage medium |
| CN113601498A (en) * | 2021-07-11 | 2021-11-05 | 西北工业大学 | Robot vision servo and man-machine interaction hybrid control method based on weight distribution |
| CN113681549A (en) * | 2021-08-30 | 2021-11-23 | 江苏塔帝思智能科技有限公司 | Robot grabbing method and system based on three-dimensional data analysis |
| CN115284296A (en) * | 2022-08-31 | 2022-11-04 | 深圳前海瑞集科技有限公司 | Hand-eye calibration method, robot and robot operation method |
| US20220392012A1 (en) * | 2021-06-07 | 2022-12-08 | Alcon Inc. | Optical axis calibration of robotic camera system |
| CN119596833A (en) * | 2024-12-03 | 2025-03-11 | 上海碧帝数据科技有限公司 | State space model-based optimal control method and system for PLC (programmable logic controller) implementation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6430472B1 (en) * | 1999-12-20 | 2002-08-06 | Servo-Robot Inc. | Robot feature tracking devices and methods |
| CN101623867A (en) * | 2008-07-11 | 2010-01-13 | 中国科学院沈阳自动化研究所 | A device and method for enabling a robot to track a specified path with high precision |
| CN107186752A (en) * | 2017-03-28 | 2017-09-22 | 江苏科技大学 | A kind of compensation of undulation fishing robot system |
| CN107627303A (en) * | 2017-09-22 | 2018-01-26 | 哈尔滨工程大学 | A kind of PD SMC control methods based on vision servo system of the eye in structure on hand |
| CN110039542A (en) * | 2019-06-13 | 2019-07-23 | 东北大学 | Visual servo tracking and controlling method and robot system with directional velocity control |
-
2020
- 2020-01-16 CN CN202010045487.0A patent/CN111203880B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6430472B1 (en) * | 1999-12-20 | 2002-08-06 | Servo-Robot Inc. | Robot feature tracking devices and methods |
| CN101623867A (en) * | 2008-07-11 | 2010-01-13 | 中国科学院沈阳自动化研究所 | A device and method for enabling a robot to track a specified path with high precision |
| CN107186752A (en) * | 2017-03-28 | 2017-09-22 | 江苏科技大学 | A kind of compensation of undulation fishing robot system |
| CN107627303A (en) * | 2017-09-22 | 2018-01-26 | 哈尔滨工程大学 | A kind of PD SMC control methods based on vision servo system of the eye in structure on hand |
| CN110039542A (en) * | 2019-06-13 | 2019-07-23 | 东北大学 | Visual servo tracking and controlling method and robot system with directional velocity control |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113057673A (en) * | 2021-03-11 | 2021-07-02 | 华中科技大学同济医学院附属协和医院 | Robot ultrasonic scanning control method, system, equipment and storage medium |
| US20220392012A1 (en) * | 2021-06-07 | 2022-12-08 | Alcon Inc. | Optical axis calibration of robotic camera system |
| US12175556B2 (en) * | 2021-06-07 | 2024-12-24 | Alcon Inc. | Optical axis calibration of robotic camera system |
| CN113601498A (en) * | 2021-07-11 | 2021-11-05 | 西北工业大学 | Robot vision servo and man-machine interaction hybrid control method based on weight distribution |
| CN113601498B (en) * | 2021-07-11 | 2024-02-02 | 西北工业大学 | Hybrid control method of robot visual servoing and human-computer interaction based on weight distribution |
| CN113681549A (en) * | 2021-08-30 | 2021-11-23 | 江苏塔帝思智能科技有限公司 | Robot grabbing method and system based on three-dimensional data analysis |
| CN115284296A (en) * | 2022-08-31 | 2022-11-04 | 深圳前海瑞集科技有限公司 | Hand-eye calibration method, robot and robot operation method |
| CN119596833A (en) * | 2024-12-03 | 2025-03-11 | 上海碧帝数据科技有限公司 | State space model-based optimal control method and system for PLC (programmable logic controller) implementation |
| CN119596833B (en) * | 2024-12-03 | 2025-11-11 | 上海碧帝数据科技有限公司 | State space model-based optimal control method and system for PLC (programmable logic controller) implementation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111203880B (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111203880A (en) | Data-driven image visual servo control system and method | |
| CN111360827B (en) | A visual servo switching control method and system | |
| Zhu et al. | Dual-arm robotic manipulation of flexible cables | |
| CN107767423B (en) | A binocular vision-based target positioning and grasping method for manipulators | |
| CN105014667B (en) | A Relative Pose Calibration Method of Camera and Robot Based on Pixel Space Optimization | |
| Chaumette | Potential problems of stability and convergence in image-based and position-based visual servoing | |
| CN116872216B (en) | Robot vision servo operation method based on finite time control | |
| CN113284179B (en) | Robot multi-object sorting method based on deep learning | |
| CN113715016B (en) | Robot grabbing method, system, device and medium based on 3D vision | |
| CN108789404A (en) | A kind of serial manipulator kinematic calibration method of view-based access control model | |
| CN109358507B (en) | A Visual Servo Adaptive Tracking Control Method with Time-varying Performance Boundary Constraints | |
| CN111203849A (en) | Mobile robot grabbing operation system and control method | |
| CN113733088A (en) | Mechanical arm kinematics self-calibration method based on binocular vision | |
| CN111360821A (en) | Picking control method, device and equipment and computer scale storage medium | |
| CN111331604A (en) | A machine vision-based valve screwing and compliant operation method | |
| CN118721200A (en) | Visual servo control method, device and storage medium for dual-arm collaborative robot | |
| CN109465829A (en) | A Geometric Parameter Identification Method of Industrial Robot Based on Transformation Matrix Error Model | |
| CN111618861A (en) | Double-follow-up intelligent arm control method based on four-axis structure | |
| CN117162094A (en) | Multi-target self-adaptive angle grabbing method of visual servo mechanical arm | |
| Nori et al. | Autonomous learning of 3D reaching in a humanoid robot | |
| Song et al. | On-line stable evolutionary recognition based on unit quaternion representation by motion-feedforward compensation | |
| Cheng et al. | Real-time robot end-effector pose estimation with deep network | |
| CN112706161B (en) | Gluing control system with intelligent sensing capability | |
| CN107414826B (en) | Tendon-driven manipulator tension constraint tail end operation control method | |
| CN119992048A (en) | A target detection method and system based on improved YOLOv11 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |