TWI845450B - 3d object outline data establishment system based on robotic arm and method thereof - Google Patents

Abstract

A 3D object outline data establishment system based on robotic arm and a method thereof are provided. Object flange motion matrix with observation object and flange end is established by object outline calculation device. Image parameters of image device is received by object outline calculation device from image device. Arm space coordinate and flange arm motion matrix are received by object outline calculation device from control device. Arm image motion matrix is calculated according to arm spatial coordinate and spatial coordinate by object outline calculation device. Object image motion matrix is calculated multiply object flange motion matrix, flange arm motion matrix, and arm image motion matrix. Use the triangulation method to calculate the projected coordinate points of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the object image motion matrix with the internal parameters of the image device, so as to calculate the contour image of the observed object on the two-dimensional plane. Therefore, the efficiency of providing convenient 3D object outline data creation may be achieved.

Description

Three-dimensional object contour data establishment system and method based on robotic arm

A data establishment system and method thereof, in particular, a system and method thereof for establishing three-dimensional object contour data by moving an observed object to different observation positions through a robotic arm.

With the development of artificial intelligence, artificial intelligence technology is also widely used in various fields. Artificial intelligence can also be applied to the contour recognition of three-dimensional objects. To use artificial intelligence to recognize the contour of three-dimensional objects, it is necessary to first provide artificial intelligence training data. Only after training can artificial intelligence accurately recognize the contour of three-dimensional objects.

The existing training data of artificial intelligence needs to be established by placing three-dimensional objects at different positions in a fixed scene. However, the placement of three-dimensional objects at different positions in a fixed scene is mostly done manually, which will make the training data of artificial intelligence need to spend time on manual placement of three-dimensional objects, which will be not conducive to the establishment of training data of artificial intelligence.

In summary, it can be seen that the existing artificial intelligence training data has long been a problem that it takes too long to establish, so it is necessary to propose improved technical means to solve this problem.

In view of the problem that the establishment of training data for artificial intelligence in the prior art is too time-consuming, the present invention discloses a system and method for establishing three-dimensional object contour data based on a robot arm, wherein:

The first embodiment of the present invention discloses a three-dimensional object contour data establishment system based on a robotic arm, which includes: an imaging device, a robotic arm, an observation object, a control device, and an object contour calculation device.

The imaging device has a fixed imaging range; the robot arm has a flange end; and the observed object is fixed on the flange end.

The control device is connected to the robot arm to control the movement and rotation of the flange end of the robot arm, so that the observation object fixed on the flange end moves and rotates to different sampling points. The position of the observation object moving and rotating to different sampling points is located in the image range, and the flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation is calculated.

The object contour calculation device establishes a connection with the control device and the imaging device, receives the image parameters of the imaging device from the imaging device, establishes the object flange motion matrix of the observed object and the flange end, receives the spatial coordinates of the imaging device, receives the arm spatial coordinates and the flange arm motion matrix from the control device, calculates the arm image motion matrix according to the arm spatial coordinates and the spatial coordinates, multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate the object image motion matrix, and uses the object image motion matrix with the internal parameters of the imaging device to use triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane to calculate the contour image of the observed object on the two-dimensional plane.

The first embodiment of the present invention discloses a method for establishing three-dimensional object contour data based on a robot arm, which comprises the following steps:

First, the imaging device has a fixed imaging range; then, the robot arm has a flange end; then, the observation object is fixed to the flange end; then, the control device establishes a connection with the robot arm, and the control device controls the movement and rotation of the flange end of the robot arm, so that the observation object fixed to the flange end moves and rotates to different sampling points, and the position of the observation object moving and rotating to different sampling points is located in the imaging range; then, the control device calculates the flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation; then, the object contour calculation device establishes a connection with the control device and the imaging device; then, the object contour calculation device receives the image parameters of the imaging device from the imaging device; then, the object contour calculation device establishes an observation The object and the object flange motion matrix at the flange end; then, the object contour calculation device receives the spatial coordinates of the imaging device; then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device; then, the object contour calculation device calculates the arm image motion matrix according to the arm spatial coordinates and the spatial coordinates; then, the object contour calculation device multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate the object image motion matrix; finally, the object contour calculation device uses the triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object in the two-dimensional plane by combining the object image motion matrix with the internal parameters of the imaging device to calculate the contour image of the observed object in the two-dimensional plane.

The second embodiment of the present invention discloses a three-dimensional object contour data establishment system based on a robot arm, which includes: an observed object, a robot arm, an imaging device, a control device, and an object contour calculation device.

The observation object is fixedly placed; the robot arm has a flange end; the imaging device is fixed to the flange end and has an imaging range.

The control device is connected to the robot arm to control the movement and rotation of the flange end of the robot arm, so that the imaging device fixed on the flange end moves and rotates to different sampling points, the object is observed to be located in the imaging range of the different sampling points, and the flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after the movement and rotation is calculated.

The object contour calculation device establishes a connection with the control device and the imaging device, receives the image parameters of the imaging device from the imaging device, establishes an image flange motion matrix between the imaging device and the flange end, receives the object space coordinates of the observed object, receives the arm space coordinates and the flange arm motion matrix from the control device, calculates the arm object motion matrix according to the arm space coordinates and the object space coordinates, multiplies the image flange motion matrix, the flange arm motion matrix and the arm object motion matrix to calculate the image object motion matrix, and uses the triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane with the image object motion matrix and the internal parameters of the imaging device to calculate the contour image of the observed object on the two-dimensional plane.

The second embodiment of the present invention discloses a method for establishing three-dimensional object contour data based on a robot arm, which comprises the following steps:

First, the object to be observed is fixedly placed; then, the robot arm has a flange end; then, the imaging device is fixed to the flange end and has an imaging range; then, the control device establishes a connection with the robot arm to control the movement and rotation of the flange end of the robot arm, so that the imaging device fixed to the flange end moves and rotates to different sampling points, and the object to be observed is located in the imaging range of the movement and rotation to different sampling points; then, the control device calculates the flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after the movement and rotation; then, the object contour calculation device establishes a connection with the control device and the imaging device; then, the object contour calculation device receives the image parameters of the imaging device from the imaging device; then, the object contour calculation device establishes a relationship between the imaging device and the control device. The image flange motion matrix at the flange end; then, the object contour calculation device receives the object space coordinates of the observed object; then, the object contour calculation device receives the arm space coordinates and the flange arm motion matrix from the control device; then, the object contour calculation device calculates the arm object motion matrix according to the arm space coordinates and the object space coordinates; then, the object contour calculation device multiplies the image flange motion matrix, the flange arm motion matrix and the arm object motion matrix to calculate the image object motion matrix; finally, the object contour calculation device uses the triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the image object motion matrix with the internal parameters of the imaging device, so as to calculate the contour image of the observed object on the two-dimensional plane.

The system and method disclosed in the present invention are as described above. An object contour calculation device establishes an object flange motion matrix of an observed object and a flange end, receives image parameters of an imaging device from an imaging device, receives arm spatial coordinates and a flange-arm motion matrix from a control device, calculates an arm image motion matrix based on the arm spatial coordinates and the spatial coordinates, multiplies the object flange motion matrix, the flange-arm motion matrix and the arm image motion matrix to calculate an object image motion matrix, and uses triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on a two-dimensional plane using the object image motion matrix in conjunction with the internal parameters of the imaging device, so as to calculate a contour image of the observed object on a two-dimensional plane.

Through the above-mentioned technical means, the present invention can achieve the technical effect of providing convenient three-dimensional object contour data establishment.

The following will be used in conjunction with drawings and embodiments to explain the implementation of the present invention in detail, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

The following will first explain the first implementation of the three-dimensional object contour data establishment system based on a robot arm disclosed in the present invention, and please refer to "Figure 1", which is a system block diagram of the first implementation of the three-dimensional object contour data establishment system based on a robot arm of the present invention.

The first embodiment of the present invention discloses a three-dimensional object contour data establishment system based on a robot arm, which includes: an imaging device 10, a robot arm 20, an observation object 30, a control device 40 and an object contour calculation device 50.

Please refer to "Figure 2", which is a schematic diagram of the image range and flange end movement of the first implementation of the present invention based on the three-dimensional object contour data of the robot arm.

The imaging device 10 has a fixed imaging range 11, and the robot arm 20 has a flange end 21. The robot arm 20 used in the present invention is a six-axis serial robot arm, which is only used as an example here, and the application scope of the present invention is not limited thereto. The observed object 30 can be fixed to the flange end 21 of the robot arm 20 by screwing, clamping, magnetic attraction, etc., which is only used as an example here, and the application scope of the present invention is not limited thereto.

The control device 40 and the robot arm 20 are connected to each other via a wired transmission method or a wireless transmission method. The aforementioned wired transmission method is, for example, a cable network, an optical fiber network, etc., and the aforementioned wireless transmission method is, for example, Wi-Fi, a mobile communication network (for example, 3G, 4G, 5G, etc.), etc. These are merely examples for illustration and are not intended to limit the scope of application of the present invention.

The control device 40 controls the position and angle of the flange end 21 of the robot arm 20, that is, the control device 40 generates control instructions for each axis of the robot arm 20 to control the rotation speed and rotation angle of each axis of the robot arm 20, and then controls the movement and rotation of the flange end 21 of the robot arm 20, so that the observation object 30 fixed on the flange end 21 of the robot arm 20 moves and rotates to different sampling points, and the position of the observation object 30 moving and rotating to different sampling points is located in the image range 11, that is, the observation object 30 is fixed on the flange end 21 of the robot arm 20 and produces different postures through the movement and rotation of the robot arm 20.

The control device 40 then calculates the flange arm motion matrix from the flange end 21 of the robot arm 20 to the arm space coordinates of the robot arm 20 after movement and rotation. (B represents the robot arm 20, and E represents the flange end 21). The arm space coordinates of the aforementioned robot arm 20 are the coordinate points where the robot arm 20 is fixed to the ground.

It is worth noting that the flange arm motion matrix is a 4x4 transformation matrix derived from the 6-DOF posture [x, y, z, r, p, w] of the robot arm 20. The control device 40 generally obtains the flange arm motion matrix by reading the controller parameters of the robot arm 20. as follows:

Among them, ca = cos(a), sa = sin(a), cb = cos(b), sb = sin(b), cc = cos(c), sc = sin(c), a = r*π/180, b = p*π/180, c = w*π/180.

The object contour calculation device 50, the imaging device 10, and the control device 40 are connected to the robot arm 20 via a wired transmission method or a wireless transmission method. The aforementioned wired transmission method is, for example, a cable network, an optical fiber network, etc., and the aforementioned wireless transmission method is, for example, Wi-Fi, a mobile communication network (for example, 3G, 4G, 5G, etc.), etc. These are merely examples for illustration and are not intended to limit the scope of application of the present invention.

The object contour calculation device 50 receives internal parameters of the imaging device 10 from the imaging device 10. The internal parameters of the imaging device 10 include, for example, focal length, aperture, sensitivity, etc. These are merely examples for illustration and are not intended to limit the scope of application of the present invention.

The object contour calculation device 50 receives the spatial coordinates of the imaging device 10 through the user interface, and then receives the arm spatial coordinates and the flange arm motion matrix from the control device 40. The object contour calculation device 50 can calculate the arm imaging motion matrix according to the arm spatial coordinates and the spatial coordinates.

It is worth noting that the arm image motion matrix (C represents the imaging device 10) is obtained through hand-to-eye calibration, that is, solving , the above formula is ,in , , , , The transformation matrix from the observed object 30 to the imaging device 10 is obtained by the 2D-3D corresponding point PnP posture estimation method. That is calculated by the following formula:

in, is a 3x3 rotation matrix, is a 3x1 displacement matrix.

The object contour calculation device 50 establishes the object flange motion matrix of the observed object 30 and the flange end 21 of the robot arm 20. The object contour calculation device 50 establishes the object flange motion matrix of the observed object 30 and the flange end 21 of the robot arm 20 by the following formula: (O indicates observation object 30):

= , is a 3x3 rotation matrix, is a 3x1 displacement matrix.

in, Obtained using 6-DOF pose marking software, That is the arm image motion matrix. It is worth noting that the object flange motion matrix of the observed object 30 and the flange end 21 of the robot arm 20 is a fixed matrix and only needs to be calculated once.

The object contour calculation device 50 calculates the object image motion matrix by the following formula: :

That is, the object contour calculation device 50 multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate the object image motion matrix.

The object contour calculation device 50 then uses triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object 30 on the two-dimensional plane by combining the object image motion matrix with the internal parameters of the imaging device 10, so as to calculate the contour image of the observed object 30 on the two-dimensional plane.

Please refer to "Figure 3A" and "Figure 3B", "Figure 3A" is a schematic diagram of a three-dimensional object observation at the first sampling point of the first implementation state of the present invention established based on the three-dimensional object contour data of the robot arm; "Figure 3B" is a schematic diagram of a two-dimensional object contour observed at the first sampling point of the first implementation state of the present invention established based on the three-dimensional object contour data of the robot arm.

Through the above process, the object contour calculation device 50 calculates and converts the observed object 30 at the first sampling point in "Figure 3A" into a two-dimensional plane contour image 31. Please refer to "Figure 3B" for the two-dimensional plane contour image 31 of the observed object 30 at the first sampling point.

Please refer to "Figure 4A" and "Figure 4B", "Figure 4A" is a schematic diagram of a three-dimensional object observation at the second sampling point of the first implementation state of the present invention established based on the three-dimensional object contour data of the robot arm; "Figure 4B" is a schematic diagram of a two-dimensional object contour observed at the second sampling point of the first implementation state of the present invention established based on the three-dimensional object contour data of the robot arm.

Through the above process, the object contour calculation device 50 calculates and converts the observed object 30 at the second sampling point in "Figure 4A" into a two-dimensional plane contour image 31. Please refer to "Figure 4B" for the two-dimensional plane contour image 31 of the observed object 30 at the second sampling point.

Next, the operation method of the present invention will be described below, and please refer to "Figure 5A" and "Figure 5B" at the same time. "Figure 5A" and "Figure 5B" are method flow charts of the first implementation of the method for establishing three-dimensional object contour data based on a robot arm of the present invention.

The first embodiment of the present invention discloses a method for establishing three-dimensional object contour data based on a robot arm, which comprises the following steps:

First, the imaging device has a fixed imaging range (step 601); then, the robot arm has a flange end (step 602); then, the observation object is fixed to the flange end (step 603); then, the control device establishes a connection with the robot arm, and the control device controls the movement and rotation of the flange end of the robot arm, so that the observation object fixed to the flange end moves and rotates to different sampling points, and the observation object moves and rotates to different The position of the sampling point is located in the image range (step 604); then, the control device calculates the flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation (step 605); then, the object contour calculation device establishes a connection with the control device (step 606); then, the object contour calculation device establishes the internal parameters of the imaging device (step 607); then, the object contour calculation device establishes the observed object The object flange motion matrix of the flange end is obtained by the object contour calculation device (step 608); then, the object contour calculation device receives the spatial coordinates of the imaging device (step 609); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 610); then, the object contour calculation device calculates the arm imaging motion matrix according to the arm spatial coordinates and the spatial coordinates (step 611); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 612); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 613); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 614); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 615); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 616); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 617); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 618); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 619); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 620); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 621); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 622); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 623); then, the object contour calculation device receives the arm spatial coordinates and the flange arm motion matrix from the control device (step 624); then, the object contour calculation device receives the arm spatial The calculation device multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate the object image motion matrix (step 612); finally, the object contour calculation device uses the triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the object image motion matrix with the internal parameters of the imaging device to calculate the contour image of the observed object on the two-dimensional plane (step 613).

The following is an explanation of the second embodiment of the present invention, which is based on a robot arm to establish a three-dimensional object contour data system. Please refer to "Figure 6", which shows the image range and flange end movement diagram of the second embodiment of the present invention based on the three-dimensional object contour data of the robot arm. The difference between the second embodiment and the first embodiment is that the observed object 30 is fixedly placed, and the imaging device 10 can be fixed to the flange end 21 of the robot arm 20 by screwing, clamping, magnetic attraction, etc. This is only an example, and the scope of application of the present invention is not limited thereto.

The control device 40 controls the position and angle of the flange end 21 of the robot arm 20, so that the imaging device 10 fixed on the flange end 21 of the robot arm 20 moves and rotates to different sampling points. When the imaging device 10 moves and rotates to different sampling points, the observed object 30 is still located in the image range 11 of the imaging device 10 that moves and rotates to different sampling points. That is, the imaging device 10 is fixed on the flange end 21 of the robot arm 20, and the observed object 30 is photographed at different viewing angles by the movement and rotation of the robot arm 20 to obtain images including different orientations of the observed object 30. The control device 40 then calculates the arm flange motion matrix from the flange end 21 of the robot arm 20 to the arm space coordinates of the robot arm 20 after the movement and rotation. .

It is worth noting that the arm flange motion matrix is a 4x4 transformation matrix derived from the 6-DOF posture [x, y, z, r, p, w] of the robot arm 20. The control device 40 generally obtains the arm flange motion matrix by reading the controller parameters of the robot arm 20. as follows:

.

The object contour calculation device 50 receives the object space coordinates of the observed object 30 through the user interface, and then receives the arm space coordinates and the arm flange motion matrix from the control device 40. The object contour calculation device 50 can calculate the object arm motion matrix according to the arm space coordinates and the object space coordinates. , object arm motion matrix The following formula:

In the second embodiment, since the observation object 30 is fixed, no matter how the imaging device 10 moves and rotates, the object arm motion matrix between the observation object 30 and the arm space coordinates of the robot arm 20 is The object contour calculation device 50 establishes the flange image motion matrix of the imaging device 10 and the flange end 21 through hand-eye calibration. .

The object contour calculation device 50 converts the flange image motion matrix , Arm flange motion matrix and the object arm motion matrix Multiply to calculate the image object motion matrix , image object motion matrix This is the following formula:

Next, the operation method of the present invention will be described below, and please refer to "Figure 7A" and "Figure 7B" at the same time. "Figure 7A" and "Figure 7B" are method flow charts of the second implementation of the method for establishing three-dimensional object contour data based on a robot arm of the present invention.

The second embodiment of the present invention discloses a method for establishing three-dimensional object contour data based on a robot arm, which comprises the following steps:

First, the observation object is fixedly placed (step 701); then, the robot arm has a flange end (step 702); then, the imaging device is fixed to the flange end and has an imaging range (step 703); then, the control device establishes a connection with the robot arm to control the movement and rotation of the flange end of the robot arm, so that the imaging device fixed to the flange end moves and rotates to different sampling points, and the observation object is located at the different sampling points. The image range of the point (step 704); then, the control device calculates the arm flange motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation (step 705); then, the object contour calculation device establishes a connection with the control device and the imaging device (step 706); then, the object contour calculation device receives the image parameters of the imaging device from the imaging device (step 707); then, the object contour calculation device establishes an image The device and the flange image motion matrix of the flange end (step 708); then, the object contour calculation device receives the object space coordinates of the observed object (step 709); then, the object contour calculation device receives the arm space coordinates and the arm flange motion matrix from the control device (step 710); then, the object contour calculation device calculates the object arm motion matrix based on the arm space coordinates and the object space coordinates (step 711); then, the object The contour calculation device multiplies the flange image motion matrix, the arm flange motion matrix and the object arm motion matrix to calculate the image object motion matrix (step 712); finally, the object contour calculation device uses triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the image object motion matrix with the internal parameters of the imaging device to calculate the contour image of the observed object on the two-dimensional plane (step 713).

In summary, it can be seen that the present invention establishes an object flange motion matrix of the observed object and the flange end by an object contour calculation device, receives image parameters of the imaging device from the imaging device, receives arm spatial coordinates and flange-arm motion matrix from the control device, calculates the arm image motion matrix according to the arm spatial coordinates and spatial coordinates, multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate the object image motion matrix, and uses triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane using the object image motion matrix in conjunction with the internal parameters of the imaging device to calculate the contour image of the observed object on the two-dimensional plane.

This technical means can solve the problem of the time-consuming establishment of existing artificial intelligence training data in previous technologies, thereby achieving the technical effect of providing convenient three-dimensional object contour data establishment.

Although the implementation methods disclosed in the present invention are as above, the above contents are not used to directly limit the scope of patent protection of the present invention. Any person with ordinary knowledge in the technical field to which the present invention belongs can make slight changes in the form and details of implementation without departing from the spirit and scope disclosed in the present invention. The scope of patent protection of the present invention shall still be defined by the scope of the attached patent application.

10: imaging device 11: imaging range 20: robotic arm 21: flange end 30: observation object 31: contour image 40: control device 50: object contour calculation device Step 601: imaging device has a fixed imaging range Step 602: robotic arm has a flange end Step 603: the observation object is fixed at the flange end Step 604: the control device establishes a connection with the robotic arm, and the control device controls the movement and rotation of the flange end of the robotic arm, so that the observation object fixed at the flange end moves and rotates to different sampling points, and the position where the observation object moves and rotates to different sampling points is located in the imaging range Step 605: The control device calculates the flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation Step 606: The object contour calculation device establishes a connection with the control device Step 607: The object contour calculation device establishes the internal parameters of the imaging device Step 608: The object contour calculation device establishes the object flange motion matrix of the observed object and the flange end Step 609: The object contour calculation device receives the space coordinates of the imaging device Step 610: The object contour calculation device receives the arm space coordinates and the flange arm motion matrix from the control device Step 611: The object contour calculation device calculates the arm image motion matrix based on the arm space coordinates and the space coordinates Step 612: The object contour calculation device multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate the object image motion matrix Step 613: The object contour calculation device uses triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the object image motion matrix with the internal parameters of the imaging device to calculate the contour image of the observed object on the two-dimensional plane Step 701: The observed object is fixedly placed Step 702: The robot arm has a flange end Step 703: The imaging device is fixed to the flange end and has an imaging range Step 704: The control device establishes a connection with the robot arm to control the movement and rotation of the flange end of the robot arm, so that the imaging device fixed to the flange end moves and rotates to different sampling points, and the object is observed to be located in the image range of the movement and rotation to different sampling points. Step 705: The control device calculates the arm flange motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation. Step 706: The object contour calculation device establishes a connection with the control device and the imaging device. Step 707: The object contour calculation device receives the image parameters of the imaging device from the imaging device. Step 708: The object contour calculation device establishes the flange image motion matrix of the imaging device and the flange end. Step 709: The object contour calculation device receives the object space coordinates of the observed object Step 710: The object contour calculation device receives the arm space coordinates and the arm flange motion matrix from the control device Step 711: The object contour calculation device calculates the object arm motion matrix based on the arm space coordinates and the object space coordinates Step 712: The object contour calculation device multiplies the flange image motion matrix, the arm flange motion matrix and the object arm motion matrix to calculate the image object motion matrix Step 713: The object contour calculation device uses triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the image object motion matrix with the internal parameters of the imaging device, so as to calculate the contour image of the observed object on the two-dimensional plane.

FIG. 1 is a system block diagram of the first implementation of the system for establishing the three-dimensional object contour data of the robot arm according to the present invention. FIG. 2 is a schematic diagram of the image range and the movement of the flange end according to the first implementation of the system for establishing the three-dimensional object contour data of the robot arm according to the present invention. FIG. 3A is a schematic diagram of the first sampling point three-dimensional observation of the object according to the first implementation of the system for establishing the three-dimensional object contour data of the robot arm according to the present invention. FIG. 3B is a schematic diagram of the first sampling point two-dimensional observation of the object contour according to the first implementation of the system for establishing the three-dimensional object contour data of the robot arm according to the present invention. FIG. 4A is a schematic diagram of the second sampling point three-dimensional observation of the object according to the first implementation of the system for establishing the three-dimensional object contour data of the robot arm according to the present invention. FIG. 4B is a schematic diagram of the second sampling point two-dimensional observation object contour of the first implementation of the present invention based on the three-dimensional object contour data of the robot arm. FIG. 5A and FIG. 5B are method flow charts of the first implementation of the method for establishing the three-dimensional object contour data of the robot arm of the present invention. FIG. 6 is a schematic diagram of the image range and flange end movement of the second implementation of the present invention based on the three-dimensional object contour data of the robot arm. FIG. 7A and FIG. 7B are method flow charts of the second implementation of the method for establishing the three-dimensional object contour data of the robot arm of the present invention.

10: Imaging device

20:Robotic arm

40: Control device

50: Object outline calculation device

Claims (10)

A three-dimensional object contour data establishment system based on a robot arm, comprising: an imaging device having a fixed imaging range; a robot arm having a flange end; an observation object fixed to the flange end; a control device connected to the robot arm to control the movement and rotation of the flange end of the robot arm, so that the observation object fixed to the flange end moves and rotates to different sampling points, and the positions of the observation object moving and rotating to different sampling points are located in the imaging range, and a flange arm motion matrix of the arm space coordinates from the flange end to the robot arm after movement and rotation is calculated; and An object contour calculation device is connected to the control device and the imaging device, receives the imaging parameters of the imaging device from the imaging device, establishes an object flange motion matrix of the observed object and the flange end, receives a spatial coordinate of the imaging device, receives the arm spatial coordinate and the flange arm motion matrix from the control device, calculates an arm image motion matrix according to the arm spatial coordinate and the spatial coordinate, multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate an object image motion matrix, and uses triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on a two-dimensional plane using the object image motion matrix in conjunction with the internal parameters of the imaging device, so as to calculate the contour image of the observed object on the two-dimensional plane. A three-dimensional object contour data establishment system based on a robot arm as described in claim 1, wherein the internal parameters of the imaging device include focal length, aperture and sensitivity. The three-dimensional object contour data establishment system based on the robot arm as described in claim 1, wherein the object flange motion matrix is obtained by Calculated, among which, Obtained using 6-DOF pose marking software, = , is a 3x3 rotation matrix, is a 3x1 displacement matrix, That is the arm image motion matrix. A method for establishing three-dimensional object contour data based on a robot arm, comprising the following steps: An imaging device has a fixed imaging range; A robot arm has a flange end; An observation object is fixed to the flange end; A control device is connected to the robot arm, and the control device controls the movement and rotation of the flange end of the robot arm, so that the observation object fixed to the flange end moves and rotates to different sampling points, and the position of the observation object moving and rotating to different sampling points is located in the imaging range; The control device calculates a flange arm motion matrix of an arm space coordinate from the flange end to the robot arm after movement and rotation; An object contour calculation device is connected to the control device and the imaging device; The object contour calculation device receives the image parameters of the imaging device from the imaging device; The object contour calculation device establishes an object flange motion matrix of the observed object and the flange end; The object contour calculation device receives a spatial coordinate of the imaging device; The object contour calculation device receives the arm spatial coordinate and the flange arm motion matrix from the control device; The object contour calculation device calculates an arm image motion matrix based on the arm spatial coordinate and the spatial coordinate; The object contour calculation device multiplies the object flange motion matrix, the flange arm motion matrix and the arm image motion matrix to calculate an object image motion matrix; and The object contour calculation device uses the triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the object image motion matrix with the internal parameters of the imaging device, so as to calculate the contour image of the observed object on the two-dimensional plane. A method for establishing three-dimensional object contour data based on a robot arm as described in claim 4, wherein the internal parameters of the imaging device include focal length, aperture and sensitivity. The method for establishing three-dimensional object contour data based on a robot arm as described in claim 4, wherein the object flange motion matrix is obtained by Calculated, among which, Obtained using 6-DOF pose marking software, = , is a 3x3 rotation matrix, is a 3x1 displacement matrix, That is the arm image motion matrix. A three-dimensional object contour data establishment system based on a robot arm, comprising: an observation object, which is fixed and placed; a robot arm, which has a flange end; an imaging device, which is fixed to the flange end and has an imaging range; a control device, which establishes a connection with the robot arm to control the movement and rotation of the flange end of the robot arm, so that the imaging device fixed to the flange end moves and rotates to different sampling points, the observation object is located in the imaging range that moves and rotates to different sampling points, and a flange arm motion matrix of an arm space coordinate from the flange end to the robot arm after movement and rotation is calculated; and An object contour calculation device is connected to the control device and the imaging device, receives the imaging parameters of the imaging device from the imaging device, establishes an image flange motion matrix of the imaging device and the flange end, receives an object space coordinate of the observed object, receives the arm space coordinate and the flange arm motion matrix from the control device, and calculates the arm space coordinate and the object space coordinate according to the arm space coordinate. Calculate an arm object motion matrix, multiply the image flange motion matrix, the flange arm motion matrix and the arm object motion matrix to calculate an image object motion matrix, and use triangulation to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane with the image object motion matrix and the internal parameters of the imaging device to calculate the contour image of the observed object on the two-dimensional plane. A three-dimensional object contour data establishment system based on a robot arm as described in claim 7, wherein the object arm motion matrix is obtained by Calculated. A method for establishing three-dimensional object contour data based on a robot arm, comprising the following steps: An observation object is fixedly placed; A robot arm has a flange end; An imaging device is fixed to the flange end and has an imaging range; A control device is connected to the robot arm to control the movement and rotation of the flange end of the robot arm, so that the imaging device fixed to the flange end moves and rotates to different sampling points, and the observation object is located in the imaging range that moves and rotates to different sampling points; The control device calculates a flange arm motion matrix of an arm space coordinate from the flange end to the robot arm after movement and rotation; An object contour calculation device is connected to the control device and the imaging device; The object contour calculation device receives the image parameters of the imaging device from the imaging device; The object contour calculation device establishes an image flange motion matrix of the image device and the flange end; The object contour calculation device receives an object space coordinate of the observed object; The object contour calculation device receives the arm space coordinate and the flange arm motion matrix from the control device; The object contour calculation device calculates an arm object motion matrix based on the arm space coordinate and the object space coordinate; The object contour calculation device multiplies the image flange motion matrix, the flange arm motion matrix and the arm object motion matrix to calculate an image object motion matrix; and The object contour calculation device uses the triangulation method to calculate the projection coordinate point of each three-dimensional coordinate point of the observed object on the two-dimensional plane by combining the image object motion matrix with the internal parameters of the imaging device, so as to calculate the contour image of the observed object on the two-dimensional plane. The method for establishing three-dimensional object contour data based on a robot arm as described in claim 9, wherein the object arm motion matrix is obtained by Calculated.