TWI849599B - Welding path generating system and welding path generating method - Google Patents

Abstract

A welding path generating system and a welding path generating method are provided. The welding path generating system includes an image sensor, a storage device and a processor. The image sensor obtains a sensing image of a welding target. The storage device stores a vision analysis module, an analysis module and a path planning module. The processor is coupled to the storage device and the image sensor. The processor executes the vision analysis module to analyze the sensing image and create scan data. The processor executes the analysis module to identify weld bead profile information according to the scan data. The processor executes the path planning module to generate welding path information according to the weld bead profile information.

Description

Soldering path generation system and soldering path generation method

The present invention relates to a path estimation technology, and in particular to a welding path generation system and a welding path generation method.

Currently, the welding between plates is performed by users manually operating welding equipment, which is inefficient. In particular, when the plates to be welded are thick plates and multiple layers of welding are required to achieve the welding of two plates, the welding operation will become complicated and prone to human error or failure to pay attention to welding procedure specifications and standard welding templates, resulting in welding failures or collisions.

The present invention provides a welding path generation system and a welding path generation method, which can automatically generate welding paths to facilitate automatic welding operations.

The welding path generation system of the present invention includes an image sensor, a storage device and a processor. The image sensor is used to obtain a sensing image of a welding target. The storage device is used to store a visual analysis module, a parsing module and a path planning module. The processor is coupled to the storage device and the image sensor. The processor executes the visual analysis module to analyze the sensing image and establish scan data. The processor executes the parsing module to identify the weld path profile information according to the scan data. The processor executes the path planning module to generate the welding path information according to the weld path profile information.

The welding path generation method of the present invention includes the following steps: obtaining a sensing image of a welding target through an image sensor; analyzing the sensing image through a processor and establishing scanning data; identifying welding track contour information based on the scanning data through the processor; and generating welding path information based on the welding track contour information through the processor.

Based on the above, the welding path generation system and welding path generation method of the present invention can identify the welding path contour by image scanning, and can generate welding path information based on the welding path contour information.

Although the present invention has been disclosed as above by the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the attached patent application.

100: Welding path generation system

110: Processor

120: Storage device

121: Visual analysis module

122:Analysis module

123: Path planning module

130: Image sensor

140:Robotic arm

201: Sensing images

202: Scan data

203: Welding path morphology recognition results

204: Welding parent material segmentation point cloud

205: Welding path information

401, 402: Plates

403: Welding area

404: Solder path

410: Substrate

710~719: Welding materials

S310~S340, S610~S660: Steps

X, Y, Z: axis

FIG1 is a schematic diagram of a welding path generation system of an embodiment of the present invention.

Figure 2 is a schematic diagram of multiple modules of an embodiment of the present invention.

Figure 3 is a flow chart of a method for generating a welding path according to an embodiment of the present invention.

FIG4 is a schematic diagram of a welding scenario of an embodiment of the present invention.

Figure 5 is a schematic diagram of scan data of an embodiment of the present invention.

Figure 6 is a flow chart of a welding path generation method of another embodiment of the present invention.

FIG7 is a side view of the welding result of another embodiment of the present invention.

In order to make the content of the present invention more understandable, the following embodiments are specifically cited as examples of how the present disclosure can be implemented. In addition, wherever possible, elements/components/steps with the same number in the drawings and embodiments represent the same or similar parts.

FIG1 is a schematic diagram of a welding path generation system according to an embodiment of the present invention. Referring to FIG1 , the welding path generation system 100 includes a processor 110, a storage device 120, an image sensor 130, and a robot arm 140. The processor 110 is coupled to the storage device 120, the image sensor 130, and the robot arm 140. The robot arm 140 may be equipped with relevant welding equipment (for example, a heating device for welding). In this embodiment, the processor 110 can obtain the current welding situation through the image sensor 130, and execute the relevant module (i.e., software, program or algorithm, etc.) stored in the storage device 120 to determine the welding track contour in the current welding situation by visual analysis to generate welding path information. The processor 110 can control the robot arm 140 according to the welding path information to realize automatic welding operation.

In this embodiment, the processor 110 may be disposed in, for example, a personal computer (PC), a notebook computer, a tablet computer, an industrial computer, an embedded computer, or a cloud server, or other electronic devices having computing functions, but the present invention is not limited thereto. In this embodiment, the storage device 120 may include a memory, wherein the memory may be, for example, a non-volatile memory such as a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a volatile memory such as a random access memory (RAM), a hard disk drive, a semiconductor memory, and other memories. The storage device 120 is used to store various modules, images, information, parameters, and data mentioned in the present invention, and can be read and executed by the processor 110 to realize the image analysis, analytical calculation, and control functions of the robot arm 140 described in each embodiment of the present invention.

In this embodiment, the image sensor 130 may be a depth camera or a structured light camera, for example, to emit three-dimensional structured light to scan the welding target. In other words, the sensing image obtained by the processor 110 through the image sensor 130 may be an image with depth information. In this embodiment, the robot arm 140 may include, for example, a plurality of joint axes, for example, to realize a robot arm with six degrees of freedom in space, but the present invention is not limited thereto.

FIG. 2 is a schematic diagram of multiple modules of an embodiment of the present invention. FIG. 3 is a flow chart of a welding path generation method of an embodiment of the present invention. FIG. 4 is a schematic diagram of a welding scenario of an embodiment of the present invention. FIG. 5 is a schematic diagram of scan data of an embodiment of the present invention. Referring to FIG. 1 to FIG. 5, the storage device 120 can store a visual analysis module 121, a parsing module 122, and a path planning module 123. In this embodiment, the welding path generation system 100 can execute the visual analysis module 121, the parsing module 122, and the path planning module 123 to perform operations such as the following steps S310~S340 to realize the function of welding path generation.

In step S310, the processor 110 can obtain the sensing image 201 of the welding target through the image sensor 130. As shown in FIG4, the plate 401 and the plate 402 can be thick plates of metal materials. The plate 401 and the plate 402 can be placed or fixed on the substrate 410 first, so that the welding path generation system 100 can perform welding operations on the preset welding area 403 between the plate 401 and the plate 402, and weld the plate 401 and the plate 402 together through the welding material (not shown). The plate 401 and the plate 402 can be placed parallel to the plane of the substrate 410 extending in the Y direction and the X direction. In this embodiment, the image sensor 130 may, for example, photograph the plate 401 and the plate 402 in a direction with a fixed angle with the Z axis to obtain a sensing image 201 with depth information.

In step S320, the processor 110 may analyze the sensing image 201 and create the scanning data 202. In this embodiment, the processor 110 may execute the visual analysis module 121 and input the sensing image 201 to the visual analysis module 121, so that the visual analysis module 121 analyzes the sensing image 201 and creates the scanning data 202. The scanning data 202 may be a three-dimensional point cloud data as shown in FIG5.

In step S330, the processor 110 may identify the bead contour information according to the scan data 202. In this embodiment, the parsing module 122 may include a trained deep point cloud parsing network. The deep point cloud parsing network may be a deep convolutional neural network (CNN). The processor 110 may execute the parsing module 122 to perform point cloud feature sampling on the point cloud data and extract features from deep to shallow. Then, the parsing module 122 may perform bead shape recognition to generate a bead shape recognition result 203. The weld bead morphology recognition result refers to the classification and judgment of the morphology corresponding to the plate 401, the plate 402 and the weld material for multiple point clouds in the point cloud data. The analysis module 122 can also perform weld bead parent material segmentation to separate the point cloud data of the plate 401, the plate 402 and the weld material area, and generate the weld bead parent material segmentation point cloud 204. Therefore, the analysis module 122 can output the weld bead morphology recognition result 203 including the welding target and the weld bead parent material segmentation point cloud 204 to the path planning module 123.

In step S340, the processor 110 may generate welding path information according to the weld profile information. In this embodiment, the processor 110 may execute the path planning module 123 so that the path planning module 123 may fit the weld shape recognition result 203 and the weld parent material segmentation point cloud 204 according to the point cloud model shown in FIG. 5 to generate welding path information 205. The welding path information 205 may refer to the position information of the welding path 404 of the current welding pass in the welding area 403 (for example, including parameters such as coordinates and vectors used to define the path). Therefore, the welding path generation system 100 can automatically generate the welding path 404 of the current welding pass by visual analysis.

Furthermore, in one embodiment, the storage device 120 can also store a robot arm operation module. The processor 110 can execute the robot arm operation module to operate the robot arm 140 to perform welding operations on the welding target through the robot arm operation module according to the welding path information 205. As shown in FIG. 4, the processor 110 can operate the robot arm 140 to perform welding operations on the welding area 403 along the welding path 404.

FIG6 is a flow chart of a welding path generation method of another embodiment of the present invention. Referring to FIG1, FIG2, FIG4 and FIG6, the processor 110 may also pre-train the deep point cloud parsing network by inputting a welding procedure specification (WPS) and multi-layer channel welding data to the deep point cloud parsing network. In this way, the welding path generation system 100 can implement multi-layer channel welding to effectively weld the plate 401 and the plate 402 together through multiple layers of welding materials. In addition, the processor 110 may also pre-fine-tune the model parameters of the deep point cloud parsing network to verify the deep point cloud parsing network to effectively improve the accuracy of the path for performing welding. Even, the processor 110 can also pre-input multiple standard welding templates (Golden samples) to the deep point cloud parsing network to train the deep point cloud parsing network. The standard welding template can be, for example, a standard straight welding template, a standard curved welding template, etc., and the present invention is not limited. In this regard, the welding path generation system 100 can perform the following steps S610~S660 to achieve accurate welding operations.

In step S610, the processor 110 may input point cloud data to the parsing module 122. In step S620, the parsing module 122 may generate welding path information. In step S630, the path planning module 123 may first fit (multi-dimensionally fit) the welding path of the welding path information into robot arm points. The robot arm points may refer to a plurality of continuous position points where a feature point or contour at the end of the arm of the robot arm 140 moves along the welding path 404 during a welding operation. In step S640, the path planning module 123 may then compare the robot arm points with the welding procedure specifications. The welding procedure specification may, for example, define a welding operation range with a restricted area. In step S650, the processor 110 may determine whether the robot arm position is completely within the restricted area (stereoscopic area). If not, the processor 110 may execute the parsing module 122 again to generate new adjusted welding path information. If yes, in step S660, the processor 110 may operate the robot arm 140 to perform welding operation on the welding target (perform welding operation on the welding area 403 along the welding path 404). In other words, the processor 110 can operate the robot arm 140 to perform safe and appropriate welding operations on the welding target according to the welding path information after the aforementioned safety judgment, and can effectively avoid the robot arm 140 from colliding during the automatic welding operation (i.e., the robot arm 140 and the plate, board or welding material have an unexpected collision).

FIG7 is a side view of the welding result of another embodiment of the present invention. FIG7 may, for example, be a schematic diagram corresponding to the Y direction of FIG4. Referring to FIG1, FIG2 and FIG7, in this embodiment, after the robot arm 140 completes the first welding operation on the welding target (i.e., the welding area 403), a first welding material 710 may be formed between the plate 401 and the plate 402. Then, the image sensor 130 may obtain the next sensing image of the welding target (i.e., the welding area 403) again. The processor 110 may execute the welding path generation method as described above again. The processor 110 may execute the visual analysis module 121 to analyze the next sensing image and establish the next scanning data. The processor 110 may execute the parsing module 122 to identify the next weld profile information according to the next scan data. The processor 110 may execute the path planning module 123 to generate the next welding path information according to the next weld profile information. Therefore, the processor 110 may operate the robot arm 140 to perform a second welding operation on the welding target (i.e., the welding area 403). After the robot arm 140 completes the second welding operation on the welding target (i.e., the welding area 403), a second welding material 711 may be formed between the plate 401 and the plate 402. By analogy, the processor 110 repeatedly executes the above-mentioned welding path generation method to form multiple welding materials 711~719 between the plate 401 and the plate 402 through multi-layer welding. Therefore, the plate 401 and the plate 402 can be effectively welded together through the welding materials 711~719.

In summary, the welding path generation system and welding path generation method of the present invention can establish a model through visual analysis and establishment of scan data, and can automatically generate welding paths through analytical models and path planning to achieve automatic welding function. During the path planning process, the soldering path generation system and soldering path generation method of the present invention can also generate soldering paths based on soldering procedure specifications and standard soldering templates to achieve safe and reliable soldering operations. Although the present invention has been disclosed as above by the embodiments, it is not intended to limit the present invention. Any person with ordinary knowledge in the relevant technical field can make some changes and modifications within the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope of the attached patent application.

S310~S340: Steps

Claims (9)

A welding path generation system includes: an image sensor for obtaining a sensing image of a welding target; a storage device for storing a visual analysis module, a parsing module and a path planning module; and a processor coupled to the storage device and the image sensor, wherein the processor executes the visual analysis module to analyze the sensing image and establish a scanning data, wherein the processor executes the parsing module to identify a welding path profile information according to the scanning data, wherein the processor executes the path planning module to generate a welding path profile according to the welding path profile information. The processor generates a welding path information, wherein the processor operates the robot arm according to the welding path information to complete the welding operation on the welding target, wherein the processor obtains the next sensing image of the welding target through the image sensor, wherein the processor executes the visual analysis module to analyze the next sensing image and establish the next scanning data, wherein the processor executes the analysis module to identify the next welding track contour information according to the next scanning data, wherein the processor executes the path planning module to generate the next welding path information according to the next welding track contour information. A welding path generation system as described in claim 1, wherein the image sensor is a depth camera or a structured light camera, and the sensed image is an image with depth information. A welding path generation system as described in claim 2, wherein the parsing module includes a deep point cloud parsing network, and the scanning data is a point cloud data. A welding path generation system as described in claim 3, wherein the weld profile information includes a weld shape recognition result of the welding target and a weld base material segmentation point cloud, and the path planning module fits the weld shape recognition result and the weld base material segmentation point cloud to generate the welding path information. A welding path generation system as described in claim 3, wherein the processor pre-inputs multi-layer welding data to the deep point cloud parsing network to train the deep point cloud parsing network, and the processor pre-fine-tunes the model parameters of the deep point cloud parsing network to verify the deep point cloud parsing network. A welding path generation system as described in claim 5, wherein the processor also pre-inputs a plurality of standard welding templates to the deep point cloud parsing network to train the deep point cloud parsing network. A welding path generation system as described in claim 1, wherein the path planning module simulates a welding path of the welding path information into a robot arm point, and the path planning module determines whether the robot arm point of the welding path information is completely located in a restricted area according to a welding procedure specification, wherein when the robot arm point is not completely located in a restricted area, the processor executes the path planning module to regenerate new welding path information. A welding path generation system as described in claim 1, wherein the storage device also stores a robot arm operation module, wherein the processor executes the robot arm operation module to operate the robot arm through the robot arm operation module to perform the welding operation on the welding target according to the welding path information. A welding path generation method includes: obtaining a sensing image of a welding target through an image sensor; analyzing the sensing image through a processor and establishing a scanning data; identifying a welding track contour information according to the scanning data through the processor; generating a welding path information according to the welding track contour information through the processor; and completing a welding operation on the welding target according to the welding path information through a robot arm. ; obtain the next sensing image of the welding target through the image sensor; execute the visual analysis module through the processor to analyze the next sensing image and establish the next scanning data; execute the analysis module through the processor to identify the next welding track contour information according to the next scanning data; and execute the path planning module through the processor to generate the next welding path information according to the next welding track contour information.