[go: up one dir, main page]

WO2025005907A1 - Correction de trajectoire de soudage de cobot avec vision intelligente - Google Patents

Correction de trajectoire de soudage de cobot avec vision intelligente Download PDF

Info

Publication number
WO2025005907A1
WO2025005907A1 PCT/US2023/026351 US2023026351W WO2025005907A1 WO 2025005907 A1 WO2025005907 A1 WO 2025005907A1 US 2023026351 W US2023026351 W US 2023026351W WO 2025005907 A1 WO2025005907 A1 WO 2025005907A1
Authority
WO
WIPO (PCT)
Prior art keywords
welded
welding
vision
trajectory
edge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2023/026351
Other languages
English (en)
Inventor
Yongzao CHEN
Jean-Pierre Planckaert
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Air Liquide SA
LAir Liquide SA pour lEtude et lExploitation des Procedes Georges Claude
American Air Liquide Inc
Air Liquide America Corp
Original Assignee
Air Liquide SA
LAir Liquide SA pour lEtude et lExploitation des Procedes Georges Claude
American Air Liquide Inc
Air Liquide America Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Air Liquide SA, LAir Liquide SA pour lEtude et lExploitation des Procedes Georges Claude, American Air Liquide Inc, Air Liquide America Corp filed Critical Air Liquide SA
Priority to PCT/US2023/026351 priority Critical patent/WO2025005907A1/fr
Publication of WO2025005907A1 publication Critical patent/WO2025005907A1/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
    • B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
    • B23K9/00—Arc welding or cutting
    • B23K9/095—Monitoring or automatic control of welding parameters
    • B23K9/0956—Monitoring or automatic control of welding parameters using sensing means, e.g. optical
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
    • B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
    • B23K9/00—Arc welding or cutting
    • B23K9/12—Automatic feeding or moving of electrodes or work for spot or seam welding or cutting
    • B23K9/127—Means for tracking lines during arc welding or cutting
    • B23K9/1272—Geometry oriented, e.g. beam optical trading
    • B23K9/1274—Using non-contact, optical means, e.g. laser means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • B25J9/1684—Tracking a line or surface by means of sensors
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/45—Nc applications
    • G05B2219/45104—Lasrobot, welding robot
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/47—Tracing, tracking
    • G05B2219/4705—Detect edge during machining, welding, sewing

Definitions

  • the features like customizable stop time and stop distance limits in the cobot joints can ensure safety when cobots work with operators.
  • the universal robot is one of the most popular collaborative robots in the market.
  • Commercially available cobots address the skilled labor shortage by allowing companies to "hire" easy-to-use automated welding labor through short or long-term rental or lease programs.
  • the user teaches the welding trajectory for the current cobot welding system before welding.
  • the user needs to move the cobot arm, record each weld's start and end pose, and repeat the trajectory during welding.
  • the welder must conduct repeated welds on workpieces of the same shape and size.
  • the welder must ensure each workpiece is placed in the same position as the reference one.
  • the welder must change the pre-programmed trajectory if any displacement is introduced by loading a new part.
  • the parts repositioning error may cause misalignment with the taught trajectory for repeating welding tasks.
  • the robot Without machine vision, the robot is blind and needs to be programmed and led by operators. The user must adjust the welding trajectory based on the current workpiece position.
  • developing vision capacity is a major task to improve the intelligence of the existing cobot welding system and automatically correct human errors.
  • the vision guide universal robot was developed for pin-picking and has been applied in the industry (see for example US patent US9079308). However, the welding process needs much higher repeat accuracy than the pin-picking task.
  • a compact vision-sensing device for a robotic welding arm having a high- resolution camera, a multi-color light source configured to have multi-color selectivity, and a means of dust and welding fume protection configured to automatically close and protect the high-resolution camera and multi-color light source during a welding operation.
  • Figure 1 is a schematic representation of a typical cobot welding cell as known in the art.
  • Figure 2a is a schematic representation of the components of a robot arm in accordance with one embodiment of the current invention.
  • Figure 2b is a schematic representation of the components of a robot arm in accordance with one embodiment of the current invention.
  • Figure 3a is a schematic representation of the components of the vision-sensing device in accordance with one embodiment of the current invention.
  • Figure 3b is a schematic representation of the components of the vision-sensing device in accordance with one embodiment of the current invention.
  • Figure 4 is a schematic representation of a typical cobot welding cell in accordance with one embodiment of the current invention.
  • Figure 5a is a schematic representation of the calibration procedure in accordance with one embodiment of the current invention.
  • Figure 5b is a schematic representation of the calibration procedure in accordance with one embodiment of the current invention.
  • Figure 6a is a schematic representation of first model generation option, wherein 2022P00156 the software interface will require the user to select the point along the edge of the target workpiece in the image, in accordance with one embodiment of the present invention.
  • Figure 6b is a schematic representation of the welding waypoints, in accordance with one embodiment of the present invention.
  • Figure 7a is a schematic representation of second model generation option, wherein the boundary of the workpieces is automatically generated in the image with an image-processing algorithm, which requires a uniform background, in accordance with one embodiment of the current invention.
  • Figure 7b is a schematic representation of second model generation option, wherein the boundary of the workpieces is automatically generated in the image with an image-processing algorithm, which requires a uniform background, in accordance with one embodiment of the current invention.
  • Figure 8a is a schematic representation of reference trajectory procedure in accordance with one embodiment of the current invention.
  • Figure 8b is a schematic representation of reference trajectory procedure in accordance with one embodiment of the current invention.
  • Figure 9 is a flowchart representation of the basic steps required for the application of an automatic trajectory for repeatable welding tasks, in accordance with one embodiment of the current invention.
  • Figure 10 is a flowchart representation of the basic steps required for the application of an automatic trajectory for welding tasks involving multiple objects, in accordance with one embodiment of the current invention.
  • an intelligent vision-guided cobotic welding system that can automatically locate the object in the workpiece and adjust the welding trajectory to make a weld.
  • the computer vision algorithms can automatically correct any displacement error caused by loading the new part.
  • the intelligent vision system also finds multiple welding workpieces in the workspace and calculates the welding trajectory for each one which highly reduces the programming time for the operators.
  • Available systems can perform fully automated arc welding (GMAW) with various shielding gas, filler metals, and base metals.
  • Control system 101 controls both power source 102 and robot arm 103 to make them work simultaneously in the frame of a welding strategy.
  • Control system 101 controls the trajectory of robot arm 103 and power source 102 controls the welding parameters (amperage, voltage, wire-feeding speed).
  • Power source 102 controls all consumables (gas and wire).
  • Robot arm 103 will typically be attached to a worktable or bench 104, whereupon item 105 to be welded will be positioned.
  • Control system 101 is functionally connected to power source 102 my means of power source interface communication cable 106.
  • Power source 102 is functionally connected to robot arm 103 by means of hose package 107.
  • Control system 101 may be functionally connected to robot arm 103 by means of robot arm interface communication cable 108.
  • the operator provides input to control system 101 by means of teach pendant 109.
  • the cobot can be summarized as a high-end torch handler with all safety features (interlocks) embedded.
  • the operator may use mobile devices such as smartphones to program the moving pass instead of using the original teach pendant.
  • Figures 2a and 2b illustrate the components of robot arm 103 in accordance with one embodiment of the current invention.
  • Base plate (sometimes referred to as the waist) 201 is affixed to the worktable or workbench (not shown), and to shoulder 202.
  • Welding torch 209 is attached to wrist 208.
  • Weld wire holder 210 may be attached 2022P00156 to upper arm 204, or to some other location that is functionally acceptable.
  • Weld wire conduit 211 is locate between weld wire holder 210 and wire feeder 212 and provides the conduit for the wire to travel to the feeder.
  • Weld wire conduit 211 passes through wire feeder 212 and is typically then referred to as torch cable 213.
  • Weld wire 211 and torch cable 213 are the same cable.
  • Torch cable (sometimes referred to as a whip) 213 connects welding torch 209 with weld wire holder 210 and provides wire to the torch.
  • Vision-sensing device 214 may be attached to lower arm 206, to wrist 208, or welding torch 209 facing the working bench (not shown)
  • Figures 3a and 3b illustrate the component of vision-sensing device 214, in accordance with one embodiment of the current invention.
  • Camera (and lens) 301 is located inside camera shell 305.
  • Camera 301 may be a digital camera designed to capture and process a two-dimensional map of reflected intensity or contrast.
  • Camera 301 may be used to evaluate the color, size, shape or location of item 105 to be welded.
  • Camera shell 305 is designed to protect the vision sensors camera and lens 301 from spatters and welding fumes during operation.
  • Automatic lens cap 303 is a front lens cover that automatically opens and closes. In Figure 3a automatic lens cap 303 is open, and in Figure 3b automatic lens cap 303 is closed. This opening and closing is controlled by control system 101. Camera and lens 301 is connected to control system 101 through an ethernet cable (not shown).
  • Light source 304 may be added to the outside of camera shell 305 and may be able to vary colors. A typical machine vision system utilizes ambient, white light. This is not always ideal but is obviously readily available. Multi-wavelength (RGB) lights may be used to facilitate optimal contrast and visibility. Light source 304 may have multi-color selectivity.
  • Control system 101 controls both power source 102 and robot arm 103.
  • Control system 101 is functionally connected to power source 102 my means of power source interface communication cable 106.
  • Power source 102 is functionally connected to robot arm 103 by means of hose package 107.
  • Control system 101 may be functionally connected to robot arm 103 by means of robot arm interface communication cable 108.
  • Control system 101 may be functionally connected to vision-sensing device 214 by means of vision-sensing interface communication cable 401.
  • the main procedures for welding with a vision-guided cobot include four steps: 2022P00156 I. vision system calibration, II. model generation, III. reference trajectory programming, and IV. vision-guided welding.
  • I. Vision System Calibration The vision system calibration and model generation are performed before welding.
  • the vision system must be calibrated or recalibrated under the following conditions: 1. first installation of the camera, 2. the camera position on the cobot is moved, 3. the working distance between the camera and the target workpiece is changed, and the lens focus also needs adjustment.
  • Figures 5a and 5b the user starts the following calibration procedure after the camera installation. First, robot arm 103 is moved into an initial image acquisition position (Position A). Calibration plate 501 is positioned in front of robot arm 103.
  • Calibration plate 501 is a target painted with special patterns which is recognizable by the control system.
  • Camera 214 is focused and the first image of calibration plate 501 is made.
  • Robot arm 103 is then moved (Position B or Position C) to change the visual angle of calibration plate 501, and multiple images are taken of the target from different positions.
  • Intrinsic camera parameters are then calculated based on these images.
  • Intrinsic camera parameters include the focal length, the optical center, and the skew coefficient. These intrinsic parameters are used to map the coordinates of calibration plate 501 into an image plane. Multiple images of calibration plate 501 must be taken from different camera poses and positions. Extrinsic camera parameters are then calculated based on these images.
  • the robot arm 103 After finishing programming, set the robot arm 103 at the image acquisition position and take the reference image of the first part.
  • the algorithm will automatically find the part boundary which the user defined in model generation. As illustrated in Figures 8a and 8b, after finishing welding the first workpiece (801), the user will remove first workpiece 801 and place second workpiece 802 in position. After obtaining the new image, the edge-based algorithm will automatically find the object in the new photo and calculate the displacement of second workpiece 802 relative to the first workpiece 801. The algorithm calculates a new set of waypoints for second workpiece 802 and will be updated automatically. Hence, the vision algorithm can guide robot arm 103 to welding the identical size workpieces in any location on the workbench.
  • the algorithm can also guide robot arm 103 to weld multiple same-size workpieces on the bench.
  • the algorithm can create new trajectories of each part automatically.
  • IV: Vision Guided Welding Turning to the process flowchart in Figure 9, we see the basic steps required for the application of an automatic trajectory for repeatable welding tasks.
  • the system is calibrated as discussed above.
  • a 2D model is created using a reference object, which is typically the first workpiece to be welded.
  • the first workpiece is placed on the table in the working zone.
  • the system takes an image of the first object and identifies the required edge.
  • the waypoints for the first workpiece are programed into the system, and the associated trajectories are calculated.
  • the system then utilizes these trajectories to 2022P00156 weld the first workpiece.
  • the first workpiece is removed, and a second workpiece is placed in the working zone.
  • the system takes an image of the second object and identifies the required edge. If the system detects a significant variation in the size or shape of the second object relative to the calibration plate (or first workpiece) this variation is reported, and if necessary, the process is stopped, and this variation is addressed. If no significant variations are detected, the system then calculates the displacement between the first workpiece and the second workpiece. Typically, displacements of greater than 0.5 mm but less than 20 mm in either the x direction or the y direction are acceptable. A rotational displacement of between 0.1 degree and 15 degrees is also generally acceptable.
  • the system then adjusts for the displacement and calculates new trajectories.
  • the system then utilizes these trajectories to weld the first workpiece.
  • 2022P00156 Examples/Data The user used one of the workpieces as the reference for initial trajectory planning. The user placed the second workpiece 30 cm away from the reference workpiece. Then, the user obtained the photos of the two workpieces before welding, and the algorithm created a new trajectory for the second workpiece. The geometry of the weld bead is close to each other. And both of them passed the inspection. The result shows that the vision system can guide the robot in performing repeatable welding tasks even if the second welding part position is changed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Plasma & Fusion (AREA)
  • Robotics (AREA)
  • Optics & Photonics (AREA)
  • Geometry (AREA)
  • Manipulator (AREA)

Abstract

Un dispositif de détection de vision compact pour un bras de soudage robotique, ayant une caméra à haute résolution, une source de lumière multicolore configurée pour avoir une sélectivité multicolore, et un moyen de protection contre la poussière et les fumées de soudage configuré pour fermer et protéger automatiquement la caméra à haute résolution et la source de lumière multicolore pendant une opération de soudage.
PCT/US2023/026351 2023-06-27 2023-06-27 Correction de trajectoire de soudage de cobot avec vision intelligente Pending WO2025005907A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2023/026351 WO2025005907A1 (fr) 2023-06-27 2023-06-27 Correction de trajectoire de soudage de cobot avec vision intelligente

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2023/026351 WO2025005907A1 (fr) 2023-06-27 2023-06-27 Correction de trajectoire de soudage de cobot avec vision intelligente

Publications (1)

Publication Number Publication Date
WO2025005907A1 true WO2025005907A1 (fr) 2025-01-02

Family

ID=87468625

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/026351 Pending WO2025005907A1 (fr) 2023-06-27 2023-06-27 Correction de trajectoire de soudage de cobot avec vision intelligente

Country Status (1)

Country Link
WO (1) WO2025005907A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9079308B2 (en) 2008-08-29 2015-07-14 Abb Research Ltd. Robotic picking of parts from a bin
US20220324110A1 (en) * 2020-07-17 2022-10-13 Path Robotics Inc. Real time feedback and dynamic adjustment for welding robots

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9079308B2 (en) 2008-08-29 2015-07-14 Abb Research Ltd. Robotic picking of parts from a bin
US20220324110A1 (en) * 2020-07-17 2022-10-13 Path Robotics Inc. Real time feedback and dynamic adjustment for welding robots

Similar Documents

Publication Publication Date Title
Kah et al. Robotic arc welding sensors and programming in industrial applications
US5572102A (en) Method and apparatus for vision control of welding robots
US10363628B2 (en) Automatic and semi-automatic welding systems and methods
Nele et al. An image acquisition system for real-time seam tracking
US4568816A (en) Method and apparatus for manipulator welding apparatus with improved weld path definition
CN113146172B (zh) 一种基于多视觉的检测与装配系统及方法
US20190047068A1 (en) Welding apparatus
US20090179021A1 (en) Welding robot
CN111390915A (zh) 一种基于ai的自动焊缝路径识别方法
SE8106670L (sv) Manipulatorsvetsapparat och sett att manovrera sadan
CN109862989B (zh) 激光焊接时的基于图像的技术选择
CN107414248A (zh) 面向船舶中大型弯管环焊缝的自动化焊接系统及方法
US20250001598A1 (en) Cobot welding trajectory correction with smart vision
JP6550985B2 (ja) ロボット接合システム
WO2023188889A1 (fr) Procédé de commande de soudage pour soudage automatique, dispositif de commande, système de soudage, programme et procédé de soudage
WO2025005907A1 (fr) Correction de trajectoire de soudage de cobot avec vision intelligente
Kumar et al. Development of an autonomous vision sensor-actuator-based circumferential seam path tracker welding machine/device for LPG cylinders
Lin et al. Robotic welding
CN114746208B (zh) 补焊设备以及补焊方法
JP3224739B2 (ja) 管の自動溶接装置
US20250153261A1 (en) Techniques for seam localization and gap measurement
CN119609484A (zh) 一种焊接机器人的焊接方法
US20250262680A1 (en) Remote operation method for remote operation welding system, and welding system
Kumar et al. Development of an Autonomous Vision Sensor-Actuator-Based Circumferential Seam Path Tracker Welding Machine for LPG Cylinders
KR200248899Y1 (ko) 자동 용접장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23745317

Country of ref document: EP

Kind code of ref document: A1