[go: up one dir, main page]

Vision-Force Admittance Learning
for Peg Insertion into a Movable Hole

IEEE Robotics and Automation Letters (RA-L), 2026

Yuzhong Chen1, Yongqing Liang1, Yunzhi Xu2, Irving Fang1, Chase Kidder2, Hui-ping Wang2, Raihan Haque2, Yubiao Zhang2, Chen Feng1✉
1New York University 2General Motors
✉ Corresponding Author
Paper arXiv
Experiment Setup

(a) Experiment Setup

Insertion Process

(b) Insertion Process

Overview of our real-world experiments. (a) shows the mobile base insertion setup. In (b), when the base moves with unknown motion, VFAL drives the robot to an approximate pre-insertion pose, then performs precise insertion using asynchronous force and visual feedback. After insertion, the robot lifts the end-effector.

Abstract

Precise manipulation in dynamic environments, whether induced by a mobile robot base or a target with unknown motion, remains a major challenge in robotics. Manipulation in dynamic environments introduces substantial uncertainty, which fundamentally conflicts with the tight precision requirement of precise tasks such as peg-in-the-hole. We propose a Vision-Force Admittance Learning (VFAL) framework that fuses asynchronous visual feedback with a high-frequency force-based model, using visual pose estimations as a regularization term. VFAL adapts insertion strategies online to dynamic motion while maintaining millimeter-level precision. To obtain robust, low-frequency pose information, we employ state-of-the-art vision foundation models for visual pose estimation. Additionally, we incorporate failure recovery mechanisms to enhance overall robustness. We validate our approach in real-world experiments, demonstrating high success rates and strong adaptability to various pegs and dynamic environments.

Methodology

Algorithm Workflow

Algorithm Workflow. A two-stage pipeline is designed for the manipulation process. In the first stage, before making contact with the base, the robot approaches the target using the pose estimation result. In the second stage, once contact is detected via the force sensor, the model predicts the optimal next step control based on the current state and force readings, while referencing the asynchronously updated pose estimation as a regularization term.

Experiment Results

We evaluate methods in dynamic environments where pegs are inserted into moving bases with holes. All experiments use a FANUC CRX-10iA/L with an FS-15iA force/torque sensor. Control and force sensing run at 125 Hz, while vision runs at 20 Hz. A trial is successful if insertion completes with at most one fin outside the hole when the z-axis force reaches Fmax. Only successful trials are included in the wrench error, with 10 trials per method.

Description Method Modality Success Rate Full Wrench
1D Mobile Base
(Cylinder Christmas tree nut)		 Vision Vision 40% 57.11 (±59.65)
Admittance Force 60% 70.96 (±29.78)
Online Learning Force 70% 80.95 (±42.56)
Synchronous VFAL Force, Vision 90% 28.50 (±14.11)
VFAL Force, Vision 100% 13.95 (±5.05)
3D Mobile Base
(Cross-fin Christmas tree nut)		 Vision Vision 20% 114.60 (±57.30)
Admittance Force 10% 106.09 (N/A)
Online Learning Force 0% N/A (N/A)
Synchronous VFAL Force, Vision 60% 37.82 (±15.83)
VFAL Force, Vision 80% 9.22 (±5.23)

Success rate and Mean Squared Error (MSE) on full wrench across different models for both 1D and 3D mobile base insertions. We mark the best performed results for each experiment in bold.

Video Demonstration

We deploy our method on a FANUC CRX-10iA/L robot arm to qualitatively demonstrate insertion behaviors in both static and dynamic scenarios, including different pegs, different versions of target motion, and failure recovery.

Chapters (click a timestamp to jump):

  • 0:04 — Setup (3D mobile base): environment setup and base trajectory.
  • 0:31 — Experiment 1 – VFAL evaluation: peg insertion on a 3D mobile base under the same setup as the 3D mobile base experiments, with a Sankey diagram of all trials at the beginning.
  • 1:29 — Experiment 1 – baseline failures: failure cases of Vision, Admittance, Online Learning (LML), and Synchronous VFAL.
  • 2:09 — Experiment 2 – other pegs: insertion of a rigid cylinder peg and a classical pushnut into a 3D mobile base.
  • 2:42 — Setup (fixed / 1D mobile base): cylinder Christmas tree nut used in the following experiments.
  • 2:51 — Experiment 3 – 1D mobile base, low speed: 3 trials at the paper's 1D mobile base speed.
  • 3:18 — Experiment 4 – 1D mobile base, high speed: 3 trials at doubled base speed.
  • 3:45 — Experiment 5 – panel collision recovery: collision detected, retreat, and restart of the insertion.
  • 4:21 — Experiment 6 – losing-track recovery: occlusion created by hand, followed by restart of the installation.
  • 4:51 — Experiment 7 – fixed base: 2 trials at zero base speed, presenting the ideal force behavior for insertion.
  • 5:18 — Experiment 8 – hand-held base: insertion into a base with unknown, noisy human-driven random motion.

BibTeX

@article{chen2026vfal,
 title={Vision-Force Admittance Learning for Peg Insertion into a Movable Hole},
 author={Chen, Yuzhong and Liang, Yongqing and Xu, Yunzhi and Fang, Irving and Kidder, Chase and Wang, Hui-ping and Haque, Raihan and Zhang, Yubiao and Feng, Chen},
 journal={IEEE Robotics and Automation Letters},
 year={2026},
 publisher={IEEE}
}

Acknowledgements

The authors gratefully acknowledge the collaboration, support, resources, and technical expertise provided by the General Motors Autonomous Robotics Center (ARC) and the Robotics Intelligence team, which were instrumental to this work.