Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network
IEEE Transactions on Automation Science and Engineering · 2026 · 23 · 12982-12993
DOI: 10.1109/tase.2026.3711084 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Automation Science and Engineering, 2026. Verfasst von Yifan Wang und 3 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben. Gefördert von: Japan Science and Technology Agency (JST) Moonshot Research and Development, Japan China Sasakawa Medical Fellowship.
Abstract (Originalsprache)
Achieving autonomous robotic suturing in robot-assisted surgery (RAS) and biomedical experiments, remains a critical yet challenging goal in which the automation of suture needle grasping is a fundamental step. Manipulating a microsurgical suture needle requires sub-millimeter accuracy and stable, closed-loop control, making accurate and real-time visual feedback essential. This paper presents a real-time neural-network based stereo keypoint tracking method, combined with a state-machine based visual servo control system, for autonomous manipulation of a microsurgical suture needle. The method framework consists of three parts: 1) A Neural Stereo Keypoint Tracker was proposed to detect the keypoints of surgical needles and surgical instruments from stereo image pairs, enabling accurate 3D localization through triangulation. 2) An auto-calibration procedure estimates the transformation between the camera and robot coordinate frames using predefined 3D landmarks. 3) A state-machine based visual servoing controller guides the robot to grasp the needle using 3D keypoints updates at 50 Hz. The proposed method was deployed on an 8-DoF robotic platform with a 2-DoF end-effector and evaluated using two normal and two microsurgical needle types, each tested in 10 static and 10 dynamic trials. The system achieved success rates of $\geq 90$ % for normal needles and $\geq 80$ % for microsurgical needles. Furthermore, two extended experiments: 1) needle extraction and 2) needle handover validated the framework’s adaptability to more complex tasks. These results demonstrate the feasibility and generalization of high-speed and accurate vision servoing systems for autonomous manipulation in microsurgery. Note to Practitioners—This work is motivated by the challenge of automating fine needle manipulation in microsurgery, where sub-millimeter precision and stable visual feedback are required but difficult to maintain manually. The proposed system integrates stereo vision perception, automatic camera–to-robot calibration, and visual servo control to autonomously detect, localize, and grasp surgical needles. In practice, this framework could be integrated into robotic microsurgical platforms to automate needle handling tasks, reducing manual calibration effort and improving consistency in delicate procedures such as vascular suturing. The whole visual servoing framework can be transferred to other systems equipped with stereo cameras.
Förderung
- Japan Science and Technology Agency (JST) Moonshot Research and Development
- Japan China Sasakawa Medical Fellowship
Zitieren
APA
Wang, Y., Lin, X., Perez, S. A. H., & Harada, K. (2026). Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network. IEEE Transactions on Automation Science and Engineering, 23, 12982-12993. https://doi.org/10.1109/tase.2026.3711084
Vancouver
Wang Y, Lin X, Perez SAH, Harada K. Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network. IEEE Transactions on Automation Science and Engineering. 2026;23:12982-12993. doi:10.1109/tase.2026.3711084
BibTeX
@article{Wang2026,
author = {Wang, Yifan and Lin, Xiaofeng and Perez, Saul Alexis Heredia and Harada, Kanako},
title = {Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network},
journal = {IEEE Transactions on Automation Science and Engineering},
year = {2026},
volume = {23},
pages = {12982--12993},
doi = {10.1109/tase.2026.3711084}
}
EUVIMED