EUVIMEDEUVIMED entdecken
Crossref

Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network

Yifan Wang, Xiaofeng Lin, Saul Alexis Heredia Perez, Kanako Harada

IEEE Transactions on Automation Science and Engineering · 2026 · 23 · 12982-12993

DOI: 10.1109/tase.2026.3711084 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Automation Science and Engineering, 2026. Verfasst von Yifan Wang und 3 weiteren Autor:innen. Die Arbeit nennt 29 Literaturangaben. Gefördert von: Japan Science and Technology Agency (JST) Moonshot Research and Development, Japan China Sasakawa Medical Fellowship.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Achieving autonomous robotic suturing in robot-assisted surgery (RAS) and biomedical experiments, remains a critical yet challenging goal in which the automation of suture needle grasping is a fundamental step. Manipulating a microsurgical suture needle requires sub-millimeter accuracy and stable, closed-loop control, making accurate and real-time visual feedback essential. This paper presents a real-time neural-network based stereo keypoint tracking method, combined with a state-machine based visual servo control system, for autonomous manipulation of a microsurgical suture needle. The method framework consists of three parts: 1) A Neural Stereo Keypoint Tracker was proposed to detect the keypoints of surgical needles and surgical instruments from stereo image pairs, enabling accurate 3D localization through triangulation. 2) An auto-calibration procedure estimates the transformation between the camera and robot coordinate frames using predefined 3D landmarks. 3) A state-machine based visual servoing controller guides the robot to grasp the needle using 3D keypoints updates at 50 Hz. The proposed method was deployed on an 8-DoF robotic platform with a 2-DoF end-effector and evaluated using two normal and two microsurgical needle types, each tested in 10 static and 10 dynamic trials. The system achieved success rates of $\geq 90$ % for normal needles and $\geq 80$ % for microsurgical needles. Furthermore, two extended experiments: 1) needle extraction and 2) needle handover validated the framework’s adaptability to more complex tasks. These results demonstrate the feasibility and generalization of high-speed and accurate vision servoing systems for autonomous manipulation in microsurgery. Note to Practitioners—This work is motivated by the challenge of automating fine needle manipulation in microsurgery, where sub-millimeter precision and stable visual feedback are required but difficult to maintain manually. The proposed system integrates stereo vision perception, automatic camera–to-robot calibration, and visual servo control to autonomously detect, localize, and grasp surgical needles. In practice, this framework could be integrated into robotic microsurgical platforms to automate needle handling tasks, reducing manual calibration effort and improving consistency in delicate procedures such as vascular suturing. The whole visual servoing framework can be transferred to other systems equipped with stereo cameras.

Förderung

  • Japan Science and Technology Agency (JST) Moonshot Research and Development
  • Japan China Sasakawa Medical Fellowship

Zitieren

APA

Wang, Y., Lin, X., Perez, S. A. H., & Harada, K. (2026). Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network. IEEE Transactions on Automation Science and Engineering, 23, 12982-12993. https://doi.org/10.1109/tase.2026.3711084

Vancouver

Wang Y, Lin X, Perez SAH, Harada K. Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network. IEEE Transactions on Automation Science and Engineering. 2026;23:12982-12993. doi:10.1109/tase.2026.3711084

BibTeX
@article{Wang2026,
  author = {Wang, Yifan and Lin, Xiaofeng and Perez, Saul Alexis Heredia and Harada, Kanako},
  title = {Autonomous Microsurgical Needle Manipulation With Real-Time Stereo Keypoint Tracking Using Neural Network},
  journal = {IEEE Transactions on Automation Science and Engineering},
  year = {2026},
  volume = {23},
  pages = {12982--12993},
  doi = {10.1109/tase.2026.3711084}
}
Kostenlos starten →