EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation

Shih-Hao Chang, Minyechil Alehegn Tefera iD, Jun-Ming Cheng, Zhen-Cheng Li, Ching-Yuan Chang, Tzu-Hsiu Chang, Tien-Fu Lu iD, Pai-Yen Chen, Chin-Sheng Chen iD

Sensors · 08.11.2026 · 26 (21) · 7064

DOI: 10.3390/s26217064 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 08.11.2026. Verfasst von Shih-Hao Chang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 10608, Taiwan. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science and Technology Council, National Science and Technology Council, Taiwan.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this article, we propose a vision-guided robotic arm system that integrates deep learning, digital twin, and cybersecurity to enhance the maintenance of quadruped robots. The proposed system comprises two core components. (1) A YOLOv11-based object detection system with depth cameras identifies maintenance targets and extracts 3D coordinates for autonomous robotic arm localization and manipulation. (2) A digital twin environment is used to simulate and validate path planning, trajectory stability, and operational safety before real-time deployment. Moreover, we propose a secure communication system to enhance system reliability and resilience against cyber–physical threats. To verify system performance, we first used a mug as a test object to validate the robotic arm’s posture control, motion accuracy, and trajectory consistency. The platform was then extended to quadruped robot maintenance. Experimental results show that the proposed YOLO11 model achieved an mAP@0.5 of 0.995 with reduced computation time. In addition, real experiments demonstrated that the system accurately located the target, with average planar and depth deviations of 0.0277 m and 0.0301 m, respectively. Following the physical proxy validation, the workflow was extended to a simulated quadruped-robot maintenance scenario, in which screw access reachability, trajectory feasibility, and collision-free manipulator motion were evaluated. DIATwin was used for kinematic modeling, trajectory generation, and physical robot script generation, whereas NVIDIA Omniverse was employed as a high fidelity visualization and physics-verification environment using the URDF model exported from DIATwin. The trajectory-transfer experiments demonstrated that digital-twin-generated paths could be successfully transferred to and executed by the physical robotic arm across the evaluated test configurations. Therefore, the proposed system demonstrates the feasibility of integrating vision-guided manipulation, secure communication, and digital-twin-based pre-deployment verification for future smart, automated, and cyber–physical robotics systems.

Schlagwörter

Stichwörter der Autor:innen

YOLO-based detectionartificial intelligence (AI)cyber–physical systemsdigital twinobject detectionquadruped robotrobotic armvision-guided robotic arm

Autor:innen und Institutionen

  • Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 10608, Taiwan.
  • Department of Mechanical Engineering, National Taipei University of Technology, Taipei 10608, Taiwan.
  • YCL Technology Inc., Taipei 104090, Taiwan.
  • School of Electrical and Mechanical Engineering, Adelaide University, Adelaide, SA 5005, Australia.
  • Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL 60607, USA.
  • Graduate Institute of Automation Technology, National Taipei University of Technology, Taipei 10608, Taiwan.

ORCID-Profile: Minyechil Alehegn Tefera, Tien-Fu Lu, Chin-Sheng Chen

Förderung

  • National Science and Technology Council
  • National Science and Technology Council, Taiwan

Zitieren

APA

Chang, S. H., Tefera, M. A., Cheng, J. M., Li, Z. C., Chang, C. Y., Chang, T. H., Lu, T. F., Chen, P. Y., & Chen, C. S. (2026). Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation. Sensors, 26(21), 7064. https://doi.org/10.3390/s26217064

Vancouver

Chang SH, Tefera MA, Cheng JM, Li ZC, Chang CY, Chang TH, et al. Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation. Sensors. 2026;26(21):7064. doi:10.3390/s26217064

BibTeX
@article{Chang2026,
  author = {Chang, Shih-Hao and Tefera, Minyechil Alehegn and Cheng, Jun-Ming and Li, Zhen-Cheng and Chang, Ching-Yuan and Chang, Tzu-Hsiu and Lu, Tien-Fu and Chen, Pai-Yen and Chen, Chin-Sheng},
  title = {Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {21},
  pages = {7064},
  doi = {10.3390/s26217064}
}
Kostenlos starten →