Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation
Sensors · 08.11.2026 · 26 (21) · 7064
DOI: 10.3390/s26217064 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 08.11.2026. Verfasst von Shih-Hao Chang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 10608, Taiwan. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science and Technology Council, National Science and Technology Council, Taiwan.
Abstract (Originalsprache)
In this article, we propose a vision-guided robotic arm system that integrates deep learning, digital twin, and cybersecurity to enhance the maintenance of quadruped robots. The proposed system comprises two core components. (1) A YOLOv11-based object detection system with depth cameras identifies maintenance targets and extracts 3D coordinates for autonomous robotic arm localization and manipulation. (2) A digital twin environment is used to simulate and validate path planning, trajectory stability, and operational safety before real-time deployment. Moreover, we propose a secure communication system to enhance system reliability and resilience against cyber–physical threats. To verify system performance, we first used a mug as a test object to validate the robotic arm’s posture control, motion accuracy, and trajectory consistency. The platform was then extended to quadruped robot maintenance. Experimental results show that the proposed YOLO11 model achieved an mAP@0.5 of 0.995 with reduced computation time. In addition, real experiments demonstrated that the system accurately located the target, with average planar and depth deviations of 0.0277 m and 0.0301 m, respectively. Following the physical proxy validation, the workflow was extended to a simulated quadruped-robot maintenance scenario, in which screw access reachability, trajectory feasibility, and collision-free manipulator motion were evaluated. DIATwin was used for kinematic modeling, trajectory generation, and physical robot script generation, whereas NVIDIA Omniverse was employed as a high fidelity visualization and physics-verification environment using the URDF model exported from DIATwin. The trajectory-transfer experiments demonstrated that digital-twin-generated paths could be successfully transferred to and executed by the physical robotic arm across the evaluated test configurations. Therefore, the proposed system demonstrates the feasibility of integrating vision-guided manipulation, secure communication, and digital-twin-based pre-deployment verification for future smart, automated, and cyber–physical robotics systems.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 10608, Taiwan.
- Department of Mechanical Engineering, National Taipei University of Technology, Taipei 10608, Taiwan.
- YCL Technology Inc., Taipei 104090, Taiwan.
- School of Electrical and Mechanical Engineering, Adelaide University, Adelaide, SA 5005, Australia.
- Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL 60607, USA.
- Graduate Institute of Automation Technology, National Taipei University of Technology, Taipei 10608, Taiwan.
ORCID-Profile: Minyechil Alehegn Tefera, Tien-Fu Lu, Chin-Sheng Chen
Förderung
- National Science and Technology Council
- National Science and Technology Council, Taiwan
Zitieren
APA
Chang, S. H., Tefera, M. A., Cheng, J. M., Li, Z. C., Chang, C. Y., Chang, T. H., Lu, T. F., Chen, P. Y., & Chen, C. S. (2026). Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation. Sensors, 26(21), 7064. https://doi.org/10.3390/s26217064
Vancouver
Chang SH, Tefera MA, Cheng JM, Li ZC, Chang CY, Chang TH, et al. Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation. Sensors. 2026;26(21):7064. doi:10.3390/s26217064
BibTeX
@article{Chang2026,
author = {Chang, Shih-Hao and Tefera, Minyechil Alehegn and Cheng, Jun-Ming and Li, Zhen-Cheng and Chang, Ching-Yuan and Chang, Tzu-Hsiu and Lu, Tien-Fu and Chen, Pai-Yen and Chen, Chin-Sheng},
title = {Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation},
journal = {Sensors},
year = {2026},
volume = {26},
number = {21},
pages = {7064},
doi = {10.3390/s26217064}
}
EUVIMED