EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Research on Deep Learning-Based Human–Robot Static/Dynamic Gesture-Driven Control Framework

Gong Zhang iD, Jiahong Su, Shuzhong Zhang, Jianzheng Qi, Zhicheng Hou, Qunxu Lin

Sensors · 25.11.2025 · 25 (23) · 7203

DOI: 10.3390/s25237203 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 25.11.2025. Verfasst von Gong Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Automation, Guangdong Polytechnic Normal University, Guangzhou 510665, China. Hauptthemen laut MEDLINE: Robotics, Gestures, Deep Learning. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Open Fund of Fujian Provincial Key Laboratory of Special Intelligent Equipment Safety and Measurement & Control, Open Fund of Fujian Provincial Key Laboratory of Intelligent Machining Technology and Equip-ment.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

For human–robot gesture-driven control, this paper proposes a deep learning-based approach that employs both static and dynamic gestures to drive and control robots for object-grasping and delivery tasks. The method utilizes two-dimensional Convolutional Neural Networks (2D-CNNs) for static gesture recognition and a hybrid architecture combining three-dimensional Convolutional Neural Networks (3D-CNNs) and Long Short-Term Memory networks (3D-CNN+LSTM) for dynamic gesture recognition. Results on a custom gesture dataset demonstrate validation accuracies of 95.38% for static gestures and 93.18% for dynamic gestures, respectively. Then, in order to control and drive the robot to perform corresponding tasks, hand pose estimation was performed. The MediaPipe machine learning framework was first employed to extract hand feature points. These 2D feature points were then converted into 3D coordinates using a depth camera-based pose estimation method, followed by coordinate system transformation to obtain hand poses relative to the robot’s base coordinate system. Finally, an experimental platform for human–robot gesture-driven interaction was established, deploying both gesture recognition models. Four participants were invited to perform 100 trials each of gesture-driven object-grasping and delivery tasks under three lighting conditions: natural light, low light, and strong light. Experimental results show that the average success rates for completing tasks via static and dynamic gestures are no less than 96.88% and 94.63%, respectively, with task completion times consistently within 20 s. These findings demonstrate that the proposed approach enables robust vision-based robotic control through natural hand gestures, showing great prospects for human–robot collaboration applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RoboticsGesturesDeep LearningHumansNeural Networks, ComputerHand

Stichwörter der Autor:innen

deep learningdynamic and static gesturegesture-driven control frameworkhuman-robot collaborationthree-dimensional Convolutional Neural Networks

Autor:innen und Institutionen

  • School of Automation, Guangdong Polytechnic Normal University, Guangzhou 510665, China.
  • Institute for Super Robotics (Huangpu), South China University of Technology, Guangzhou 510700, China.
  • School of Mechanical and Automotive Engineering, Fujian University of Technology, Fuzhou 350108, China.
  • School of Rail Transportation, Wuyi University, Jiangmen 529020, China.

ORCID-Profile: Gong Zhang

Förderung

  • National Natural Science Foundation of China
  • Open Fund of Fujian Provincial Key Laboratory of Special Intelligent Equipment Safety and Measurement & Control
  • Open Fund of Fujian Provincial Key Laboratory of Intelligent Machining Technology and Equip-ment
  • Natural Science Foundation of Fujian Province, China
  • Open Fund of Fujian Provincial Key Laboratory of Special Intelligent Equipment Safety and Measurement & Control

Zitieren

APA

Zhang, G., Su, J., Zhang, S., Qi, J., Hou, Z., & Lin, Q. (2025). Research on Deep Learning-Based Human–Robot Static/Dynamic Gesture-Driven Control Framework. Sensors, 25(23), 7203. https://doi.org/10.3390/s25237203

Vancouver

Zhang G, Su J, Zhang S, Qi J, Hou Z, Lin Q. Research on Deep Learning-Based Human–Robot Static/Dynamic Gesture-Driven Control Framework. Sensors. 2025;25(23):7203. doi:10.3390/s25237203

BibTeX
@article{Zhang2025,
  author = {Zhang, Gong and Su, Jiahong and Zhang, Shuzhong and Qi, Jianzheng and Hou, Zhicheng and Lin, Qunxu},
  title = {Research on Deep Learning-Based Human–Robot Static/Dynamic Gesture-Driven Control Framework},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {23},
  pages = {7203},
  doi = {10.3390/s25237203}
}
Kostenlos starten →