EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Human–Robot Interaction Using Dynamic Hand Gesture for Teleoperation of Quadruped Robots with a Robotic Arm

Jianan Xie, Zhen Xu, Jiayu Zeng, Yuyang Gao, Kenji Hashimoto

Electronics · 21.02.2025 · 14 (5) · 860

DOI: 10.3390/electronics14050860 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 21.02.2025. Verfasst von Jianan Xie und 4 weiteren Autor:innen. Die Arbeit wurde bisher 30-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Information, Production and Systems Research Center, Waseda University, Future Robotics Organization, Waseda University, Humanoid Robotics Institute, Waseda University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human–Robot Interaction (HRI) using hand gesture recognition offers an effective and non-contact approach to enhancing operational intuitiveness and user convenience. However, most existing studies primarily focus on either static sign language recognition or the tracking of hand position and orientation in space. These approaches often prove inadequate for controlling complex robotic systems. This paper proposes an advanced HRI system leveraging dynamic hand gestures for controlling quadruped robots equipped with a robotic arm. The proposed system integrates both semantic and pose information from dynamic gestures to enable comprehensive control over the robot’s diverse functionalities. First, a Depth–MediaPipe framework is introduced to facilitate the precise three-dimensional (3D) coordinate extraction of 21 hand bone keypoints. Subsequently, a Semantic-Pose to Motion (SPM) model is developed to analyze and interpret both the pose and semantic aspects of hand gestures. This model translates the extracted 3D coordinate data into corresponding mechanical actions in real-time, encompassing quadruped robot locomotion, robotic arm end-effector tracking, and semantic-based command switching. Extensive real-world experiments demonstrate the proposed system’s effectiveness in achieving real-time interaction and precise control, underscoring its potential for enhancing the usability of complex robotic platforms.

Förderung

  • Information, Production and Systems Research Center, Waseda University
  • Future Robotics Organization, Waseda University
  • Humanoid Robotics Institute, Waseda University
  • JSPS KAKENHI
  • Waseda University Grant for Special Research Projects
  • JST BOOST

Zitieren

APA

Xie, J., Xu, Z., Zeng, J., Gao, Y., & Hashimoto, K. (2025). Human–Robot Interaction Using Dynamic Hand Gesture for Teleoperation of Quadruped Robots with a Robotic Arm. Electronics, 14(5), 860. https://doi.org/10.3390/electronics14050860

Vancouver

Xie J, Xu Z, Zeng J, Gao Y, Hashimoto K. Human–Robot Interaction Using Dynamic Hand Gesture for Teleoperation of Quadruped Robots with a Robotic Arm. Electronics. 2025;14(5):860. doi:10.3390/electronics14050860

BibTeX
@article{Xie2025,
  author = {Xie, Jianan and Xu, Zhen and Zeng, Jiayu and Gao, Yuyang and Hashimoto, Kenji},
  title = {Human–Robot Interaction Using Dynamic Hand Gesture for Teleoperation of Quadruped Robots with a Robotic Arm},
  journal = {Electronics},
  year = {2025},
  volume = {14},
  number = {5},
  pages = {860},
  doi = {10.3390/electronics14050860}
}
Kostenlos starten →