Coupling Human Action Recognizer With Object Detector for Robotic Manipulation in Human–Robot Interaction
ASME Letters in Translational Robotics · 2025
DOI: 10.1115/1.4070090 ↗Kurz gefasst
Fachartikel in der Zeitschrift ASME Letters in Translational Robotics, 2025. Verfasst von Saurabh Sachan und 1 weiteren Autor:innen. Die Arbeit nennt 66 Literaturangaben.
Abstract (Originalsprache)
Establishing a communication mechanism for service and assistive robots to help the disabled and old age population through a human–robot interaction (HRI) is still a challenging task for further study. This research explores a vision and gesture-based control strategy, taking into account static and dynamic human gestures and full-body pose estimation without the use of any wearable devices. The human action recognition algorithm is coupled with an object detector algorithm to automatically perform a robotic task. The deep learning (DL) techniques, MediaPipe with long short-term memory (LSTM) model and you only look once (YOLO) v9 algorithms, are implemented for human gesture recognition and object detection, respectively. For the robotic manipulation task, there are eight object classes, each uniquely associated with one of the eight corresponding human gesture classes. These gestures are categorized into two types: four static gestures and four dynamic gestures. The dataset of object and gesture classes is prepared from scratch in this study. Based on the particular human gesture or action, the Intel RealSense depth camera provides position and depth information of the detected object, and the robotic arm picks and places that object using its inverse kinematics (IK) solution. The prediction outcomes and experimental demonstrations suggest that the proposed study performs quite well as compared to the recently used related studies and exhibits remarkable performance in establishing a real-time HRI system.
Zitieren
APA
Sachan, S., & Pathak, P. M. (2025). Coupling Human Action Recognizer With Object Detector for Robotic Manipulation in Human–Robot Interaction. ASME Letters in Translational Robotics. https://doi.org/10.1115/1.4070090
Vancouver
Sachan S, Pathak PM. Coupling Human Action Recognizer With Object Detector for Robotic Manipulation in Human–Robot Interaction. ASME Letters in Translational Robotics. 2025. doi:10.1115/1.4070090
BibTeX
@article{Sachan2025,
author = {Sachan, Saurabh and Pathak, Pushparaj Mani},
title = {Coupling Human Action Recognizer With Object Detector for Robotic Manipulation in Human–Robot Interaction},
journal = {ASME Letters in Translational Robotics},
year = {2025},
doi = {10.1115/1.4070090}
}
EUVIMED