EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Object-Mesh-Free Dexterous Robot Demonstrations from Human RGB-D Video and Teleoperation

Juyong Hong, Seunghwan Um, Young Gwang Son, Chul Hyun Kwon, Chemin Ahn, Taehyun Bae, Sihoon Leeu, Taehwan Yun, Taiho Ko, Hyouk Ryeol Choi

Machines · 05.11.2026 · 14 (11) · 1303

DOI: 10.3390/machines14111303 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machines, online erschienen am 05.11.2026. Verfasst von Juyong Hong und 9 weiteren Autor:innen. Die Arbeit nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Trade, Industry and Energy (MOTIE), Republic of Korea.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Teleoperating a five-fingered robot hand to collect demonstrations is costly and scales poorly, while human video is abundant yet contains neither robot commands nor robot images. Existing approaches overlay rendered robots on human video, mainly for parallel-jaw grippers; reconstruct object meshes or point clouds in place of the original RGB; or require a wearable exoskeleton during capture. We present a pipeline that composes stationary RGB-D video of a bare human hand with teleoperation logs of the same task. The video supplies the metric hand trajectory and the scene; the teleoperation log supplies recorded finger position targets. Persistent hand-state transition alignment (PHTA) matches ordered changes in a common hand-shape representation and transfers the complete finger-command vector through one temporal correspondence, without task-phase, contact, or grasp labels. Part-aware visibility transfer replaces the human hand with the robot hand while preserving which parts were in front of or behind the object, requiring no object mesh or pose. Combining N videos with M demonstrations yields N×M composed demonstrations that reuse the M recorded finger-command trajectories across the N scenes. On four real-robot tasks, five teleoperation demonstrations composed with twenty human episodes raise behavior-cloning success from 0–45% to 50–90%, reaching in two tasks the same observed success rate as in one hundred teleoperation demonstrations.

Förderung

  • Ministry of Trade, Industry and Energy (MOTIE), Republic of Korea

Zitieren

APA

Hong, J., Um, S., Son, Y. G., Kwon, C. H., Ahn, C., Bae, T., Leeu, S., Yun, T., Ko, T., & Choi, H. R. (2026). Object-Mesh-Free Dexterous Robot Demonstrations from Human RGB-D Video and Teleoperation. Machines, 14(11), 1303. https://doi.org/10.3390/machines14111303

Vancouver

Hong J, Um S, Son YG, Kwon CH, Ahn C, Bae T, et al. Object-Mesh-Free Dexterous Robot Demonstrations from Human RGB-D Video and Teleoperation. Machines. 2026;14(11):1303. doi:10.3390/machines14111303

BibTeX
@article{Hong2026,
  author = {Hong, Juyong and Um, Seunghwan and Son, Young Gwang and Kwon, Chul Hyun and Ahn, Chemin and Bae, Taehyun and Leeu, Sihoon and Yun, Taehwan and Ko, Taiho and Choi, Hyouk Ryeol},
  title = {Object-Mesh-Free Dexterous Robot Demonstrations from Human RGB-D Video and Teleoperation},
  journal = {Machines},
  year = {2026},
  volume = {14},
  number = {11},
  pages = {1303},
  doi = {10.3390/machines14111303}
}
Kostenlos starten →