Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity
Science Robotics · 28.01.2026 · 11 (110)
DOI: 10.1126/scirobotics.ady2869 ↗Kurz gefasst
Fachartikel in der Zeitschrift Science Robotics, erschienen am 28.01.2026. Verfasst von Qi Ye und 9 weiteren Autor:innen. Beteiligte Einrichtung: College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China. Hauptthemen laut MEDLINE: Robotics. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 99 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Achieving humanlike dexterity with anthropomorphic multifingered robotic hands requires precise finger coordination. However, dexterous manipulation remains highly challenging because of high-dimensional action-observation spaces, complex hand-object contact dynamics, and frequent occlusions. To address this, we drew inspiration from the human learning paradigm of observation and practice and propose a two-stage learning framework by learning visual-tactile integration representations via self-supervised learning from human demonstrations. We trained a unified multitask policy through reinforcement learning and online imitation learning. This decoupled learning enabled the robot to acquire generalizable manipulation skills using only monocular images and simple binary tactile signals. With the unified policy, we built a multifingered hand manipulation system that performs multiple complicated tasks with low-cost sensing. It achieved an 85% success rate across five complex tasks and 25 objects and further generalized to three unseen tasks that share similar hand-object coordination patterns with the training tasks.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China.
ORCID-Profile: Qi Ye, Qingtao Liu, Ke Jin, Huajin Chen, Gaofeng Li, Jiming Chen
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Ye, Q., Liu, Q., Wang, S., Chen, J., Cui, Y., Jin, K., Chen, H., Cai, X., Li, G., & Chen, J. (2026). Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity. Science Robotics, 11(110). https://doi.org/10.1126/scirobotics.ady2869
Vancouver
Ye Q, Liu Q, Wang S, Chen J, Cui Y, Jin K, et al. Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity. Science Robotics. 2026;11(110). doi:10.1126/scirobotics.ady2869
BibTeX
@article{Ye2026,
author = {Ye, Qi and Liu, Qingtao and Wang, Siyun and Chen, Jiaying and Cui, Yu and Jin, Ke and Chen, Huajin and Cai, Xuan and Li, Gaofeng and Chen, Jiming},
title = {Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity},
journal = {Science Robotics},
year = {2026},
volume = {11},
number = {110},
doi = {10.1126/scirobotics.ady2869}
}
EUVIMED