Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects
IEEE Transactions on Industrial Informatics · 2026-08 · 22 (8) · 7078-7089
DOI: 10.1109/tii.2026.3683166 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Industrial Informatics, 2026-08. Verfasst von Yuzhen Xie und 2 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Jiangsu Provincial Key Research and Development Program, Joint Fund Project.
Abstract (Originalsprache)
The efficient integration of visual and tactile information is essential for slip detection and evaluation of grasping stability. However, existing research generally combines visual or tactile modalities as prior information, without fully exploring mechanisms for fusing complementary modalities. In this article, we propose a visual–tactile fusion Transformer (VTF-Trans) for slip detection, designed to handle unaligned data in different formats and facilitate cross-modal information exchange. The main advantages of the proposed method are summarized as follows: first, VTF-Trans employs an improved dual-stream transformer for feature extraction. In addition, we introduce a gated modal attention module to further refine cross-modal fusion. Compared with the existing methods, VTF-Trans effectively integrates useful information from different modalities across multiple scales. Second, to extract deep multimodal information, we propose a cross-modal attention (CMA) mechanism. By defining cross-affinity based on single-modality affinities (token metrics), CMA naturally alleviates the gap between different modalities and domains. Third, we evaluate VTF-trans on three datasets and conduct unknown object grasping experiments. Compared with the state-of-the-art methods, VTF-Trans achieves the highest accuracy for robotic grasping and slip detection, highlighting its superior performance and practical applicability.
Förderung
- National Natural Science Foundation of China
- Jiangsu Provincial Key Research and Development Program
- Joint Fund Project
Zitieren
APA
Xie, Y., Song, A., & Li, H. (2026). Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects. IEEE Transactions on Industrial Informatics, 22(8), 7078-7089. https://doi.org/10.1109/tii.2026.3683166
Vancouver
Xie Y, Song A, Li H. Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects. IEEE Transactions on Industrial Informatics. 2026;22(8):7078-7089. doi:10.1109/tii.2026.3683166
BibTeX
@article{Xie2026,
author = {Xie, Yuzhen and Song, Aiguo and Li, Huijun},
title = {Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects},
journal = {IEEE Transactions on Industrial Informatics},
year = {2026},
volume = {22},
number = {8},
pages = {7078--7089},
doi = {10.1109/tii.2026.3683166}
}
EUVIMED