EUVIMEDEUVIMED entdecken
Crossref

Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects

Yuzhen Xie, Aiguo Song, Huijun Li

IEEE Transactions on Industrial Informatics · 2026-08 · 22 (8) · 7078-7089

DOI: 10.1109/tii.2026.3683166 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Industrial Informatics, 2026-08. Verfasst von Yuzhen Xie und 2 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Jiangsu Provincial Key Research and Development Program, Joint Fund Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The efficient integration of visual and tactile information is essential for slip detection and evaluation of grasping stability. However, existing research generally combines visual or tactile modalities as prior information, without fully exploring mechanisms for fusing complementary modalities. In this article, we propose a visual–tactile fusion Transformer (VTF-Trans) for slip detection, designed to handle unaligned data in different formats and facilitate cross-modal information exchange. The main advantages of the proposed method are summarized as follows: first, VTF-Trans employs an improved dual-stream transformer for feature extraction. In addition, we introduce a gated modal attention module to further refine cross-modal fusion. Compared with the existing methods, VTF-Trans effectively integrates useful information from different modalities across multiple scales. Second, to extract deep multimodal information, we propose a cross-modal attention (CMA) mechanism. By defining cross-affinity based on single-modality affinities (token metrics), CMA naturally alleviates the gap between different modalities and domains. Third, we evaluate VTF-trans on three datasets and conduct unknown object grasping experiments. Compared with the state-of-the-art methods, VTF-Trans achieves the highest accuracy for robotic grasping and slip detection, highlighting its superior performance and practical applicability.

Förderung

  • National Natural Science Foundation of China
  • Jiangsu Provincial Key Research and Development Program
  • Joint Fund Project

Zitieren

APA

Xie, Y., Song, A., & Li, H. (2026). Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects. IEEE Transactions on Industrial Informatics, 22(8), 7078-7089. https://doi.org/10.1109/tii.2026.3683166

Vancouver

Xie Y, Song A, Li H. Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects. IEEE Transactions on Industrial Informatics. 2026;22(8):7078-7089. doi:10.1109/tii.2026.3683166

BibTeX
@article{Xie2026,
  author = {Xie, Yuzhen and Song, Aiguo and Li, Huijun},
  title = {Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects},
  journal = {IEEE Transactions on Industrial Informatics},
  year = {2026},
  volume = {22},
  number = {8},
  pages = {7078--7089},
  doi = {10.1109/tii.2026.3683166}
}
Kostenlos starten →