Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking
Sensors · 22.07.2023 · 23 (14) · 6609
DOI: 10.3390/s23146609 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 22.07.2023. Verfasst von Yang Luo und 3 weiteren Autor:innen. Beteiligte Einrichtung: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China. Die Arbeit wurde bisher 37-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China.
Abstract (Originalsprache)
RGB-T tracking involves the use of images from both visible and thermal modalities. The primary objective is to adaptively leverage the relatively dominant modality in varying conditions to achieve more robust tracking compared to single-modality tracking. An RGB-T tracker based on a mixed-attention mechanism to achieve a complementary fusion of modalities (referred to as MACFT) is proposed in this paper. In the feature extraction stage, we utilize different transformer backbone branches to extract specific and shared information from different modalities. By performing mixed-attention operations in the backbone to enable information interaction and self-enhancement between the template and search images, a robust feature representation is constructed that better understands the high-level semantic features of the target. Then, in the feature fusion stage, a modality shared-specific feature interaction structure was designed based on a mixed-attention mechanism, effectively suppressing low-quality modality noise while enhancing the information from the dominant modality. Evaluation on multiple RGB-T public datasets demonstrates that our proposed tracker outperforms other RGB-T trackers on general evaluation metrics while also being able to adapt to long-term tracking scenarios.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China.
- University of Chinese Academy of Sciences, Beijing 100040, China.
- Institute of Image Recognition and Machine Intelligence, Northeastern University, Shenyang 110167, China.
ORCID-Profile: Yang Luo
Förderung
- National Key Research and Development Program of China
Zitieren
APA
Luo, Y., Guo, X., Dong, M., & Yu, J. (2023). Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking. Sensors, 23(14), 6609. https://doi.org/10.3390/s23146609
Vancouver
Luo Y, Guo X, Dong M, Yu J. Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking. Sensors. 2023;23(14):6609. doi:10.3390/s23146609
BibTeX
@article{Luo2023,
author = {Luo, Yang and Guo, Xiqing and Dong, Mingtao and Yu, Jin},
title = {Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking},
journal = {Sensors},
year = {2023},
volume = {23},
number = {14},
pages = {6609},
doi = {10.3390/s23146609}
}
EUVIMED