EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking

Yang Luo iD, Xiqing Guo, Mingtao Dong, Jin Yu

Sensors · 22.07.2023 · 23 (14) · 6609

DOI: 10.3390/s23146609 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 22.07.2023. Verfasst von Yang Luo und 3 weiteren Autor:innen. Beteiligte Einrichtung: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China. Die Arbeit wurde bisher 37-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

RGB-T tracking involves the use of images from both visible and thermal modalities. The primary objective is to adaptively leverage the relatively dominant modality in varying conditions to achieve more robust tracking compared to single-modality tracking. An RGB-T tracker based on a mixed-attention mechanism to achieve a complementary fusion of modalities (referred to as MACFT) is proposed in this paper. In the feature extraction stage, we utilize different transformer backbone branches to extract specific and shared information from different modalities. By performing mixed-attention operations in the backbone to enable information interaction and self-enhancement between the template and search images, a robust feature representation is constructed that better understands the high-level semantic features of the target. Then, in the feature fusion stage, a modality shared-specific feature interaction structure was designed based on a mixed-attention mechanism, effectively suppressing low-quality modality noise while enhancing the information from the dominant modality. Evaluation on multiple RGB-T public datasets demonstrates that our proposed tracker outperforms other RGB-T trackers on general evaluation metrics while also being able to adapt to long-term tracking scenarios.

Schlagwörter

Stichwörter der Autor:innen

RGB-T trackingmixed-attention mechanismmulti-modality adaptive fusion

Autor:innen und Institutionen

  • Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China.
  • University of Chinese Academy of Sciences, Beijing 100040, China.
  • Institute of Image Recognition and Machine Intelligence, Northeastern University, Shenyang 110167, China.

ORCID-Profile: Yang Luo

Förderung

  • National Key Research and Development Program of China

Zitieren

APA

Luo, Y., Guo, X., Dong, M., & Yu, J. (2023). Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking. Sensors, 23(14), 6609. https://doi.org/10.3390/s23146609

Vancouver

Luo Y, Guo X, Dong M, Yu J. Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking. Sensors. 2023;23(14):6609. doi:10.3390/s23146609

BibTeX
@article{Luo2023,
  author = {Luo, Yang and Guo, Xiqing and Dong, Mingtao and Yu, Jin},
  title = {Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {14},
  pages = {6609},
  doi = {10.3390/s23146609}
}
Kostenlos starten →