Progressive Transformer With Multimodality Adaptation for RGB–T Tracking
IEEE Transactions on Instrumentation and Measurement · 2025 · 74 · 1-17
DOI: 10.1109/tim.2025.3575970 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Instrumentation and Measurement, 2025. Verfasst von Binxin Luo und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 60 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Postdoctoral Fellowship Program of China Postdoctoral Science Foundation.
Abstract (Originalsprache)
With the continuous advancement of sensor technologies, the combined use of RGB images and thermal-infrared (TIR) images has become a critical approach for improving tracking performance by generating powerful fusion features. However, the inherent spatial-spectral discrepancies between RGB and thermal modalities create feature distribution mismatches that hinder effective cross-modality alignment, while their distinct focus regions introduce cross-modality semantic conflicts during fusion. To tackle this issue, we propose a progressive Transformer network with multimodality adaptation (PTrMA) to perform cross-modality interaction for visible-thermal tracking. Specifically, we introduce a progressive Transformer network consisting of shallow interaction and deep interaction. The shallow interaction enables coarse cross-modality information transmission through adaptive channel exchange guided by channel correlations, in a bid to maintain the semantic consistency of multiple modalities and reduce the cross-modality distribution gap. Cross-modality features processed by shallow interaction are propagated into deep interaction to perform modality adaptation, where a coupling-decoupling module is designed to efficiently enhance modality-aware complementary information under the mutual interaction of multiple multimodal images. Finally, the obtained features undergo further modality integration through shallow interaction to enhance semantic consistency for prediction, further improving the tracking performance. This study contributes an innovative fusion method that advances the application of multisensors in tracking and monitoring. We conduct extensive experiments on three widely used tracking benchmarks, showcasing the superior performance of the proposed tracker.
Förderung
- National Natural Science Foundation of China
- Postdoctoral Fellowship Program of China Postdoctoral Science Foundation
Zitieren
APA
Luo, B., Liu, D., Peng, X., Zuo, H., Zhang, J., Li, M., & Wei, Y. (2025). Progressive Transformer With Multimodality Adaptation for RGB–T Tracking. IEEE Transactions on Instrumentation and Measurement, 74, 1-17. https://doi.org/10.1109/tim.2025.3575970
Vancouver
Luo B, Liu D, Peng X, Zuo H, Zhang J, Li M, et al. Progressive Transformer With Multimodality Adaptation for RGB–T Tracking. IEEE Transactions on Instrumentation and Measurement. 2025;74:1-17. doi:10.1109/tim.2025.3575970
BibTeX
@article{Luo2025,
author = {Luo, Binxin and Liu, Dongxu and Peng, Xianrong and Zuo, Haorui and Zhang, Jianlin and Li, Meihui and Wei, Yuxing},
title = {Progressive Transformer With Multimodality Adaptation for RGB–T Tracking},
journal = {IEEE Transactions on Instrumentation and Measurement},
year = {2025},
volume = {74},
pages = {1--17},
doi = {10.1109/tim.2025.3575970}
}
EUVIMED