EUVIMEDEUVIMED entdecken
Crossref

Progressive Transformer With Multimodality Adaptation for RGB–T Tracking

Binxin Luo, Dongxu Liu, Xianrong Peng, Haorui Zuo, Jianlin Zhang, Meihui Li, Yuxing Wei

IEEE Transactions on Instrumentation and Measurement · 2025 · 74 · 1-17

DOI: 10.1109/tim.2025.3575970 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Instrumentation and Measurement, 2025. Verfasst von Binxin Luo und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 60 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Postdoctoral Fellowship Program of China Postdoctoral Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the continuous advancement of sensor technologies, the combined use of RGB images and thermal-infrared (TIR) images has become a critical approach for improving tracking performance by generating powerful fusion features. However, the inherent spatial-spectral discrepancies between RGB and thermal modalities create feature distribution mismatches that hinder effective cross-modality alignment, while their distinct focus regions introduce cross-modality semantic conflicts during fusion. To tackle this issue, we propose a progressive Transformer network with multimodality adaptation (PTrMA) to perform cross-modality interaction for visible-thermal tracking. Specifically, we introduce a progressive Transformer network consisting of shallow interaction and deep interaction. The shallow interaction enables coarse cross-modality information transmission through adaptive channel exchange guided by channel correlations, in a bid to maintain the semantic consistency of multiple modalities and reduce the cross-modality distribution gap. Cross-modality features processed by shallow interaction are propagated into deep interaction to perform modality adaptation, where a coupling-decoupling module is designed to efficiently enhance modality-aware complementary information under the mutual interaction of multiple multimodal images. Finally, the obtained features undergo further modality integration through shallow interaction to enhance semantic consistency for prediction, further improving the tracking performance. This study contributes an innovative fusion method that advances the application of multisensors in tracking and monitoring. We conduct extensive experiments on three widely used tracking benchmarks, showcasing the superior performance of the proposed tracker.

Förderung

  • National Natural Science Foundation of China
  • Postdoctoral Fellowship Program of China Postdoctoral Science Foundation

Zitieren

APA

Luo, B., Liu, D., Peng, X., Zuo, H., Zhang, J., Li, M., & Wei, Y. (2025). Progressive Transformer With Multimodality Adaptation for RGB–T Tracking. IEEE Transactions on Instrumentation and Measurement, 74, 1-17. https://doi.org/10.1109/tim.2025.3575970

Vancouver

Luo B, Liu D, Peng X, Zuo H, Zhang J, Li M, et al. Progressive Transformer With Multimodality Adaptation for RGB–T Tracking. IEEE Transactions on Instrumentation and Measurement. 2025;74:1-17. doi:10.1109/tim.2025.3575970

BibTeX
@article{Luo2025,
  author = {Luo, Binxin and Liu, Dongxu and Peng, Xianrong and Zuo, Haorui and Zhang, Jianlin and Li, Meihui and Wei, Yuxing},
  title = {Progressive Transformer With Multimodality Adaptation for RGB–T Tracking},
  journal = {IEEE Transactions on Instrumentation and Measurement},
  year = {2025},
  volume = {74},
  pages = {1--17},
  doi = {10.1109/tim.2025.3575970}
}
Kostenlos starten →