Transformer-Based RGBT Tracking With Spatio-Temporal Information Fusion
IEEE Sensors Journal · 01.07.2025 · 25 (13) · 25386-25396
DOI: 10.1109/jsen.2025.3575188 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 01.07.2025. Verfasst von Di Yuan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Postdoctoral Science Foundation, Basic and Applied Basic Research Foundation of Guangdong Province.
Abstract (Originalsprache)
RGB-thermal (RGBT) tracking usually uses an RGB tracker as the base model, and then uses the RGBT dataset to fully fine-tune the model. These methods ignore the differences in target features in the RGB domain and the TIR domain. At the same time, existing RGBT trackers match the spatial features of the initial template and the search image, ignoring the role of temporal information in RGBT tracking, resulting in the failure of the tracker to track in complex scenarios such as change in the appearance of the target and occlusion. To address the above problems, we propose a simple and efficient tracker called STTrack. The tracker adopts a symmetric dual-stream structure, which consists of several fine tuning transformer (FT Transformer) encoders, a prediction head, and an online update module. Specifically, the FT Transformer encoder first adds some trainable parameters to the frozen pretrained RGB-based tracker, transfers the feature extraction capability from the RGB domain to the TIR domain, and enhances the model’s perception of cross-modal data; second, the output features of the RGB and TIR modalities are fused and fed into the prediction head to obtain the target’s position; finally, the online update module obtains an online template with temporal information, which complements the spatial information provided by the initial template. The spatio-temporal information provided by the dual templates improves the RGBT tracker’s ability to locate targets in complex environments. Extensive quantitative and qualitative experiments demonstrate that our approach achieves state-of-the-art performance on four most popular RGBT benchmarks and runs at 32 FPS in real time.
Förderung
- National Natural Science Foundation of China
- China Postdoctoral Science Foundation
- Basic and Applied Basic Research Foundation of Guangdong Province
- Science and Technology Projects in Guangzhou
Zitieren
APA
Yuan, D., Zhang, H., Liu, Q., Chang, X., & He, Z. (2025). Transformer-Based RGBT Tracking With Spatio-Temporal Information Fusion. IEEE Sensors Journal, 25(13), 25386-25396. https://doi.org/10.1109/jsen.2025.3575188
Vancouver
Yuan D, Zhang H, Liu Q, Chang X, He Z. Transformer-Based RGBT Tracking With Spatio-Temporal Information Fusion. IEEE Sensors Journal. 2025;25(13):25386-25396. doi:10.1109/jsen.2025.3575188
BibTeX
@article{Yuan2025,
author = {Yuan, Di and Zhang, Haiping and Liu, Qiao and Chang, Xiaojun and He, Zhenyu},
title = {Transformer-Based RGBT Tracking With Spatio-Temporal Information Fusion},
journal = {IEEE Sensors Journal},
year = {2025},
volume = {25},
number = {13},
pages = {25386--25396},
doi = {10.1109/jsen.2025.3575188}
}
EUVIMED