Multi-Stage Cross-Modality Feature Interaction for RGB-Thermal Multi-Object Tracking
IEEE Transactions on Circuits and Systems for Video Technology · 2026-02 · 36 (2) · 2449-2463
DOI: 10.1109/tcsvt.2025.3612499 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-02. Verfasst von Jianbo Ma und 6 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: Frontier Research Fund of the Institute of Optics and Electronics, China Academy of Sciences.
Abstract (Originalsprache)
RGB-Thermal multi-object tracking (RGB-T MOT) focuses on tracking multiple objects in complex scenarios, such as nighttime and low-light conditions, which is crucial for various applications, including video surveillance and drone monitoring. Existing RGB-T MOT studies typically merge multi-source features in a single stage before the backbone network. These works fail to preserve fine-grained details and struggle with context-dependent fusion, resulting in suboptimal tracking performance. In this paper, we propose a novel multi-stage cross-modality spatial-temporal feature interaction network (MCTrack), which emphasizes two main aspects: temporal-aware learning and cross-modality information interaction. For temporal-aware learning, we introduce the temporal salient feature interaction (TSFI) module, which ensures trajectory continuity by capturing dynamic spatial variation of objects across RGB-T video pairs, enabling consistent object tracking over time. For cross-modality information interaction, we propose the bidirectional modality interaction (Bi-MI) module, which employs cross-modality transformers to extract complementary features from both RGB and thermal modalities at multiple stages of feature extraction, thereby improving tracking adaptability in diverse tracking scenarios. Additionally, we propose the cross-modality complementary mask (CCM) strategy, which applies non-overlapping random masks to feature maps to improve the robustness of cross-modality feature interaction. Extensive experiments and comparative analyses demonstrate that our MCTrack surpasses state-of-the-art trackers on both publicly available VT-MOT and UniRTL-MOT RGB-T datasets. The code is available at https://github.com/ydhcg-BoBo/RGB-T-MOT
Förderung
- Frontier Research Fund of the Institute of Optics and Electronics, China Academy of Sciences
Zitieren
APA
Ma, J., Luo, H., Niu, S., Zhao, P., Liu, Y., Wei, Y., & Zhang, J. (2026). Multi-Stage Cross-Modality Feature Interaction for RGB-Thermal Multi-Object Tracking. IEEE Transactions on Circuits and Systems for Video Technology, 36(2), 2449-2463. https://doi.org/10.1109/tcsvt.2025.3612499
Vancouver
Ma J, Luo H, Niu S, Zhao P, Liu Y, Wei Y, et al. Multi-Stage Cross-Modality Feature Interaction for RGB-Thermal Multi-Object Tracking. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(2):2449-2463. doi:10.1109/tcsvt.2025.3612499
BibTeX
@article{Ma2026,
author = {Ma, Jianbo and Luo, Hui and Niu, Shuaicheng and Zhao, Peilin and Liu, Yunfeng and Wei, Yuxing and Zhang, Jianlin},
title = {Multi-Stage Cross-Modality Feature Interaction for RGB-Thermal Multi-Object Tracking},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {2},
pages = {2449--2463},
doi = {10.1109/tcsvt.2025.3612499}
}
EUVIMED