BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts
IEEE Transactions on Circuits and Systems for Video Technology · 2025-12 · 35 (12) · 11908-11921
DOI: 10.1109/tcsvt.2025.3587703 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-12. Verfasst von Qingguo Meng und 2 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Anhui Province.
Abstract (Originalsprache)
Multi-modal object tracking combines visible light with auxiliary modalities for better tracking accuracy, including sub-tasks like RGB-T (RGB and Thermal), RGB-D (RGB and Depth), and RGB-E (RGB and Event) tracking. However, existing algorithms, designed and trained for specific tasks, impose high costs on universal tracking systems as they rely on multiple independent trackers. Despite efforts to develop universal multi-modal trackers, these methods often fail to effectively integrate both modality-agnostic and modality-specific information across various modalities, constraining their performance. In this paper, we propose the Blind Multi-Modal Tracking with Route-Dynamic Mixture of Experts (BR-MoE). This approach can simultaneously model both modality-agnostic and modality-specific features of each modality within a unified structure and model parameters, all without prior knowledge of the modality types. In particular, BR-MoE employs Vision Transformer (ViT) to extract modality-agnostic features, while embedding four parallel modality-specific experts in each ViT block to capture specific features. Additionally, BR-MoE incorporates a Modality-Aware Module (MAM) that adaptively assigns weights to the modality-specific experts based on the input and combines their outputs to generate the final feature representation. To achieve this, BR-MoE is trained in two phases. In the first, we manually provide modality types to supervise MAM and select the right expert. In the second, we do a weighted aggregation of experts’ features based on MAM’s decision and fine-tune the model for feature changes. Experiments on five multi-modal tracking datasets show BR-MoE gets state-of-the-art performance on RGBT234, DepthTrack and VisEvent datasets, and comparable performance on LasHeR and VOT-RGBD22 datasets.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Anhui Province
Zitieren
APA
Meng, Q., Lu, A., & Jin, Z. (2025). BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts. IEEE Transactions on Circuits and Systems for Video Technology, 35(12), 11908-11921. https://doi.org/10.1109/tcsvt.2025.3587703
Vancouver
Meng Q, Lu A, Jin Z. BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(12):11908-11921. doi:10.1109/tcsvt.2025.3587703
BibTeX
@article{Meng2025,
author = {Meng, Qingguo and Lu, Andong and Jin, Zhe},
title = {BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {12},
pages = {11908--11921},
doi = {10.1109/tcsvt.2025.3587703}
}
EUVIMED