EUVIMEDEUVIMED entdecken
Crossref

BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts

Qingguo Meng, Andong Lu, Zhe Jin

IEEE Transactions on Circuits and Systems for Video Technology · 2025-12 · 35 (12) · 11908-11921

DOI: 10.1109/tcsvt.2025.3587703 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-12. Verfasst von Qingguo Meng und 2 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Anhui Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modal object tracking combines visible light with auxiliary modalities for better tracking accuracy, including sub-tasks like RGB-T (RGB and Thermal), RGB-D (RGB and Depth), and RGB-E (RGB and Event) tracking. However, existing algorithms, designed and trained for specific tasks, impose high costs on universal tracking systems as they rely on multiple independent trackers. Despite efforts to develop universal multi-modal trackers, these methods often fail to effectively integrate both modality-agnostic and modality-specific information across various modalities, constraining their performance. In this paper, we propose the Blind Multi-Modal Tracking with Route-Dynamic Mixture of Experts (BR-MoE). This approach can simultaneously model both modality-agnostic and modality-specific features of each modality within a unified structure and model parameters, all without prior knowledge of the modality types. In particular, BR-MoE employs Vision Transformer (ViT) to extract modality-agnostic features, while embedding four parallel modality-specific experts in each ViT block to capture specific features. Additionally, BR-MoE incorporates a Modality-Aware Module (MAM) that adaptively assigns weights to the modality-specific experts based on the input and combines their outputs to generate the final feature representation. To achieve this, BR-MoE is trained in two phases. In the first, we manually provide modality types to supervise MAM and select the right expert. In the second, we do a weighted aggregation of experts’ features based on MAM’s decision and fine-tune the model for feature changes. Experiments on five multi-modal tracking datasets show BR-MoE gets state-of-the-art performance on RGBT234, DepthTrack and VisEvent datasets, and comparable performance on LasHeR and VOT-RGBD22 datasets.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Anhui Province

Zitieren

APA

Meng, Q., Lu, A., & Jin, Z. (2025). BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts. IEEE Transactions on Circuits and Systems for Video Technology, 35(12), 11908-11921. https://doi.org/10.1109/tcsvt.2025.3587703

Vancouver

Meng Q, Lu A, Jin Z. BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(12):11908-11921. doi:10.1109/tcsvt.2025.3587703

BibTeX
@article{Meng2025,
  author = {Meng, Qingguo and Lu, Andong and Jin, Zhe},
  title = {BR-MoE: Blind Multi-Modal Tracking With Route-Dynamic Mixture of Experts},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {12},
  pages = {11908--11921},
  doi = {10.1109/tcsvt.2025.3587703}
}
Kostenlos starten →