EUVIMEDEUVIMED entdecken
Crossref

Mamba Adapter: Efficient Multi-Modal Fusion for Vision-Language Tracking

Liangtao Shi, Bineng Zhong, Qihua Liang, Xiantao Hu, Zhiyi Mo, Shuxiang Song

IEEE Transactions on Circuits and Systems for Video Technology · 2025-09 · 35 (9) · 9300-9311

DOI: 10.1109/tcsvt.2025.3557570 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-09. Verfasst von Liangtao Shi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 72 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Project of Guangxi Science and Technology, Research Project of Guangxi Normal University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Utilizing the high-level semantic information of language to compensate for the limitations of vision information is a highly regarded approach in single-object tracking. However, most existing vision-language (VL) trackers employ full-parameter fine-tuning, which can easily lead to catastrophic forgetting. Therefore, they fail to fully exploit the prior knowledge of pre-trained models from upstream tasks, resulting in unsatisfactory tracking performance. To alleviate the above problem, we propose a simple yet effective Vision- Language Tracking pipeline based on Mamba Adapter, named MAVLT, which adopts the idea of parameter-efficient fine-tuning (PEFT) to realize the interaction between vision-language modalities. This novel approach offers the following advantages: 1) The knowledge of the upstream pre-trained model is efficiently inherited by freezing its parameters. This ensures that the VL tracking framework only learns the modules for vision and language interaction, with a focus on the fusion between modalities. 2) The modal interaction between language and vision encoders is flexibly bridged in each encoder layer via proposed mamba adapter, enabling efficient interaction of visual and language information at multiple levels. Extensive experiments on five popular vision-language tracking benchmarks validate the effectiveness of the proposed MAVLT. Particularly, the MAVLT achieves 73.4% AUC score on the LaSOT benchmarks with only 0.18%(0.32M) of the total parameters updates. Code and models are available at https://github.com/GXNU-ZhongLab/MAVLT.

Förderung

  • National Natural Science Foundation of China
  • Project of Guangxi Science and Technology
  • Research Project of Guangxi Normal University
  • Innovation Project of Guangxi Graduate Education
  • Guangxi Colleges and Universities Key Laboratory of Intelligent Software

Zitieren

APA

Shi, L., Zhong, B., Liang, Q., Hu, X., Mo, Z., & Song, S. (2025). Mamba Adapter: Efficient Multi-Modal Fusion for Vision-Language Tracking. IEEE Transactions on Circuits and Systems for Video Technology, 35(9), 9300-9311. https://doi.org/10.1109/tcsvt.2025.3557570

Vancouver

Shi L, Zhong B, Liang Q, Hu X, Mo Z, Song S. Mamba Adapter: Efficient Multi-Modal Fusion for Vision-Language Tracking. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(9):9300-9311. doi:10.1109/tcsvt.2025.3557570

BibTeX
@article{Shi2025,
  author = {Shi, Liangtao and Zhong, Bineng and Liang, Qihua and Hu, Xiantao and Mo, Zhiyi and Song, Shuxiang},
  title = {Mamba Adapter: Efficient Multi-Modal Fusion for Vision-Language Tracking},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {9},
  pages = {9300--9311},
  doi = {10.1109/tcsvt.2025.3557570}
}
Kostenlos starten →