EUVIMEDEUVIMED entdecken
Crossref

Unified Multi-Modal Tracking via Proxy Visual Prompts

Hongtao Yang, Bineng Zhong, Yaozong Zheng, Qihua Liang, Zhiruo Zhu, Shuxiang Song

IEEE Transactions on Automation Science and Engineering · 2026 · 23 · 9282-9292

DOI: 10.1109/tase.2026.3689934 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Automation Science and Engineering, 2026. Verfasst von Hongtao Yang und 5 weiteren Autor:innen. Gefördert von: Project of Guangxi Science and Technology, National Natural Science Foundation of China, Guangxi “Young Bagui Scholar” Teams for Innovation and Research Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent unified multi-modal tracking frameworks often encounter high computational overhead due to complex fusion operations. In this paper, we propose PoATrack, a proxy-based bidirectional fusion framework designed to facilitate efficient multi-modal tracking via interactive proxy prompts. Specifically, the framework treats features from all modalities equally and introduces an adaptive feature enhancement module to improve spatial representations within the search region. To further reduce the fusion cost, a lightweight proxy prompt fusion module is developed following a two-stage strategy: (1) representing modality-specific features through proxy visual prompts, and (2) dynamically learning hierarchical cross-modal relationships via proxy-guided fusion for enriched contextual modeling. Extensive experiments on six public benchmarks (RGB-T, RGB-E, and RGB-D) demonstrate that PoATrack achieves competitive accuracy while operating $1.8\times $ faster than SDSTrack under identical hardware settings. Note to Practitioners—This work addresses a practical challenge in multi-modal object tracking, where robustness under complex environmental conditions—such as low visibility, dynamic motion, or occlusion—is critical. Existing unified tracking frameworks often rely on direct fusion of full-resolution features from multiple modalities, which can introduce excessive redundancy and hamper tracking accuracy. PoATrack proposes an alternative solution by introducing proxy visual prompts—compact, informative tokens that facilitate efficient and focused multi-modal fusion without sacrificing discriminative power. In addition, the feature refinement module adaptively enhances spatial features in non-RGB modalities, enabling more accurate localization in challenging scenes. The design of PoATrack balances performance and fusion efficiency, achieving strong results across diverse RGB-T, RGB-D, and RGB-E benchmarks. These capabilities make it a promising component for high-performance perception systems in video analysis, autonomous platforms, robotic systems, or other multi-sensor applications. In future work, we aim to further explore multi-modal visual tracking from the perspectives of long-term temporal modeling, semantic consistency across modalities, and efficient adaptation to diverse application scenarios.

Förderung

  • Project of Guangxi Science and Technology
  • National Natural Science Foundation of China
  • Guangxi “Young Bagui Scholar” Teams for Innovation and Research Project
  • Research Project of Guangxi Normal University

Zitieren

APA

Yang, H., Zhong, B., Zheng, Y., Liang, Q., Zhu, Z., & Song, S. (2026). Unified Multi-Modal Tracking via Proxy Visual Prompts. IEEE Transactions on Automation Science and Engineering, 23, 9282-9292. https://doi.org/10.1109/tase.2026.3689934

Vancouver

Yang H, Zhong B, Zheng Y, Liang Q, Zhu Z, Song S. Unified Multi-Modal Tracking via Proxy Visual Prompts. IEEE Transactions on Automation Science and Engineering. 2026;23:9282-9292. doi:10.1109/tase.2026.3689934

BibTeX
@article{Yang2026,
  author = {Yang, Hongtao and Zhong, Bineng and Zheng, Yaozong and Liang, Qihua and Zhu, Zhiruo and Song, Shuxiang},
  title = {Unified Multi-Modal Tracking via Proxy Visual Prompts},
  journal = {IEEE Transactions on Automation Science and Engineering},
  year = {2026},
  volume = {23},
  pages = {9282--9292},
  doi = {10.1109/tase.2026.3689934}
}
Kostenlos starten →