EUVIMEDEUVIMED entdecken
Crossref

Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks

Andong Lu, Chenglong Li, Jiacong Zhao, Jin Tang, Bin Luo

International Journal of Computer Vision · 2025-05 · 133 (5) · 2599-2619

DOI: 10.1007/s11263-024-02311-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 07.12.2024. Verfasst von Andong Lu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Anhui Province, Natural Science Foundation of Anhui Higher Education Institution.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Current RGBT tracking research relies on the complete multi-modality input, but modal information might miss due to some factors such as thermal sensor self-calibration and data transmission error, called modality-missing challenge in this work. To address this challenge, we propose a novel invertible prompt learning approach, which integrates the content-preserving prompts into a well-trained tracking model to adapt to various modality-missing scenarios, for robust RGBT tracking. Given one modality-missing scenario, we propose to utilize the available modality to generate the prompt of the missing modality to adapt to RGBT tracking model. However, the cross-modality gap between available and missing modalities usually causes semantic distortion and information loss in prompt generation. To handle this issue, we design the invertible prompter by incorporating the full reconstruction of the input available modality from the generated prompt. To provide a comprehensive evaluation platform, we construct several high-quality benchmark datasets, in which various modality-missing scenarios are considered to simulate real-world challenges. Extensive experiments on three modality-missing benchmark datasets show that our method achieves significant performance improvements compared with state-of-the-art methods. We have released the code and simulation datasets at: https://github.com/mmic-lcl.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Anhui Province
  • Natural Science Foundation of Anhui Higher Education Institution

Zitieren

APA

Lu, A., Li, C., Zhao, J., Tang, J., & Luo, B. (2025). Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks. International Journal of Computer Vision, 133(5), 2599-2619. https://doi.org/10.1007/s11263-024-02311-4

Vancouver

Lu A, Li C, Zhao J, Tang J, Luo B. Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks. International Journal of Computer Vision. 2025;133(5):2599-2619. doi:10.1007/s11263-024-02311-4

BibTeX
@article{Lu2025,
  author = {Lu, Andong and Li, Chenglong and Zhao, Jiacong and Tang, Jin and Luo, Bin},
  title = {Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks},
  journal = {International Journal of Computer Vision},
  year = {2025},
  volume = {133},
  number = {5},
  pages = {2599--2619},
  doi = {10.1007/s11263-024-02311-4}
}
Kostenlos starten →