EUVIMEDEUVIMED entdecken
Crossref

QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking

Huijie Fan, Zhencheng Yu, Qiang Wang, Baojie Fan, Yandong Tang

IEEE Transactions on Image Processing · 2024 · 33 · 3187-3199

DOI: 10.1109/tip.2024.3393298 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2024. Verfasst von Huijie Fan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 46-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Youth Innovation Promotion Association of Chinese Academy of Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing RGB-Thermal trackers usually treat intra-modal feature extraction and inter-modal feature fusion as two separate processes, therefore the mutual promotion of extraction and fusion is neglected. Then, the complementary advantages of RGB-T fusion are not fully exploited, and the independent feature extraction is not adaptive to modal quality fluctuation during tracking. To address the limitations, we design a joint-modality query fusion network, in which the intra-modal feature extraction and the inter-modal fusion are coupled together and promote each other via joint-modality queries. The queries are initialized based on the multimodal features of the current frame, making the subsequent fusion adaptive to modal quality fluctuation during tracking. Then the joint-modality query fusion (JQF) utilizes the queries to interact with RGB-T features, allowing the intra-modal enhancement and the inter-modal interactions to be unified for mutual promotion. In this way, JQF can distinguish and enhance the complementary modality features, while filtering out redundant information. For real-time tracking, we propose regional cross-attention for cross-modal interactions to reduce computational cost. Our end-to-end tracker sets a new state-of-the-art performance on multiple RGBT tracking benchmarks including LasHeR, VTUAV, RGBT234 and GTOT, while running at a real-time speed.

Förderung

  • National Natural Science Foundation of China
  • Youth Innovation Promotion Association of Chinese Academy of Sciences

Zitieren

APA

Fan, H., Yu, Z., Wang, Q., Fan, B., & Tang, Y. (2024). QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking. IEEE Transactions on Image Processing, 33, 3187-3199. https://doi.org/10.1109/tip.2024.3393298

Vancouver

Fan H, Yu Z, Wang Q, Fan B, Tang Y. QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking. IEEE Transactions on Image Processing. 2024;33:3187-3199. doi:10.1109/tip.2024.3393298

BibTeX
@article{Fan2024,
  author = {Fan, Huijie and Yu, Zhencheng and Wang, Qiang and Fan, Baojie and Tang, Yandong},
  title = {QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking},
  journal = {IEEE Transactions on Image Processing},
  year = {2024},
  volume = {33},
  pages = {3187--3199},
  doi = {10.1109/tip.2024.3393298}
}
Kostenlos starten →