EUVIMEDEUVIMED entdecken
Crossref

Learning Multifrequency Integration Network for RGBT Tracking

Jiatian Mei, Juxiang Zhou, Jun Wang, Jia Hao, Dongming Zhou, Jinde Cao

IEEE Sensors Journal · 01.05.2024 · 24 (9) · 15517-15530

DOI: 10.1109/jsen.2024.3370144 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 01.05.2024. Verfasst von Jiatian Mei und 5 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Yunnan Provincial Basic Research Plan-General Project, Scientific Research Fund Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

RGBT tracking is an attractive topic that benefits from the complementarity of visible and thermal sensors to better handle tracking tasks in atrocious scenarios. Existing RGBT trackers typically introduce self-attention (SA) to capture long-range dependencies. However, recent findings suggest that SA is a low-pass filter, meaning that high-frequency clues involving local edges and texture may be repressed. Aiming at the problem, this article comprehensively considers the multifrequency knowledge of heterogeneous modalities and proposes a learning multifrequency integration network (LMINet) for RGBT tracking to effectively implement adaptive extraction, enhancement, and integration of multifrequency cues. The proposed LMINet primarily benefits from the deployment of three crucial components: pattern-aware reinforcement (PR), multifrequency enhancement (ME), and MI. Specifically, the PR part consists of a carefully designed reinforcement unit (RU) and learnable weighting strategy 1 (LWS1). The former extracts information from the data flow to enhance the backbone, while the latter is a data-driven regulation mechanism that adaptively adjusts the enhancement intensity via learning the input. Then, the ME component separates high- and low-frequency knowledge via high-level branch (HB) and common unit (CU) and further adjusts the improvement intensity of multifrequency cues via the learning of LWS2 to achieve intramodal refinement. Moreover, the MI part first extracts high- and low-frequency signals via HB and low-level branch (LB) and implements cross-modal integration of high- and low-frequency cues through LWS3, respectively. Extensive experimental results on GTOT, RGBT234, and LasHeR demonstrate that the proposed LMINet is effective and competitive with state-of-the-art algorithms. The code will be open-sourced at https://github.com/mjt1312/Lminet.

Förderung

  • National Natural Science Foundation of China
  • Yunnan Provincial Basic Research Plan-General Project
  • Scientific Research Fund Project
  • Education Department of Yunnan Province

Zitieren

APA

Mei, J., Zhou, J., Wang, J., Hao, J., Zhou, D., & Cao, J. (2024). Learning Multifrequency Integration Network for RGBT Tracking. IEEE Sensors Journal, 24(9), 15517-15530. https://doi.org/10.1109/jsen.2024.3370144

Vancouver

Mei J, Zhou J, Wang J, Hao J, Zhou D, Cao J. Learning Multifrequency Integration Network for RGBT Tracking. IEEE Sensors Journal. 2024;24(9):15517-15530. doi:10.1109/jsen.2024.3370144

BibTeX
@article{Mei2024,
  author = {Mei, Jiatian and Zhou, Juxiang and Wang, Jun and Hao, Jia and Zhou, Dongming and Cao, Jinde},
  title = {Learning Multifrequency Integration Network for RGBT Tracking},
  journal = {IEEE Sensors Journal},
  year = {2024},
  volume = {24},
  number = {9},
  pages = {15517--15530},
  doi = {10.1109/jsen.2024.3370144}
}
Kostenlos starten →