EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network

Jinpeng Wang, Nan Su, Chunhui Zhao, Yiming Yan, Shou Feng

Remote Sensing · 21.10.2024 · 16 (20) · 3904

DOI: 10.3390/rs16203904 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 21.10.2024. Verfasst von Jinpeng Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Excellent Youth Foundation of Heilongjiang Province of China, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the simultaneous acquisition of the infrared and optical remote sensing images of the same target becoming increasingly easy, using multi-modal data for high-performance object detection has become a research focus. In remote sensing multi-modal data, infrared images lack color information, it is hard to detect difficult targets with low contrast, and optical images are easily affected by illuminance. One of the most effective ways to solve this problem is to integrate multi-modal images for high-performance object detection. The challenge of fusion object detection lies in how to fully integrate multi-modal image features with significant modal differences and avoid introducing interference information while taking advantage of complementary advantages. To solve these problems, a new multi-modal fusion object detection method is proposed. In this paper, the method is improved in terms of two aspects: firstly, a new dual-branch asymmetric attention backbone network (DAAB) is designed, which uses a semantic information supplement module (SISM) and a detail information supplement module (DISM) to supplement and enhance infrared and RGB image information, respectively. Secondly, we propose a feature fusion pyramid network (FFPN), which uses a Transformer-like strategy to carry out multi-modal feature fusion and suppress features that are not conducive to fusion during the fusion process. This method is a state-of-the-art process for both FLIR-aligned and DroneVehicle datasets. Experiments show that this method has strong competitiveness and generalization performance.

Förderung

  • National Natural Science Foundation of China
  • Excellent Youth Foundation of Heilongjiang Province of China
  • Fundamental Research Funds for the Central Universities
  • Heilongjiang Province Key Research and Development Project
  • Key Laboratory of Target Cognition and Application Technology

Zitieren

APA

Wang, J., Su, N., Zhao, C., Yan, Y., & Feng, S. (2024). Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network. Remote Sensing, 16(20), 3904. https://doi.org/10.3390/rs16203904

Vancouver

Wang J, Su N, Zhao C, Yan Y, Feng S. Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network. Remote Sensing. 2024;16(20):3904. doi:10.3390/rs16203904

BibTeX
@article{Wang2024,
  author = {Wang, Jinpeng and Su, Nan and Zhao, Chunhui and Yan, Yiming and Feng, Shou},
  title = {Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {20},
  pages = {3904},
  doi = {10.3390/rs16203904}
}
Kostenlos starten →