EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Multispectral Object Detection Based on Multilevel Feature Fusion and Dual Feature Modulation

Jin Sun, Mingfeng Yin, Zhiwei Wang, Tao Xie, Shaoyi Bei

Electronics · 2024-01-21 · 13 (2) · 443

DOI: 10.3390/electronics13020443 ↗

At a glance

Journal article in Electronics, published online on 2024-01-21. Written by Jin Sun and 4 co-authors. Cited 29 times so far (Crossref); 53 references. The full text is freely available (open access, CC BY 4.0). Funded by: National Natural Science Foundation of China, Natural Science Research Project of Colleges and Universities of Jiangsu Province, Changzhou Applied Basic Research Project (Medium subsidy).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Multispectral object detection is a crucial technology in remote sensing image processing, particularly in low-light environments. Most current methods extract features at a single scale, resulting in the fusion of invalid features and the failure to detect small objects. To address these issues, we propose a multispectral object detection network based on multilevel feature fusion and dual feature modulation (GMD-YOLO). Firstly, a novel dual-channel CSPDarknet53 network is used to extract deep features from visible-infrared images. This network incorporates a Ghost module, which generates additional feature maps through a series of linear operations, achieving a balance between accuracy and speed. Secondly, the multilevel feature fusion (MLF) module is designed to utilize cross-modal information through the construction of hierarchical residual connections. This approach strengthens the complementarity between different modalities, allowing the network to improve multiscale representation capabilities at a more refined granularity level. Finally, a dual feature modulation (DFM) decoupling head is introduced to enhance small object detection. This decoupled head effectively meets the distinct requirements of classification and localization tasks. GMD-YOLO is validated on three public visible-infrared datasets: DroneVehicle, KAIST, and LLVIP. DroneVehicle and LLVIP achieved mAP@0.5 of 78.0% and 98.0%, outperforming baseline methods by 3.6% and 4.4%, respectively. KAIST exhibited an MR of 7.73% with an FPS of 61.7. Experimental results demonstrated that our method surpasses existing advanced methods and exhibits strong robustness.

Funding

  • National Natural Science Foundation of China
  • Natural Science Research Project of Colleges and Universities of Jiangsu Province
  • Changzhou Applied Basic Research Project (Medium subsidy)

Cite

APA

Sun, J., Yin, M., Wang, Z., Xie, T., & Bei, S. (2024). Multispectral Object Detection Based on Multilevel Feature Fusion and Dual Feature Modulation. Electronics, 13(2), 443. https://doi.org/10.3390/electronics13020443

Vancouver

Sun J, Yin M, Wang Z, Xie T, Bei S. Multispectral Object Detection Based on Multilevel Feature Fusion and Dual Feature Modulation. Electronics. 2024;13(2):443. doi:10.3390/electronics13020443

BibTeX
@article{Sun2024,
  author = {Sun, Jin and Yin, Mingfeng and Wang, Zhiwei and Xie, Tao and Bei, Shaoyi},
  title = {Multispectral Object Detection Based on Multilevel Feature Fusion and Dual Feature Modulation},
  journal = {Electronics},
  year = {2024},
  volume = {13},
  number = {2},
  pages = {443},
  doi = {10.3390/electronics13020443}
}
Start for free →