EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion

Zhe Wang iD, Qingling Zhang iD

Sensors · 12.05.2025 · 25 (10) · 3039

DOI: 10.3390/s25103039 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 12.05.2025. Verfasst von Zhe Wang und 1 weiteren Autor:innen. Beteiligte Einrichtung: School of Aeronautics and Astronautics, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen 518107, China. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Shenzhen Science and Technology Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aerial object detection plays a critical role in numerous fields, utilizing the flexibility of airborne platforms to achieve real-time tasks. Combining visible and infrared sensors can overcome limitations under low-light conditions, enabling full-time tasks. While feature-level fusion methods exhibit comparable performances in visible–infrared multispectral object detection, they suffer from heavy model size, inadequate inference speed and visible light preferences caused by inherent modality imbalance, limiting their applications in airborne platform deployment. To address these challenges, this paper proposes a YOLO-based real-time multispectral fusion framework combining pixel-level fusion with dynamic modality-balanced augmentation called Full-time Multispectral Pixel-wise Fusion Network (FMPFNet). Firstly, we introduce the Multispectral Luminance Weighted Fusion (MLWF) module consisting of attention-based modality reconstruction and feature fusion. By leveraging YUV color space transformation, this module efficiently fuses RGB and IR modalities while minimizing computational overhead. We also propose the Dynamic Modality Dropout and Threshold Masking (DMDTM) strategy, which balances modality attention and improves detection performance in low-light scenarios. Additionally, we refine our model to enhance the detection of small rotated objects, a requirement commonly encountered in aerial detection applications. Experimental results on the DroneVehicle dataset demonstrate that our FMPFNet achieves 76.80% mAP50 and 132 FPS, outperforming state-of-the-art feature-level fusion methods in both accuracy and inference speed.

Schlagwörter

Stichwörter der Autor:innen

Dynamic Modality Dropout and Threshold MaskingMultispectral Luminance Weighted Fusionaerial object detectionfull timemodality imbalancemultispectral fusionreal time

Autor:innen und Institutionen

  • School of Aeronautics and Astronautics, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen 518107, China.
  • Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen 518107, China.

ORCID-Profile: Zhe Wang, Qingling Zhang

Förderung

  • Shenzhen Science and Technology Program

Zitieren

APA

Wang, Z., & Zhang, Q. (2025). Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion. Sensors, 25(10), 3039. https://doi.org/10.3390/s25103039

Vancouver

Wang Z, Zhang Q. Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion. Sensors. 2025;25(10):3039. doi:10.3390/s25103039

BibTeX
@article{Wang2025,
  author = {Wang, Zhe and Zhang, Qingling},
  title = {Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {10},
  pages = {3039},
  doi = {10.3390/s25103039}
}
Kostenlos starten →