Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion
Sensors · 12.05.2025 · 25 (10) · 3039
DOI: 10.3390/s25103039 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 12.05.2025. Verfasst von Zhe Wang und 1 weiteren Autor:innen. Beteiligte Einrichtung: School of Aeronautics and Astronautics, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen 518107, China. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Shenzhen Science and Technology Program.
Abstract (Originalsprache)
Aerial object detection plays a critical role in numerous fields, utilizing the flexibility of airborne platforms to achieve real-time tasks. Combining visible and infrared sensors can overcome limitations under low-light conditions, enabling full-time tasks. While feature-level fusion methods exhibit comparable performances in visible–infrared multispectral object detection, they suffer from heavy model size, inadequate inference speed and visible light preferences caused by inherent modality imbalance, limiting their applications in airborne platform deployment. To address these challenges, this paper proposes a YOLO-based real-time multispectral fusion framework combining pixel-level fusion with dynamic modality-balanced augmentation called Full-time Multispectral Pixel-wise Fusion Network (FMPFNet). Firstly, we introduce the Multispectral Luminance Weighted Fusion (MLWF) module consisting of attention-based modality reconstruction and feature fusion. By leveraging YUV color space transformation, this module efficiently fuses RGB and IR modalities while minimizing computational overhead. We also propose the Dynamic Modality Dropout and Threshold Masking (DMDTM) strategy, which balances modality attention and improves detection performance in low-light scenarios. Additionally, we refine our model to enhance the detection of small rotated objects, a requirement commonly encountered in aerial detection applications. Experimental results on the DroneVehicle dataset demonstrate that our FMPFNet achieves 76.80% mAP50 and 132 FPS, outperforming state-of-the-art feature-level fusion methods in both accuracy and inference speed.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Aeronautics and Astronautics, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen 518107, China.
- Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen 518107, China.
ORCID-Profile: Zhe Wang, Qingling Zhang
Förderung
- Shenzhen Science and Technology Program
Zitieren
APA
Wang, Z., & Zhang, Q. (2025). Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion. Sensors, 25(10), 3039. https://doi.org/10.3390/s25103039
Vancouver
Wang Z, Zhang Q. Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion. Sensors. 2025;25(10):3039. doi:10.3390/s25103039
BibTeX
@article{Wang2025,
author = {Wang, Zhe and Zhang, Qingling},
title = {Real-Time Aerial Multispectral Object Detection with Dynamic Modality-Balanced Pixel-Level Fusion},
journal = {Sensors},
year = {2025},
volume = {25},
number = {10},
pages = {3039},
doi = {10.3390/s25103039}
}
EUVIMED