EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

GEM-YOLO: A Lightweight and Real-Time RGBT Object Detector with Gated Multimodal Fusion

Lijuan Wang, Zuchao Bao, Dongming Lu

Sensors · 25.03.2026 · 26 (7) · 2035

DOI: 10.3390/s26072035 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 25.03.2026. Verfasst von Lijuan Wang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Electronic and Optical Engineering, Nanjing University of Science and Technology ZiJin College, Nanjing 210023, China. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Project supported by Basic Science (Nature Science) Research Project of Jiangsu Province for Colleges and Universities, Basic Science (Nature Science) Research Project of Jiangsu Province for Colleges and Universities.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Red–Green–Blue–Thermal (RGBT) object detection is critical for robust all-weather perception. However, deploying dual-stream networks on resource-constrained edge devices is severely hindered by insufficiently adaptive multimodal fusion, the loss of small-object features during downsampling, and substantial computational overhead. To address these challenges, we propose GEM-YOLO, a real-time and lightweight RGBT detector. Specifically, an Adaptive Multimodal Gated Fusion Mechanism (GFM) is designed to dynamically calibrate modality weights and suppress noise. Furthermore, Space-to-Depth (SPD) convolutions are integrated into the backbone to achieve lossless downsampling, preventing the feature collapse of small targets. Finally, a lightweight Ghost-Neck is constructed using Ghost modules and GSConv to eliminate computational redundancy. Extensive experiments on the Forward-Looking Infrared (FLIR) and Multi-Modal Multispectral Fusion Dataset (M3FD) datasets demonstrate the effectiveness of the proposed method. With only 7.58 Giga Floating-Point Operations (GFLOPs) and 3.44 million parameters (M), GEM-YOLO reduces the computational cost by 18.6% relative to the dual-stream YOLOv11n baseline. Concurrently, it achieves competitive mean Average Precision at IoU = 0.5 (mAP@50) scores of 82.8% and 69.0% on FLIR and M3FD, respectively, with more evident gains on small-target localization. In practice, GEM-YOLO maintains competitive detection performance while keeping computational overhead low, making it promising for real-time multispectral perception on resource-constrained edge platforms.

Schlagwörter

Stichwörter der Autor:innen

edge deploymentinfrared-visible fusionmodality calibrationmultispectral object detectionsmall object detection

Autor:innen und Institutionen

  • School of Electronic and Optical Engineering, Nanjing University of Science and Technology ZiJin College, Nanjing 210023, China.
  • School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China.

Förderung

  • Project supported by Basic Science (Nature Science) Research Project of Jiangsu Province for Colleges and Universities
  • Basic Science (Nature Science) Research Project of Jiangsu Province for Colleges and Universities

Zitieren

APA

Wang, L., Bao, Z., & Lu, D. (2026). GEM-YOLO: A Lightweight and Real-Time RGBT Object Detector with Gated Multimodal Fusion. Sensors, 26(7), 2035. https://doi.org/10.3390/s26072035

Vancouver

Wang L, Bao Z, Lu D. GEM-YOLO: A Lightweight and Real-Time RGBT Object Detector with Gated Multimodal Fusion. Sensors. 2026;26(7):2035. doi:10.3390/s26072035

BibTeX
@article{Wang2026,
  author = {Wang, Lijuan and Bao, Zuchao and Lu, Dongming},
  title = {GEM-YOLO: A Lightweight and Real-Time RGBT Object Detector with Gated Multimodal Fusion},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {7},
  pages = {2035},
  doi = {10.3390/s26072035}
}
Kostenlos starten →