EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection

Keval Thaker iD, Sumanth Chennupati iD, Nathir Rawashdeh iD, Samir A Rawashdeh iD

Journal of Imaging · 31.12.2023 · 10 (1) · 12

DOI: 10.3390/jimaging10010012 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging, online erschienen am 31.12.2023. Verfasst von Keval Thaker und 3 weiteren Autor:innen. Beteiligte Einrichtung: Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI 48128, USA. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite significant strides in achieving vehicle autonomy, robust perception under low-light conditions still remains a persistent challenge. In this study, we investigate the potential of multispectral imaging, thereby leveraging deep learning models to enhance object detection performance in the context of nighttime driving. Features encoded from the red, green, and blue (RGB) visual spectrum and thermal infrared images are combined to implement a multispectral object detection model. This has proven to be more effective compared to using visual channels only, as thermal images provide complementary information when discriminating objects in low-illumination conditions. Additionally, there is a lack of studies on effectively fusing these two modalities for optimal object detection performance. In this work, we present a framework based on the Faster R-CNN architecture with a feature pyramid network. Moreover, we design various fusion approaches using concatenation and addition operators at varying stages of the network to analyze their impact on object detection performance. Our experimental results on the KAIST and FLIR datasets show that our framework outperforms the baseline experiments of the unimodal input source and the existing multispectral object detectors.

Schlagwörter

Stichwörter der Autor:innen

RGB-T fusionlow-light object detectionmultispectral fusion

Autor:innen und Institutionen

  • Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI 48128, USA.
  • Department of Applied Computing, Michigan Technological University, Houghton, MI 49931, USA.

ORCID-Profile: Keval Thaker, Sumanth Chennupati, Nathir Rawashdeh, Samir A Rawashdeh

Zitieren

APA

Thaker, K., Chennupati, S., Rawashdeh, N., & Rawashdeh, S. A. (2023). Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection. Journal of Imaging, 10(1), 12. https://doi.org/10.3390/jimaging10010012

Vancouver

Thaker K, Chennupati S, Rawashdeh N, Rawashdeh SA. Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection. Journal of Imaging. 2023;10(1):12. doi:10.3390/jimaging10010012

BibTeX
@article{Thaker2023,
  author = {Thaker, Keval and Chennupati, Sumanth and Rawashdeh, Nathir and Rawashdeh, Samir A},
  title = {Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection},
  journal = {Journal of Imaging},
  year = {2023},
  volume = {10},
  number = {1},
  pages = {12},
  doi = {10.3390/jimaging10010012}
}
Kostenlos starten →