The Potential of Diffusion-Based Near-Infrared Image Colorization
Sensors · 28.02.2024 · 24 (5) · 1565
DOI: 10.3390/s24051565 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 28.02.2024. Verfasst von Ayk Borstelmann und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Computer Science IV, University of Bonn, Friedrich-Hirzebruch-Allee 8, 53115 Bonn, Germany. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Federal Ministry of Education and Research.
Abstract (Originalsprache)
Camera traps, an invaluable tool for biodiversity monitoring, capture wildlife activities day and night. In low-light conditions, near-infrared (NIR) imaging is commonly employed to capture images without disturbing animals. However, the reflection properties of NIR light differ from those of visible light in terms of chrominance and luminance, creating a notable gap in human perception. Thus, the objective is to enrich near-infrared images with colors, thereby bridging this domain gap. Conventional colorization techniques are ineffective due to the difference between NIR and visible light. Moreover, regular supervised learning methods cannot be applied because paired training data are rare. Solutions to such unpaired image-to-image translation problems currently commonly involve generative adversarial networks (GANs), but recently, diffusion models gained attention for their superior performance in various tasks. In response to this, we present a novel framework utilizing diffusion models for the colorization of NIR images. This framework allows efficient implementation of various methods for colorizing NIR images. We show NIR colorization is primarily controlled by the translation of the near-infrared intensities to those of visible light. The experimental evaluation of three implementations with increasing complexity shows that even a simple implementation inspired by visible-near-infrared (VIS-NIR) fusion rivals GANs. Moreover, we show that the third implementation is capable of outperforming GANs. With our study, we introduce an intersection field joining the research areas of diffusion models, NIR colorization, and VIS-NIR fusion.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute of Computer Science IV, University of Bonn, Friedrich-Hirzebruch-Allee 8, 53115 Bonn, Germany.
- Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St., Cambridge, MA 02139, USA.
ORCID-Profile: Ayk Borstelmann, Timm Haucke, Volker Steinhage
Förderung
- Federal Ministry of Education and Research
Zitieren
APA
Borstelmann, A., Haucke, T., & Steinhage, V. (2024). The Potential of Diffusion-Based Near-Infrared Image Colorization. Sensors, 24(5), 1565. https://doi.org/10.3390/s24051565
Vancouver
Borstelmann A, Haucke T, Steinhage V. The Potential of Diffusion-Based Near-Infrared Image Colorization. Sensors. 2024;24(5):1565. doi:10.3390/s24051565
BibTeX
@article{Borstelmann2024,
author = {Borstelmann, Ayk and Haucke, Timm and Steinhage, Volker},
title = {The Potential of Diffusion-Based Near-Infrared Image Colorization},
journal = {Sensors},
year = {2024},
volume = {24},
number = {5},
pages = {1565},
doi = {10.3390/s24051565}
}
EUVIMED