EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The Potential of Diffusion-Based Near-Infrared Image Colorization

Ayk Borstelmann iD, Timm Haucke iD, Volker Steinhage iD

Sensors · 28.02.2024 · 24 (5) · 1565

DOI: 10.3390/s24051565 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 28.02.2024. Verfasst von Ayk Borstelmann und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Computer Science IV, University of Bonn, Friedrich-Hirzebruch-Allee 8, 53115 Bonn, Germany. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Federal Ministry of Education and Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Camera traps, an invaluable tool for biodiversity monitoring, capture wildlife activities day and night. In low-light conditions, near-infrared (NIR) imaging is commonly employed to capture images without disturbing animals. However, the reflection properties of NIR light differ from those of visible light in terms of chrominance and luminance, creating a notable gap in human perception. Thus, the objective is to enrich near-infrared images with colors, thereby bridging this domain gap. Conventional colorization techniques are ineffective due to the difference between NIR and visible light. Moreover, regular supervised learning methods cannot be applied because paired training data are rare. Solutions to such unpaired image-to-image translation problems currently commonly involve generative adversarial networks (GANs), but recently, diffusion models gained attention for their superior performance in various tasks. In response to this, we present a novel framework utilizing diffusion models for the colorization of NIR images. This framework allows efficient implementation of various methods for colorizing NIR images. We show NIR colorization is primarily controlled by the translation of the near-infrared intensities to those of visible light. The experimental evaluation of three implementations with increasing complexity shows that even a simple implementation inspired by visible-near-infrared (VIS-NIR) fusion rivals GANs. Moreover, we show that the third implementation is capable of outperforming GANs. With our study, we introduce an intersection field joining the research areas of diffusion models, NIR colorization, and VIS-NIR fusion.

Schlagwörter

Stichwörter der Autor:innen

camera trappingdiffusion modelsmachine learningnear-infraredneural networksunpaired dataset

Autor:innen und Institutionen

  • Institute of Computer Science IV, University of Bonn, Friedrich-Hirzebruch-Allee 8, 53115 Bonn, Germany.
  • Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St., Cambridge, MA 02139, USA.

ORCID-Profile: Ayk Borstelmann, Timm Haucke, Volker Steinhage

Förderung

  • Federal Ministry of Education and Research

Zitieren

APA

Borstelmann, A., Haucke, T., & Steinhage, V. (2024). The Potential of Diffusion-Based Near-Infrared Image Colorization. Sensors, 24(5), 1565. https://doi.org/10.3390/s24051565

Vancouver

Borstelmann A, Haucke T, Steinhage V. The Potential of Diffusion-Based Near-Infrared Image Colorization. Sensors. 2024;24(5):1565. doi:10.3390/s24051565

BibTeX
@article{Borstelmann2024,
  author = {Borstelmann, Ayk and Haucke, Timm and Steinhage, Volker},
  title = {The Potential of Diffusion-Based Near-Infrared Image Colorization},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {5},
  pages = {1565},
  doi = {10.3390/s24051565}
}
Kostenlos starten →