A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images
Sensors · 21.04.2023 · 23 (8) · 4155
DOI: 10.3390/s23084155 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 21.04.2023. Verfasst von Yi-Tun Lin und 1 weiteren Autor:innen. Beteiligte Einrichtung: School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Engineering and Physical Sciences Research Council, EPSRC, Apple Inc..
Abstract (Originalsprache)
Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK.
ORCID-Profile: Graham D Finlayson
Förderung
- Engineering and Physical Sciences Research Council
- EPSRC
- Apple Inc.
Zitieren
APA
Lin, Y. T., & Finlayson, G. D. (2023). A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images. Sensors, 23(8), 4155. https://doi.org/10.3390/s23084155
Vancouver
Lin YT, Finlayson GD. A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images. Sensors. 2023;23(8):4155. doi:10.3390/s23084155
BibTeX
@article{Lin2023,
author = {Lin, Yi-Tun and Finlayson, Graham D},
title = {A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images},
journal = {Sensors},
year = {2023},
volume = {23},
number = {8},
pages = {4155},
doi = {10.3390/s23084155}
}
EUVIMED