EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Yi-Tun Lin, Graham D Finlayson iD

Sensors · 21.04.2023 · 23 (8) · 4155

DOI: 10.3390/s23084155 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 21.04.2023. Verfasst von Yi-Tun Lin und 1 weiteren Autor:innen. Beteiligte Einrichtung: School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Engineering and Physical Sciences Research Council, EPSRC, Apple Inc..

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.

Schlagwörter

Stichwörter der Autor:innen

hyperspectral imaginginverse problemmultispectral imagingspectral reconstruction

Autor:innen und Institutionen

  • School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK.

ORCID-Profile: Graham D Finlayson

Förderung

  • Engineering and Physical Sciences Research Council
  • EPSRC
  • Apple Inc.

Zitieren

APA

Lin, Y. T., & Finlayson, G. D. (2023). A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images. Sensors, 23(8), 4155. https://doi.org/10.3390/s23084155

Vancouver

Lin YT, Finlayson GD. A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images. Sensors. 2023;23(8):4155. doi:10.3390/s23084155

BibTeX
@article{Lin2023,
  author = {Lin, Yi-Tun and Finlayson, Graham D},
  title = {A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {8},
  pages = {4155},
  doi = {10.3390/s23084155}
}
Kostenlos starten →