Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks
Sensors · 04.01.2024 · 24 (1) · 296
DOI: 10.3390/s24010296 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 04.01.2024. Verfasst von Hamed Elwarfalli und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, University of Dayton, 300 College Park, Dayton, OH 45469, USA. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Convolutional neural networks (CNNs) have become instrumental in advancing multi-frame image super-resolution (SR), a technique that merges multiple low-resolution images of the same scene into a high-resolution image. In this paper, a novel deep learning multi-frame SR algorithm is introduced. The proposed CNN model, named Exponential Fusion of Interpolated Frames Network (EFIF-Net), seamlessly integrates fusion and restoration within an end-to-end network. Key features of the new EFIF-Net include a custom exponentially weighted fusion (EWF) layer for image fusion and a modification of the Residual Channel Attention Network for restoration to deblur the fused image. Input frames are registered with subpixel accuracy using an affine motion model to capture the camera platform motion. The frames are externally upsampled using single-image interpolation. The interpolated frames are then fused with the custom EWF layer, employing subpixel registration information to give more weight to pixels with less interpolation error. Realistic image acquisition conditions are simulated to generate training and testing datasets with corresponding ground truths. The observation model captures optical degradation from diffraction and detector integration from the sensor. The experimental results demonstrate the efficacy of EFIF-Net using both simulated and real camera data. The real camera results use authentic, unaltered camera data without artificial downsampling or degradation.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electrical and Computer Engineering, University of Dayton, 300 College Park, Dayton, OH 45469, USA.
- Applied Sensing Division, University of Dayton Research Institute, 300 College Park, Dayton, OH 45469, USA.
ORCID-Profile: Hamed Elwarfalli, Dylan Flaute, Russell C Hardie
Zitieren
APA
Elwarfalli, H., Flaute, D., & Hardie, R. C. (2024). Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks. Sensors, 24(1), 296. https://doi.org/10.3390/s24010296
Vancouver
Elwarfalli H, Flaute D, Hardie RC. Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks. Sensors. 2024;24(1):296. doi:10.3390/s24010296
BibTeX
@article{Elwarfalli2024,
author = {Elwarfalli, Hamed and Flaute, Dylan and Hardie, Russell C},
title = {Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks},
journal = {Sensors},
year = {2024},
volume = {24},
number = {1},
pages = {296},
doi = {10.3390/s24010296}
}
EUVIMED