Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach
Sensors · 24.05.2023 · 23 (11) · 5030
DOI: 10.3390/s23115030 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 24.05.2023. Verfasst von Wenhao Wang und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China. Hauptthemen laut MEDLINE: Acclimatization, Benchmarking. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, General Program of National Natural Science Foundation of China, Guangxi Key Research and Development Program.
Abstract (Originalsprache)
The quality of videos varies due to the different capabilities of sensors. Video super-resolution (VSR) is a technology that improves the quality of captured video. However, the development of a VSR model is very costly. In this paper, we present a novel approach for adapting single-image super-resolution (SISR) models to the VSR task. To achieve this, we first summarize a common architecture of SISR models and perform a formal analysis of adaptation. Then, we propose an adaptation method that incorporates a plug-and-play temporal feature extraction module into existing SISR models. The proposed temporal feature extraction module consists of three submodules: offset estimation, spatial aggregation, and temporal aggregation. In the spatial aggregation submodule, the features obtained from the SISR model are aligned to the center frame based on the offset estimation results. The aligned features are fused in the temporal aggregation submodule. Finally, the fused temporal feature is fed to the SISR model for reconstruction. To evaluate the effectiveness of our method, we adapt five representative SISR models and evaluate these models on two popular benchmarks. The experiment results show the proposed method is effective on different SISR models. In particular, on the Vid4 benchmark, the VSR-adapted models achieve at least 1.26 dB and 0.067 improvement over the original SISR models in terms of PSNR and SSIM metrics, respectively. Additionally, these VSR-adapted models achieve better performance than the state-of-the-art VSR models.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China.
ORCID-Profile: Wenhao Wang
Förderung
- National Natural Science Foundation of China
- General Program of National Natural Science Foundation of China
- Guangxi Key Research and Development Program
- Innovation Project of Guangxi Graduate Education
- Guangxi Science Fund for Distinguished Young Scholars
Zitieren
APA
Wang, W., Liu, Z., Lu, H., Lan, R., & Huang, Y. (2023). Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach. Sensors, 23(11), 5030. https://doi.org/10.3390/s23115030
Vancouver
Wang W, Liu Z, Lu H, Lan R, Huang Y. Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach. Sensors. 2023;23(11):5030. doi:10.3390/s23115030
BibTeX
@article{Wang2023,
author = {Wang, Wenhao and Liu, Zhenbing and Lu, Haoxiang and Lan, Rushi and Huang, Yingxin},
title = {Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach},
journal = {Sensors},
year = {2023},
volume = {23},
number = {11},
pages = {5030},
doi = {10.3390/s23115030}
}
EUVIMED