EUVIMEDEUVIMED entdecken
Crossref

VRT: A Video Restoration Transformer

Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, Luc Van Gool

IEEE Transactions on Image Processing · 2024 · 33 · 2171-2182

DOI: 10.1109/tip.2024.3372454 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2024. Verfasst von Jingyun Liang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 258-mal zitiert (Crossref) und nennt 77 Literaturangaben. Gefördert von: Eidgenössische Technische Hochschule Zürich (ETH Zurich) Fund, Huawei Technologies Oy (Finland) Project, China Scholarship Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video restoration aims to restore high-quality frames from low-quality frames. Different from single image restoration, video restoration generally requires to utilize temporal information from multiple adjacent but usually misaligned video frames. Existing deep methods generally tackle with this by exploiting a sliding window strategy or a recurrent architecture, which are restricted by frame-by-frame restoration. In this paper, we propose a Video Restoration Transformer (VRT) with parallel frame prediction ability. More specifically, VRT is composed of multiple scales, each of which consists of two kinds of modules: temporal reciprocal self attention (TRSA) and parallel warping. TRSA divides the video into small clips, on which reciprocal attention is applied for joint motion estimation, feature alignment and feature fusion, while self attention is used for feature extraction. To enable cross-clip interactions, the video sequence is shifted for every other layer. Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping. Experimental results on five tasks, including video super-resolution, video deblurring, video denoising, video frame interpolation and space-time video super-resolution, demonstrate that VRT outperforms the state-of-the-art methods by large margins (up to 2.16dB) on fourteen benchmark datasets. The codes are available at https://github.com/JingyunLiang/VRT.

Förderung

  • Eidgenössische Technische Hochschule Zürich (ETH Zurich) Fund
  • Huawei Technologies Oy (Finland) Project
  • China Scholarship Council
  • Amazon Web Services
  • Alexander von Humboldt Foundation

Zitieren

APA

Liang, J., Cao, J., Fan, Y., Zhang, K., Ranjan, R., Li, Y., Timofte, R., & Van Gool, L. (2024). VRT: A Video Restoration Transformer. IEEE Transactions on Image Processing, 33, 2171-2182. https://doi.org/10.1109/tip.2024.3372454

Vancouver

Liang J, Cao J, Fan Y, Zhang K, Ranjan R, Li Y, et al. VRT: A Video Restoration Transformer. IEEE Transactions on Image Processing. 2024;33:2171-2182. doi:10.1109/tip.2024.3372454

BibTeX
@article{Liang2024,
  author = {Liang, Jingyun and Cao, Jiezhang and Fan, Yuchen and Zhang, Kai and Ranjan, Rakesh and Li, Yawei and Timofte, Radu and Van Gool, Luc},
  title = {VRT: A Video Restoration Transformer},
  journal = {IEEE Transactions on Image Processing},
  year = {2024},
  volume = {33},
  pages = {2171--2182},
  doi = {10.1109/tip.2024.3372454}
}
Kostenlos starten →