Reference-based In-loop Filter with Robust Neural Feature Transfer for Video Coding
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.01.2025 · 21 (1) · 1-24
DOI: 10.1145/3702643 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 23.12.2024. Verfasst von Nayoung Kim und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 66 Literaturangaben. Gefördert von: Institute of Information and Communications Technology Planning and Evaluation, NRF, Ministry of Culture, Sports and Tourism in 2024.
Abstract (Originalsprache)
In this article, we propose an efficient reference-based deep in-loop filtering method for video coding. Existing reference-based in-loop filters often face challenges in improving coding efficiency due to the difficulty in capturing relevant textures from the reference frames. Our method accurately predicts the texture of a reference block and uses this information to restore the current block. To achieve this, we develop a reference-to-current feature estimation module that conveys high-quality information from previously coded frames in the feature domain, thereby preventing loss of detail due to inaccurate prediction. Although a neural network is trained to restore a coded video frame to be similar to the current frame, their performance can significantly degrade when operating with various quantization parameters (QPs) and managing different levels of distortion. This problem becomes further severe in the reference-to-current feature estimation, in which QP values are applied differently to video frames. We address this problem by developing a QP-aware convolution layer with a small number of learnable parameters to generate reliable features and adapt to fine-grained adaptive QPs among consecutive frames. The proposed method is implemented into the versatile video coding (VVC) reference software, VTM version 10.0. Experimental results demonstrate that the proposed method improves coding performance significantly in VVC.
Förderung
- Institute of Information and Communications Technology Planning and Evaluation
- NRF
- Ministry of Culture, Sports and Tourism in 2024
Zitieren
APA
Kim, N., Lee, J., & Kang, J. (2025). Reference-based In-loop Filter with Robust Neural Feature Transfer for Video Coding. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(1), 1-24. https://doi.org/10.1145/3702643
Vancouver
Kim N, Lee J, Kang J. Reference-based In-loop Filter with Robust Neural Feature Transfer for Video Coding. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(1):1-24. doi:10.1145/3702643
BibTeX
@article{Kim2025,
author = {Kim, Nayoung and Lee, Jung-Kyung and Kang, Je-Won},
title = {Reference-based In-loop Filter with Robust Neural Feature Transfer for Video Coding},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2025},
volume = {21},
number = {1},
pages = {1--24},
doi = {10.1145/3702643}
}
EUVIMED