A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.06.2024 · 20 (6) · 1-20
DOI: 10.1145/3640467 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 08.03.2024. Verfasst von Yichi Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 82 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
This article proposes a reconfigurable framework for neural network based video in-loop filtering to guide large-scale models for content-aware processing. Specifically, the backbone neural model is decomposed into several convolutional groups and the encoder systematically traverses all candidate configurations combined by these groups to find the best one. The selected configuration index is then encapsulated as side information and passed to the decoder, enabling dynamic model reconfiguration during the decoding stage. The preceding reconfiguration process is only deployed in the inference stage on top of a pre-trained backbone model. Furthermore, we devise WMSPFormer , a wavelet multi-scale Poolformer, as the backbone network structure. WMSPFormer utilizes a wavelet-based multi-scale structure to losslessly decompose the input into multiple scales for spatial-spectral features aggregation. Moreover, it uses multi-scale pooling operations ( MSPoolformer ) instead of complicated matrix calculations to substitute the attention process. We also extend MSPoolformer to a large-scale version using more parameters, referred to as MSPoolformerExt . Extensive experiments demonstrate that the proposed WMSPFormer+Reconfig. and WMSPFormerExt+Reconfig. achieve a remarkable 7.13% and 7.92% BD-Rate reduction over the anchor H.266/VVC, outperforming most existing methods evaluated under the same training and testing conditions. In addition, the low-complexity nature of the WMSPFormer series makes it attractive for practical applications.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Zhang, Y., Ding, D., Ma, Z., & Li, Z. (2024). A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(6), 1-20. https://doi.org/10.1145/3640467
Vancouver
Zhang Y, Ding D, Ma Z, Li Z. A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(6):1-20. doi:10.1145/3640467
BibTeX
@article{Zhang2024,
author = {Zhang, Yichi and Ding, Dandan and Ma, Zhan and Li, Zhu},
title = {A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2024},
volume = {20},
number = {6},
pages = {1--20},
doi = {10.1145/3640467}
}
EUVIMED