URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement
IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16175-16193
DOI: 10.1109/tcsvt.2026.3718871 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Qipu Qin und 1 weiteren Autor:innen. Die Arbeit nennt 54 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
It has been demonstrated that neural networks (NN)-based methods can generate high-quality reference frames to improve inter prediction. In this paper, we propose a random access (RA) and low-delay B (LB) unified reference frame synthesis network for VVC inter prediction enhancement, named URFS. Unlike the previous work with different network architectures in RA and LB configurations, URFS adopts a unified network architecture for bi-/uni-directional prediction to achieve reference frame generation, i.e. URFS in the LB configuration has the same network architecture as URFS under the RA configu-ration. Specifically, URFS extracts two reconstructed frames from the decoded picture buffer (DPB) and estimates a pair of bidirectional or unidirectional optical flows by exploiting intra- and inter-frame spatiotemporal correlations. Based on the flows, URFS synthesizes a new reference frame by performing back-warping alignment and feature enhancement. The generated frame is inserted into the reference picture list (RPL) to enhance inter coding efficiency. First, an image-level pyramid at three resolutions is constructed via bicubic downsampling, and the resulting multi-scale images are processed by corresponding hierarchical feature extractors to generate three feature-level pyramids across scales. Second, a progressive-recursive flow estimator utilizes these feature-level pyramids to iteratively refine optical flows from coarse to fine, capturing precise motion information. Third, a feature enhancer and a frame synthesizer, both built upon a U-net backbone equipped with lightweight depthwise separable convolution, enhance warped features and generate high-quality reference frames. To effectively train URFS, we introduce an incremental learning strategy based on quantization parameter (QP)-distance that takes the compressed data at higher QP as input and the compressed data at lower QP as label for training. The training strategy effectively addresses the QP imbalance between the compressed input and its label, while enhancing the learning capability of URFS. Experimental results show that URFS achieves average Bjøntegaard Delta rate (BD-rate) reductions of {RA: 5.36% / 6.19%, 12.43% / 13.73%, 11.38% / 13.13%} and {LB: 6.02% / 6.87%, 14.55% / 15.41%, 10.37% / 17.04%} in {Y, U, V} components over VVC in terms of PSNR / MS-SSIM metrics, respectively. The code is available at https://vcgit.hhi.fraunhofer.de/jvet-ai-ee1/VVCSoftware_VTM/-/tree/EE1-3.1
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Qin, Q., & Jung, C. (2026). URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16175-16193. https://doi.org/10.1109/tcsvt.2026.3718871
Vancouver
Qin Q, Jung C. URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16175-16193. doi:10.1109/tcsvt.2026.3718871
BibTeX
@article{Qin2026,
author = {Qin, Qipu and Jung, Cheolkon},
title = {URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {11},
pages = {16175--16193},
doi = {10.1109/tcsvt.2026.3718871}
}
EUVIMED