EUVIMEDEUVIMED entdecken
Crossref

URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement

Qipu Qin, Cheolkon Jung

IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16175-16193

DOI: 10.1109/tcsvt.2026.3718871 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Qipu Qin und 1 weiteren Autor:innen. Die Arbeit nennt 54 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

It has been demonstrated that neural networks (NN)-based methods can generate high-quality reference frames to improve inter prediction. In this paper, we propose a random access (RA) and low-delay B (LB) unified reference frame synthesis network for VVC inter prediction enhancement, named URFS. Unlike the previous work with different network architectures in RA and LB configurations, URFS adopts a unified network architecture for bi-/uni-directional prediction to achieve reference frame generation, i.e. URFS in the LB configuration has the same network architecture as URFS under the RA configu-ration. Specifically, URFS extracts two reconstructed frames from the decoded picture buffer (DPB) and estimates a pair of bidirectional or unidirectional optical flows by exploiting intra- and inter-frame spatiotemporal correlations. Based on the flows, URFS synthesizes a new reference frame by performing back-warping alignment and feature enhancement. The generated frame is inserted into the reference picture list (RPL) to enhance inter coding efficiency. First, an image-level pyramid at three resolutions is constructed via bicubic downsampling, and the resulting multi-scale images are processed by corresponding hierarchical feature extractors to generate three feature-level pyramids across scales. Second, a progressive-recursive flow estimator utilizes these feature-level pyramids to iteratively refine optical flows from coarse to fine, capturing precise motion information. Third, a feature enhancer and a frame synthesizer, both built upon a U-net backbone equipped with lightweight depthwise separable convolution, enhance warped features and generate high-quality reference frames. To effectively train URFS, we introduce an incremental learning strategy based on quantization parameter (QP)-distance that takes the compressed data at higher QP as input and the compressed data at lower QP as label for training. The training strategy effectively addresses the QP imbalance between the compressed input and its label, while enhancing the learning capability of URFS. Experimental results show that URFS achieves average Bjøntegaard Delta rate (BD-rate) reductions of {RA: 5.36% / 6.19%, 12.43% / 13.73%, 11.38% / 13.13%} and {LB: 6.02% / 6.87%, 14.55% / 15.41%, 10.37% / 17.04%} in {Y, U, V} components over VVC in terms of PSNR / MS-SSIM metrics, respectively. The code is available at https://vcgit.hhi.fraunhofer.de/jvet-ai-ee1/VVCSoftware_VTM/-/tree/EE1-3.1

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Qin, Q., & Jung, C. (2026). URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16175-16193. https://doi.org/10.1109/tcsvt.2026.3718871

Vancouver

Qin Q, Jung C. URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16175-16193. doi:10.1109/tcsvt.2026.3718871

BibTeX
@article{Qin2026,
  author = {Qin, Qipu and Jung, Cheolkon},
  title = {URFS: RA and LB Unified Reference Frame Synthesis for VVC Inter-Prediction Enhancement},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {11},
  pages = {16175--16193},
  doi = {10.1109/tcsvt.2026.3718871}
}
Kostenlos starten →