Implicit Representation-based Volumetric Video Streaming for Photorealistic Full-scene Experience
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.03.2026 · 22 (3) · 1-21
DOI: 10.1145/3728472 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 27.02.2026. Verfasst von Jianxin Shi und 6 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
The widespread integration of the Internet of Things with sensors like depth-of-field cameras, LiDAR scanners, and eye-tracking infrared sensors, in head-mounted devices, has ushered in a new era of immersive digital experiences. Full-scene volumetric video (VV), a key innovation in this integration, provides a deeply immersive experience by capturing the richness and detail of the 3D world. However, its massive data volume presents significant streaming challenges. While 3D tile-based viewport approaches have been proposed, they struggle to full-scene VV given the small video buffer limitation, high tile segmentation overhead, and lack of full-scene consideration. In this work, inspired by the advancements of implicit neural radiance field (NeRF), we present \({\mathsf{V}^{2}\mathsf{NeRF}}\) , a novel full-scene VV streaming system featured by layered representation. It harmonizes the NeRF with explicit point clouds to represent the static background and dynamic foreground, thereby avoiding large data transfers and achieving photorealistic content representation. To tackle the issues of intensive computation requirements and multiscale adaptation scheduling within \({\mathsf{V}^{2}\mathsf{NeRF}}\) system, we propose a lightweight non-visible background removal method and a two-stage decoupled architecture. In addition, an efficient buffer-aware simulated annealing algorithm is developed, alongside the utilization of a perceptually learned metric, to enhance user experience. We further discuss the concerns about practical development and deployment. Extensive prototype evaluations demonstrate \({\mathsf{V}^{2}\mathsf{NeRF}}\) ’s superior streaming and viewing performance on a wide variety of networks, viewing motions, and scenes. For instance, compared to state-of-the-art approaches, it achieves a 24% increment in perceptual quality, an 83% reduction in rebuffering time, and a 54% enhancement in user experience on average.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Shi, J., Zhang, M., Shen, L., Liu, J., Zhang, Y., Pu, L., & Xu, J. (2026). Implicit Representation-based Volumetric Video Streaming for Photorealistic Full-scene Experience. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(3), 1-21. https://doi.org/10.1145/3728472
Vancouver
Shi J, Zhang M, Shen L, Liu J, Zhang Y, Pu L, et al. Implicit Representation-based Volumetric Video Streaming for Photorealistic Full-scene Experience. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(3):1-21. doi:10.1145/3728472
BibTeX
@article{Shi2026,
author = {Shi, Jianxin and Zhang, Miao and Shen, Linfeng and Liu, Jiangchuan and Zhang, Yuan and Pu, Lingjun and Xu, Jingdong},
title = {Implicit Representation-based Volumetric Video Streaming for Photorealistic Full-scene Experience},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {3},
pages = {1--21},
doi = {10.1145/3728472}
}
EUVIMED