ISCom: Interest-Aware Semantic Communication Scheme for Point Cloud Video Streaming on Metaverse XR Devices
IEEE Journal on Selected Areas in Communications · 2024-04 · 42 (4) · 1003-1021
DOI: 10.1109/jsac.2023.3345430 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Journal on Selected Areas in Communications, 2024-04. Verfasst von Yakun Huang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 54 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, China Post–Doctoral Science Foundation.
Abstract (Originalsprache)
In the metaverse era, point cloud video (PCV) streaming on mobile XR devices is pivotal. While most current methods focus on PCV compression from traditional 3-DoF video services, emerging AI techniques extract vital semantic information, producing content resembling the original. However, these are early-stage and computationally intensive. To enhance the inference efficacy of AI-based approaches, accommodate dynamic environments, and facilitate applicability to metaverse XR devices, we present ISCom, an interest-aware semantic communication scheme for lightweight PCV streaming. ISCom is featured with a region-of-interest (ROI) selection module, a lightweight encoder-decoder training module, and a learning-based scheduler to achieve real-time PCV decoding and rendering on resource-constrained devices. ISCom’s dual-stage ROI selection provides significantly reduces data volume according to real-time interest. The lightweight PCV encoder-decoder training is tailored to resource-constrained devices and adapts to the heterogeneous computing capabilities of devices. Furthermore, We provide a deep reinforcement learning (DRL)-based scheduler to select optimal encoder-decoder model for various devices adaptivelly, considering the dynamic network environments and device computing capabilities. Our extensive experiments demonstrate that ISCom outperforms baselines on mobile devices, achieving a minimum rendering frame rate improvement of 10 FPS and up to 22 FPS. Furthermore, our method significantly reduces memory usage by 41.7% compared to the state-of-the-art AITransfer method. These results highlight the effectiveness of ISCom in enabling lightweight PCV streaming and its potential to improve immersive experiences for emerging metaverse application.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- China Post–Doctoral Science Foundation
Zitieren
APA
Huang, Y., Bai, B., Zhu, Y., Qiao, X., Su, X., Yang, L., & Zhang, P. (2024). ISCom: Interest-Aware Semantic Communication Scheme for Point Cloud Video Streaming on Metaverse XR Devices. IEEE Journal on Selected Areas in Communications, 42(4), 1003-1021. https://doi.org/10.1109/jsac.2023.3345430
Vancouver
Huang Y, Bai B, Zhu Y, Qiao X, Su X, Yang L, et al. ISCom: Interest-Aware Semantic Communication Scheme for Point Cloud Video Streaming on Metaverse XR Devices. IEEE Journal on Selected Areas in Communications. 2024;42(4):1003-1021. doi:10.1109/jsac.2023.3345430
BibTeX
@article{Huang2024,
author = {Huang, Yakun and Bai, Boyuan and Zhu, Yuanwei and Qiao, Xiuquan and Su, Xiang and Yang, Lei and Zhang, Ping},
title = {ISCom: Interest-Aware Semantic Communication Scheme for Point Cloud Video Streaming on Metaverse XR Devices},
journal = {IEEE Journal on Selected Areas in Communications},
year = {2024},
volume = {42},
number = {4},
pages = {1003--1021},
doi = {10.1109/jsac.2023.3345430}
}
EUVIMED