CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2025 · 21 (8) · 1-24
DOI: 10.1145/3746284 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 13.08.2025. Verfasst von Guibiao Liao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Major Key Project of PCL.
Abstract (Originalsprache)
Exploiting 3D Gaussian Splatting (3DGS) with Contrastive Language-Image Pre-Training (CLIP) models for open-vocabulary 3D semantic understanding of indoor scenes has emerged as an attractive research focus. Existing methods typically attach high-dimensional CLIP semantic embeddings to 3D Gaussians and leverage view-inconsistent 2D CLIP semantics as Gaussian supervision, resulting in efficiency bottlenecks and deficient 3D semantic consistency. To address these challenges, we present CLIP-GS, efficiently achieving a coherent semantic understanding of 3D indoor scenes via the proposed Semantic Attribute Compactness (SAC) and 3D Coherent Regularization (3DCR). SAC approach exploits the naturally unified semantics within objects to learn compact, yet effective, semantic Gaussian representations, enabling highly efficient rendering (>100 FPS). 3DCR enforces semantic consistency in 2D and 3D domains: In 2D, 3DCR utilizes refined view-consistent semantic outcomes derived from 3DGS to establish cross-view coherence constraints; in 3D, 3DCR encourages features similar among 3D Gaussian primitives associated with the same object, leading to more precise and coherent segmentation results. Extensive experimental results demonstrate that our method remarkably suppresses existing state-of-the-art approaches, achieving mIoU improvements of 21.20% and 13.05% on ScanNet and Replica datasets, respectively, while maintaining real-time rendering speed. Furthermore, our approach exhibits superior performance even with sparse input data, substantiating its robustness.
Förderung
- National Natural Science Foundation of China
- Major Key Project of PCL
Zitieren
APA
Liao, G., Li, J., Bao, Z., Ye, X., Li, Q., & Liu, K. (2025). CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(8), 1-24. https://doi.org/10.1145/3746284
Vancouver
Liao G, Li J, Bao Z, Ye X, Li Q, Liu K. CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(8):1-24. doi:10.1145/3746284
BibTeX
@article{Liao2025,
author = {Liao, Guibiao and Li, Jiankun and Bao, Zhenyu and Ye, Xiaoqing and Li, Qing and Liu, Kanglin},
title = {CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2025},
volume = {21},
number = {8},
pages = {1--24},
doi = {10.1145/3746284}
}
EUVIMED