EUVIMEDEUVIMED entdecken
Crossref

CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding

Guibiao Liao, Jiankun Li, Zhenyu Bao, Xiaoqing Ye, Qing Li, Kanglin Liu

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2025 · 21 (8) · 1-24

DOI: 10.1145/3746284 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 13.08.2025. Verfasst von Guibiao Liao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Major Key Project of PCL.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Exploiting 3D Gaussian Splatting (3DGS) with Contrastive Language-Image Pre-Training (CLIP) models for open-vocabulary 3D semantic understanding of indoor scenes has emerged as an attractive research focus. Existing methods typically attach high-dimensional CLIP semantic embeddings to 3D Gaussians and leverage view-inconsistent 2D CLIP semantics as Gaussian supervision, resulting in efficiency bottlenecks and deficient 3D semantic consistency. To address these challenges, we present CLIP-GS, efficiently achieving a coherent semantic understanding of 3D indoor scenes via the proposed Semantic Attribute Compactness (SAC) and 3D Coherent Regularization (3DCR). SAC approach exploits the naturally unified semantics within objects to learn compact, yet effective, semantic Gaussian representations, enabling highly efficient rendering (>100 FPS). 3DCR enforces semantic consistency in 2D and 3D domains: In 2D, 3DCR utilizes refined view-consistent semantic outcomes derived from 3DGS to establish cross-view coherence constraints; in 3D, 3DCR encourages features similar among 3D Gaussian primitives associated with the same object, leading to more precise and coherent segmentation results. Extensive experimental results demonstrate that our method remarkably suppresses existing state-of-the-art approaches, achieving mIoU improvements of 21.20% and 13.05% on ScanNet and Replica datasets, respectively, while maintaining real-time rendering speed. Furthermore, our approach exhibits superior performance even with sparse input data, substantiating its robustness.

Förderung

  • National Natural Science Foundation of China
  • Major Key Project of PCL

Zitieren

APA

Liao, G., Li, J., Bao, Z., Ye, X., Li, Q., & Liu, K. (2025). CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(8), 1-24. https://doi.org/10.1145/3746284

Vancouver

Liao G, Li J, Bao Z, Ye X, Li Q, Liu K. CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(8):1-24. doi:10.1145/3746284

BibTeX
@article{Liao2025,
  author = {Liao, Guibiao and Li, Jiankun and Bao, Zhenyu and Ye, Xiaoqing and Li, Qing and Liu, Kanglin},
  title = {CLIP-GS: CLIP-Informed Gaussian Splatting for View-Consistent 3D Indoor Semantic Understanding},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2025},
  volume = {21},
  number = {8},
  pages = {1--24},
  doi = {10.1145/3746284}
}
Kostenlos starten →