E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS
Sensors · 2025
DOI: 10.3390/s26010188 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Xiaoting Yin und 5 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Extreme Photonics and Instrumentation, National Engineering Research Center of Optical Instrumentation, Zhejiang University, …. Hauptthemen laut MEDLINE: Imaging, Three-Dimensional. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 77 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Zhejiang Province, National Natural Science Foundation of China, Hunan Provincial Research and Development Project.
Abstract (Originalsprache)
Reconstructing animatable humans, together with their surrounding static environments, from monocular, motion-blurred videos is still challenging for current neural rendering methods. Existing monocular human reconstruction approaches achieve impressive quality and efficiency, but they are designed for clean intensity inputs and mainly focus on the foreground human, leading to degraded performance under motion blur and incomplete scene modeling. Event cameras provide high temporal resolution and robustness to motion blur, making them a natural complement to standard video sensors. We present E-Sem3DGS, a semantically augmented 3D Gaussian Splatting framework that leverages hybrid event-intensity streams to jointly reconstruct explicit 3D volumetric representations of human avatars and static scenes. E-Sem3DGS maintains a single set of 3D Gaussians in Euclidean space, each endowed with a learnable semantic attribute that softly separates dynamic human and static scene content. We initialize human Gaussians from Skinned Multi-Person Linear (SMPL) model priors with semantic values set to 1 and scene Gaussians by sampling a surrounding cube with semantic values set to 0, then jointly optimize geometry, appearance, and semantics. To mitigate motion blur, we derive optical flow from events and use it to supervise image-based optical flow between rendered frames, enforcing temporal coherence in high-motion regions and sharpening both humans and backgrounds. On the motion-blurred ZJU-MoCap-Blur dataset, E-Sem3DGS improves the average full-frame PSNR from 21.75 to 32.56 (+49.7%) over previous methods. On MMHPSD-Blur, our method improves PSNR from 25.23 to 28.63 (+13.48%).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State Key Laboratory of Extreme Photonics and Instrumentation, National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou 310027, China.
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore.
- School of Artificial Intelligence and Robotics, National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China.
ORCID-Profile: Kailun Yang
Förderung
- Natural Science Foundation of Zhejiang Province
- National Natural Science Foundation of China
- Hunan Provincial Research and Development Project
- Open Research Project of the State Key Laboratory of Industrial Control Technology, China
- State Key Laboratory of Autonomous Intelligent Unmanned Systems
Zitieren
APA
Yin, X., Shi, H., Yang, K., Zhai, J., Guo, S., & Wang, K. (2025). E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS. Sensors. https://doi.org/10.3390/s26010188
Vancouver
Yin X, Shi H, Yang K, Zhai J, Guo S, Wang K. E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS. Sensors. 2025. doi:10.3390/s26010188
BibTeX
@article{Yin2025,
author = {Yin, Xiaoting and Shi, Hao and Yang, Kailun and Zhai, Jiajun and Guo, Shangwei and Wang, Kaiwei},
title = {E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS},
journal = {Sensors},
year = {2025},
doi = {10.3390/s26010188}
}
EUVIMED