EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS

Xiaoting Yin, Hao Shi, Kailun Yang iD, Jiajun Zhai, Shangwei Guo, Kaiwei Wang

Sensors · 2025

DOI: 10.3390/s26010188 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Xiaoting Yin und 5 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Extreme Photonics and Instrumentation, National Engineering Research Center of Optical Instrumentation, Zhejiang University, …. Hauptthemen laut MEDLINE: Imaging, Three-Dimensional. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 77 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Zhejiang Province, National Natural Science Foundation of China, Hunan Provincial Research and Development Project.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Reconstructing animatable humans, together with their surrounding static environments, from monocular, motion-blurred videos is still challenging for current neural rendering methods. Existing monocular human reconstruction approaches achieve impressive quality and efficiency, but they are designed for clean intensity inputs and mainly focus on the foreground human, leading to degraded performance under motion blur and incomplete scene modeling. Event cameras provide high temporal resolution and robustness to motion blur, making them a natural complement to standard video sensors. We present E-Sem3DGS, a semantically augmented 3D Gaussian Splatting framework that leverages hybrid event-intensity streams to jointly reconstruct explicit 3D volumetric representations of human avatars and static scenes. E-Sem3DGS maintains a single set of 3D Gaussians in Euclidean space, each endowed with a learnable semantic attribute that softly separates dynamic human and static scene content. We initialize human Gaussians from Skinned Multi-Person Linear (SMPL) model priors with semantic values set to 1 and scene Gaussians by sampling a surrounding cube with semantic values set to 0, then jointly optimize geometry, appearance, and semantics. To mitigate motion blur, we derive optical flow from events and use it to supervise image-based optical flow between rendered frames, enforcing temporal coherence in high-motion regions and sharpening both humans and backgrounds. On the motion-blurred ZJU-MoCap-Blur dataset, E-Sem3DGS improves the average full-frame PSNR from 21.75 to 32.56 (+49.7%) over previous methods. On MMHPSD-Blur, our method improves PSNR from 25.23 to 28.63 (+13.48%).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Imaging, Three-DimensionalHumansAlgorithmsSemanticsAvatarNormal Distribution

Stichwörter der Autor:innen

3D Gaussian Splatting3D reconstructionevent-based vision

Autor:innen und Institutionen

  • State Key Laboratory of Extreme Photonics and Instrumentation, National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou 310027, China.
  • School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore.
  • School of Artificial Intelligence and Robotics, National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China.

ORCID-Profile: Kailun Yang

Förderung

  • Natural Science Foundation of Zhejiang Province
  • National Natural Science Foundation of China
  • Hunan Provincial Research and Development Project
  • Open Research Project of the State Key Laboratory of Industrial Control Technology, China
  • State Key Laboratory of Autonomous Intelligent Unmanned Systems

Zitieren

APA

Yin, X., Shi, H., Yang, K., Zhai, J., Guo, S., & Wang, K. (2025). E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS. Sensors. https://doi.org/10.3390/s26010188

Vancouver

Yin X, Shi H, Yang K, Zhai J, Guo S, Wang K. E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS. Sensors. 2025. doi:10.3390/s26010188

BibTeX
@article{Yin2025,
  author = {Yin, Xiaoting and Shi, Hao and Yang, Kailun and Zhai, Jiajun and Guo, Shangwei and Wang, Kaiwei},
  title = {E-Sem3DGS: Monocular Human and Scene Reconstruction via Event-Aided Semantic 3DGS},
  journal = {Sensors},
  year = {2025},
  doi = {10.3390/s26010188}
}
Kostenlos starten →