EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

EventEgo3D++: 3D Human Motion Capture from A Head-Mounted Event Camera

Christen Millerdurai, Hiroyasu Akada, Jian Wang, Diogo Luvizon, Alain Pagani, Didier Stricker, Christian Theobalt, Vladislav Golyanik iD

International Journal of Computer Vision · 2025-09 · 133 (9) · 6432-6455

DOI: 10.1007/s11263-025-02489-1 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 11.06.2025. Verfasst von Christen Millerdurai und 7 weiteren Autor:innen. Beteiligte Einrichtung: Visual Computing and Artificial Intelligence, Max Planck Institute for Informatics, SIC, Stuhlsatzenhausweg E1 4, Saarbrücken, 66123 Saarland Germany. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 90 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Monocular egocentric 3D human motion capture remains a significant challenge, particularly under conditions of low lighting and fast movements, which are common in head-mounted device applications. Existing methods that rely on RGB cameras often fail under these conditions. To address these limitations, we introduce EventEgo3D++, the first approach that leverages a monocular event camera with a fisheye lens for 3D human motion capture. Event cameras excel in high-speed scenarios and varying illumination due to their high temporal resolution, providing reliable cues for accurate 3D human motion capture. EventEgo3D++ leverages the LNES representation of event streams to enable precise 3D reconstructions. We have also developed a mobile head-mounted device (HMD) prototype equipped with an event camera, capturing a comprehensive dataset that includes real event observations from both controlled studio environments and in-the-wild settings, in addition to a synthetic dataset. Additionally, to provide a more holistic dataset, we include allocentric RGB streams that offer different perspectives of the HMD wearer, along with their corresponding SMPL body model. Our experiments demonstrate that EventEgo3D++ achieves superior 3D accuracy and robustness compared to existing solutions, even in challenging conditions. Moreover, our method supports real-time 3D pose updates at a rate of 140Hz. This work is an extension of the EventEgo3D approach (CVPR 2024) and further advances the state of the art in egocentric 3D human motion capture. For more details, visit the project page at https://eventego3d.mpi-inf.mpg.de .

Schlagwörter

Stichwörter der Autor:innen

3D Human Pose EstimationEgocentric VisionEvent-based visionVR/AR

Autor:innen und Institutionen

  • Visual Computing and Artificial Intelligence, Max Planck Institute for Informatics, SIC, Stuhlsatzenhausweg E1 4, Saarbrücken, 66123 Saarland Germany.
  • Augmented Vision, German Research Center for Artificial Intelligence (DFKI), Trippstadter Str. 122, Kaiserslautern, 67663 Rhineland-Palatinate Germany.

ORCID-Profile: Vladislav Golyanik

Förderung

  • European Research Council

Zitieren

APA

Millerdurai, C., Akada, H., Wang, J., Luvizon, D., Pagani, A., Stricker, D., Theobalt, C., & Golyanik, V. (2025). EventEgo3D++: 3D Human Motion Capture from A Head-Mounted Event Camera. International Journal of Computer Vision, 133(9), 6432-6455. https://doi.org/10.1007/s11263-025-02489-1

Vancouver

Millerdurai C, Akada H, Wang J, Luvizon D, Pagani A, Stricker D, et al. EventEgo3D++: 3D Human Motion Capture from A Head-Mounted Event Camera. International Journal of Computer Vision. 2025;133(9):6432-6455. doi:10.1007/s11263-025-02489-1

BibTeX
@article{Millerdurai2025,
  author = {Millerdurai, Christen and Akada, Hiroyasu and Wang, Jian and Luvizon, Diogo and Pagani, Alain and Stricker, Didier and Theobalt, Christian and Golyanik, Vladislav},
  title = {EventEgo3D++: 3D Human Motion Capture from A Head-Mounted Event Camera},
  journal = {International Journal of Computer Vision},
  year = {2025},
  volume = {133},
  number = {9},
  pages = {6432--6455},
  doi = {10.1007/s11263-025-02489-1}
}
Kostenlos starten →