A Lightweight 3DMM-CNN Pipeline for Real-Time Single-Image 3D Face Reconstruction: Prototyping Personalised Avatars for Extended Reality Applications
Informatics · 24.07.2026 · 13 (8) · 122
DOI: 10.3390/informatics13080122 ↗Kurz gefasst
Fachartikel in der Zeitschrift Informatics, online erschienen am 24.07.2026. Verfasst von Qianqian He und 4 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Personalised three-dimensional (3D) facial avatars underpin a wide range of immersive virtual, augmented, and mixed reality (XR) experiences, yet conventional 3D capture pipelines remain prohibitively expensive and computationally demanding for prototype-stage XR applications. This study presents and evaluates a lightweight hybrid 3D Morphable Model–Convolutional Neural Network (3DMM-CNN) pipeline that reconstructs an animation-ready 3D facial mesh from a single unconstrained RGB photograph and exposes it through an interactive prototype with native export to XR-ready asset formats. A four-channel ResNet-50 backbone fuses RGB pixels with a landmark-mask channel, regresses the 3DMM shape, expression, pose, and illumination parameters, and is refined through a multi-task loss that combines 3D parameter regression, 2D landmark consistency, and image-to-mesh-to-image cycle consistency. The model is trained on a curated 2000-image subset of the LFW-People corpus and evaluated under four yaw-angle strata. The results indicate that on a held-out 400-image test set, the pipeline attains R2 = 0.854, MSE = 0.022, Pearson r = 0.92, and MAPE = 10.6%, with a single-frame inference latency of 35 ms on a commodity RTX-class GPU. Robustness to head rotation improves by 29.9% at extreme poses (60–90° yaw) compared with a single-modality baseline. A Blender-integrated prototype successfully exports the reconstructed mesh as a deformation-ready asset for Unity- and Unreal-based XR engines. The proposed pipeline offers a cost-effective, real-time-capable component for XR avatar prototyping, lowering the entry barrier for small studios, immersive-learning developers, and AR/MR telepresence research. On the standard AFLW2000-3D benchmark, the pipeline additionally attains a Normalised Mean Error of 2.47% and a full-vertex reconstruction error of 1.50%, which is competitive with published lightweight baselines while retaining sub-50 ms inference latency.
Zitieren
APA
He, Q., Chansanam, W., Nguyen, L. T., Intawong, K., & Puritat, K. (2026). A Lightweight 3DMM-CNN Pipeline for Real-Time Single-Image 3D Face Reconstruction: Prototyping Personalised Avatars for Extended Reality Applications. Informatics, 13(8), 122. https://doi.org/10.3390/informatics13080122
Vancouver
He Q, Chansanam W, Nguyen LT, Intawong K, Puritat K. A Lightweight 3DMM-CNN Pipeline for Real-Time Single-Image 3D Face Reconstruction: Prototyping Personalised Avatars for Extended Reality Applications. Informatics. 2026;13(8):122. doi:10.3390/informatics13080122
BibTeX
@article{He2026,
author = {He, Qianqian and Chansanam, Wirapong and Nguyen, Lan Thi and Intawong, Kannikar and Puritat, Kitti},
title = {A Lightweight 3DMM-CNN Pipeline for Real-Time Single-Image 3D Face Reconstruction: Prototyping Personalised Avatars for Extended Reality Applications},
journal = {Informatics},
year = {2026},
volume = {13},
number = {8},
pages = {122},
doi = {10.3390/informatics13080122}
}
EUVIMED