Landmark and Pose Prediction in Occluded Facial Point Cloud via Explicit Joint Feature Fusion Network
IEEE Transactions on Circuits and Systems for Video Technology · 2025-10 · 35 (10) · 10148-10161
DOI: 10.1109/tcsvt.2025.3563402 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-10. Verfasst von Yifei Yang und 9 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: National Science Foundation Program of China, National Key Research and Development Program of China, China Postdoctoral Science Foundation.
Abstract (Originalsprache)
Facial point clouds collected in practical applications often suffer from pose variations and occlusion. Existing studies typically focus on either pose estimation or landmarks localization, neglecting to fully utilize the effective information from various facial features, thus limiting the improvement of prediction accuracy. Therefore, we propose an innovative 3D facial multi-task prediction network. The proposed network embeds the output of related tasks into feature extraction from the point level to the global level based on the physical dependencies between tasks. This facilitates explicit multi-task knowledge transfer, enabling the simultaneous prediction of facial landmarks, occlusion, and head pose. We introduce a training strategy based on posterior knowledge correction to iteratively refine and improve multi-task prediction results. Moreover, no single dataset provides annotations for all these tasks at once, so we synthesized a 3D landmarks, occlusion and pose (3D-LOP) dataset, which includes annotations for landmarks coordinates, occlusion probability, and head pose. The proposed method was compared with state-of-the-art methods on two public datasets and 3D-LOP. The landmarks localization accuracy improved by 7.1% on the two public datasets, and the pose estimation accuracy and stability on 3D-LOP improved by 28.5% and 32.7%, respectively. The performance on wild data also shows its potential in practical applications.
Förderung
- National Science Foundation Program of China
- National Key Research and Development Program of China
- China Postdoctoral Science Foundation
Zitieren
APA
Yang, Y., Fan, J., Shao, L., Lei, M., Fu, T., Ai, D., Xiao, D., Song, H., Lin, Y., & Yang, J. (2025). Landmark and Pose Prediction in Occluded Facial Point Cloud via Explicit Joint Feature Fusion Network. IEEE Transactions on Circuits and Systems for Video Technology, 35(10), 10148-10161. https://doi.org/10.1109/tcsvt.2025.3563402
Vancouver
Yang Y, Fan J, Shao L, Lei M, Fu T, Ai D, et al. Landmark and Pose Prediction in Occluded Facial Point Cloud via Explicit Joint Feature Fusion Network. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(10):10148-10161. doi:10.1109/tcsvt.2025.3563402
BibTeX
@article{Yang2025,
author = {Yang, Yifei and Fan, Jingfan and Shao, Long and Lei, Mingyang and Fu, Tianyu and Ai, Danni and Xiao, Deqiang and Song, Hong and Lin, Yucong and Yang, Jian},
title = {Landmark and Pose Prediction in Occluded Facial Point Cloud via Explicit Joint Feature Fusion Network},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {10},
pages = {10148--10161},
doi = {10.1109/tcsvt.2025.3563402}
}
EUVIMED