SMR: Spatial-Guided Model-Based Regression for 3D Hand Pose and Mesh Reconstruction
IEEE Transactions on Circuits and Systems for Video Technology · 2024-01 · 34 (1) · 299-314
DOI: 10.1109/tcsvt.2023.3285153 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-01. Verfasst von Haifeng Sun und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Ministry of Education and China Mobile Joint Fund, Beijing University of Posts and Telecommunications-China Mobile Research Institute Joint Innovation Center.
Abstract (Originalsprache)
3D hand reconstruction is an important technique for human-computer interaction. Interactive experience depends on the accuracy, efficiency, and robustness of the algorithm. Therefore, in this paper, we first propose a balanced framework called spatial-aware regression (SAR) to achieve precise and fast reconstruction. SAR can bridge convolutional networks and graph-structure networks more effectively than existing frameworks to fully exploit extracted spatial information using a novel spatial-aware initial graph building module. In addition, SAR uses adaptive-GCN to make keypoints interact efficiently and effectively; and regresses 2.5D belief maps to characterize uncertainty. SAR is highly flexible because it can predict an arbitrary number of keypoints and apply pose-guided refinement for coarse to fine regression. To produce more rational results for challenging cases and mitigate 3D label reliance, we also propose a more robust model-based framework called spatial-guided model-based regression (SMR) that is based on SAR. There are two critical designs of SMR: 1) it uses SAR to enhance the features with pose information to help the regression of hand model parameters; and 2) it regresses parameters in a spatially aware manner that is similar to SAR. Experiments demonstrate that the proposed frameworks surpass existing fully-supervised approaches on the FreiHAND, HO-3D, RHD, and STB datasets. Also, the performances of the proposed frameworks under weakly/self-supervised settings outperform other competitors. Meanwhile, the proposed frameworks are accurate and efficient.
Förderung
- National Natural Science Foundation of China
- Ministry of Education and China Mobile Joint Fund
- Beijing University of Posts and Telecommunications-China Mobile Research Institute Joint Innovation Center
Zitieren
APA
Sun, H., Zheng, X., Ren, P., Wang, J., Qi, Q., & Liao, J. (2024). SMR: Spatial-Guided Model-Based Regression for 3D Hand Pose and Mesh Reconstruction. IEEE Transactions on Circuits and Systems for Video Technology, 34(1), 299-314. https://doi.org/10.1109/tcsvt.2023.3285153
Vancouver
Sun H, Zheng X, Ren P, Wang J, Qi Q, Liao J. SMR: Spatial-Guided Model-Based Regression for 3D Hand Pose and Mesh Reconstruction. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(1):299-314. doi:10.1109/tcsvt.2023.3285153
BibTeX
@article{Sun2024,
author = {Sun, Haifeng and Zheng, Xiaozheng and Ren, Pengfei and Wang, Jingyu and Qi, Qi and Liao, Jianxin},
title = {SMR: Spatial-Guided Model-Based Regression for 3D Hand Pose and Mesh Reconstruction},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {1},
pages = {299--314},
doi = {10.1109/tcsvt.2023.3285153}
}
EUVIMED