Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling
Electronics · 05.03.2025 · 14 (5) · 1029
DOI: 10.3390/electronics14051029 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 05.03.2025. Verfasst von Yuting Cai und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the Natural Science Foundation of China, the Shaanxi Provincial Key Industry Innovation Chain Program, Natural Science Basis Research Plan in Shaanxi Province of China.
Abstract (Originalsprache)
Hand–object occlusion poses a significant challenge in 3D pose estimation. During hand–object interactions, parts of the hand or object are frequently occluded by the other, making it difficult to extract discriminative features for accurate pose estimation. Traditional methods typically extract features for both the hand and object from a single image using a shared backbone network. However, this approach often results in feature contamination, where hand and object features are mixed, especially in occluded regions. To address these issues, we propose a novel 3D hand–object pose estimation framework that explicitly tackles the problem of occlusion through two key innovations. While existing methods rely on a single backbone for feature extraction, our framework introduces a feature decoupling strategy that shares low-level features (using ResNet-50) to capture interaction contexts, while separating high-level features into two independent branches. This design ensures that hand-specific features and object-specific features are processed separately, reducing feature contamination and improving pose estimation accuracy under occlusion. Recognizing the correlation between the hand’s occluded regions and the object’s geometry, we introduce the Hand–Object Cross-Attention Transformer (HOCAT) module. Unlike traditional attention mechanisms that focus solely on feature correlations, the HOCAT leverages the geometric stability of the object as prior knowledge to guide the reconstruction of occluded hand regions. Specifically, the object features (key/value) provide contextual information to enhance the hand features (query), enabling the model to infer the positions of occluded hand joints based on the object’s known structure. This approach significantly improves the model’s ability to handle complex occlusion scenarios. The experimental results demonstrate that our method achieves significant improvements in hand–object pose estimation tasks on publicly available datasets such as HO3D V2 and Dex-YCB. On the HO3D V2 dataset, the PAMPJPE reaches 9.1 mm, the PAMPVPE is 9.0 mm, and the F-score reaches 95.8%.
Förderung
- the Natural Science Foundation of China
- the Shaanxi Provincial Key Industry Innovation Chain Program
- Natural Science Basis Research Plan in Shaanxi Province of China
- Shaanxi Provincial Innovation Capacity Support Programme Project
- Xi’an Major Scientific and Technological Achievements Transformation Industrialization Project
Zitieren
APA
Cai, Y., Pan, H., Yang, J., Liu, Y., Gao, Q., & Wang, X. (2025). Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling. Electronics, 14(5), 1029. https://doi.org/10.3390/electronics14051029
Vancouver
Cai Y, Pan H, Yang J, Liu Y, Gao Q, Wang X. Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling. Electronics. 2025;14(5):1029. doi:10.3390/electronics14051029
BibTeX
@article{Cai2025,
author = {Cai, Yuting and Pan, Huimin and Yang, Jiayi and Liu, Yichen and Gao, Quanli and Wang, Xihan},
title = {Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling},
journal = {Electronics},
year = {2025},
volume = {14},
number = {5},
pages = {1029},
doi = {10.3390/electronics14051029}
}
EUVIMED