EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling

Yuting Cai, Huimin Pan, Jiayi Yang, Yichen Liu, Quanli Gao, Xihan Wang

Electronics · 05.03.2025 · 14 (5) · 1029

DOI: 10.3390/electronics14051029 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 05.03.2025. Verfasst von Yuting Cai und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the Natural Science Foundation of China, the Shaanxi Provincial Key Industry Innovation Chain Program, Natural Science Basis Research Plan in Shaanxi Province of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hand–object occlusion poses a significant challenge in 3D pose estimation. During hand–object interactions, parts of the hand or object are frequently occluded by the other, making it difficult to extract discriminative features for accurate pose estimation. Traditional methods typically extract features for both the hand and object from a single image using a shared backbone network. However, this approach often results in feature contamination, where hand and object features are mixed, especially in occluded regions. To address these issues, we propose a novel 3D hand–object pose estimation framework that explicitly tackles the problem of occlusion through two key innovations. While existing methods rely on a single backbone for feature extraction, our framework introduces a feature decoupling strategy that shares low-level features (using ResNet-50) to capture interaction contexts, while separating high-level features into two independent branches. This design ensures that hand-specific features and object-specific features are processed separately, reducing feature contamination and improving pose estimation accuracy under occlusion. Recognizing the correlation between the hand’s occluded regions and the object’s geometry, we introduce the Hand–Object Cross-Attention Transformer (HOCAT) module. Unlike traditional attention mechanisms that focus solely on feature correlations, the HOCAT leverages the geometric stability of the object as prior knowledge to guide the reconstruction of occluded hand regions. Specifically, the object features (key/value) provide contextual information to enhance the hand features (query), enabling the model to infer the positions of occluded hand joints based on the object’s known structure. This approach significantly improves the model’s ability to handle complex occlusion scenarios. The experimental results demonstrate that our method achieves significant improvements in hand–object pose estimation tasks on publicly available datasets such as HO3D V2 and Dex-YCB. On the HO3D V2 dataset, the PAMPJPE reaches 9.1 mm, the PAMPVPE is 9.0 mm, and the F-score reaches 95.8%.

Förderung

  • the Natural Science Foundation of China
  • the Shaanxi Provincial Key Industry Innovation Chain Program
  • Natural Science Basis Research Plan in Shaanxi Province of China
  • Shaanxi Provincial Innovation Capacity Support Programme Project
  • Xi’an Major Scientific and Technological Achievements Transformation Industrialization Project

Zitieren

APA

Cai, Y., Pan, H., Yang, J., Liu, Y., Gao, Q., & Wang, X. (2025). Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling. Electronics, 14(5), 1029. https://doi.org/10.3390/electronics14051029

Vancouver

Cai Y, Pan H, Yang J, Liu Y, Gao Q, Wang X. Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling. Electronics. 2025;14(5):1029. doi:10.3390/electronics14051029

BibTeX
@article{Cai2025,
  author = {Cai, Yuting and Pan, Huimin and Yang, Jiayi and Liu, Yichen and Gao, Quanli and Wang, Xihan},
  title = {Geometry-Aware 3D Hand–Object Pose Estimation Under Occlusion via Hierarchical Feature Decoupling},
  journal = {Electronics},
  year = {2025},
  volume = {14},
  number = {5},
  pages = {1029},
  doi = {10.3390/electronics14051029}
}
Kostenlos starten →