EUVIMEDEUVIMED entdecken
Crossref

EvHandPose: Event-Based 3D Hand Pose Estimation With Sparse Supervision

Jianping Jiang, Jiahe Li, Baowen Zhang, Xiaoming Deng, Boxin Shi

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-09 · 46 (9) · 6416-6430

DOI: 10.1109/tpami.2024.3380648 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-09. Verfasst von Jianping Jiang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: National Science and Technology Major, Beijing Natural Science Foundation, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Event camera shows great potential in 3D hand pose estimation, especially addressing the challenges of fast motion and high dynamic range in a low-power way. However, due to the asynchronous differential imaging mechanism, it is challenging to design event representation to encode hand motion information especially when the hands are not moving (causing motion ambiguity), and it is infeasible to fully annotate the temporally dense event stream. In this paper, we propose EvHandPose with novel hand flow representations in Event-to-Pose module for accurate hand pose estimation and alleviating the motion ambiguity issue. To solve the problem under sparse annotation, we design contrast maximization and hand-edge constraints in Pose-to-IWE (Image with Warped Events) module and formulate EvHandPose in a weakly-supervision framework. We further build EvRealHands, the first large-scale real-world event-based hand pose dataset on several challenging scenes to bridge the real-synthetic domain gap. Experiments on EvRealHands demonstrate that EvHandPose outperforms previous event-based methods under all evaluation scenes, achieves accurate and stable hand pose estimation with high temporal resolution in fast motion and strong light scenes compared with RGB-based methods, generalizes well to outdoor scenes and another type of event camera, and shows the potential for the hand gesture recognition task.

Förderung

  • National Science and Technology Major
  • Beijing Natural Science Foundation
  • National Natural Science Foundation of China
  • Beijing Hospitals Authority Clinical Medicine Development of Special Funding

Zitieren

APA

Jiang, J., Li, J., Zhang, B., Deng, X., & Shi, B. (2024). EvHandPose: Event-Based 3D Hand Pose Estimation With Sparse Supervision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9), 6416-6430. https://doi.org/10.1109/tpami.2024.3380648

Vancouver

Jiang J, Li J, Zhang B, Deng X, Shi B. EvHandPose: Event-Based 3D Hand Pose Estimation With Sparse Supervision. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(9):6416-6430. doi:10.1109/tpami.2024.3380648

BibTeX
@article{Jiang2024,
  author = {Jiang, Jianping and Li, Jiahe and Zhang, Baowen and Deng, Xiaoming and Shi, Boxin},
  title = {EvHandPose: Event-Based 3D Hand Pose Estimation With Sparse Supervision},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {9},
  pages = {6416--6430},
  doi = {10.1109/tpami.2024.3380648}
}
Kostenlos starten →