EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction

Hao Xu, Yilin Liu, Yinqiao Wang, Chi-Wing Fu, Niloy Mitra

ACM Transactions on Graphics · 09.11.2026 · 45 (6) · 1-12

DOI: 10.1145/3842509 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 09.11.2026. Verfasst von Hao Xu und 4 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Grants Council of the Hong Kong Special Administrative Region, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We ask whether everyday open-world monocular videos can be turned into reusable 4D interaction primitives: articulated hand motion, object shape with 6D pose over time, and the when/where of contact. Such a capability would enable scalable mining of real interactions and, beyond reconstruction, support scene-aware synthesis and planning. However, reconstructing hand-object interaction (HOI) from challenging monocular videos remains difficult: methods often assume known objects or curated scenes, and separately estimated hands and objects easily become misaligned under clutter, occlusion, and unseen object geometries. Targeting this setting, we present Open-CHOIR, an Open-world Contact-aware HOI Reconstruction framework for a monocular camera, using contact as an explicit coupling signal between hands and objects. Open-CHOIR first initializes a coarse, contact-agnostic 4D HOI sequence from open-world visual priors. It then introduces a generative HOI spatial rectification module to predict ray-depth corrections and rectify hand-object relative placement, then derive initial per-frame contact correspondences on the rectified geometry. Last, a contact-aware joint optimization with dynamically updated contact constraints enforces geometric, temporal, and contact consistency. Experiments on controlled and challenging videos show that Open-CHOIR improves object reconstruction, physical plausibility, and temporal consistency over state-of-the-art methods. Code, data, and pretrained weights are available at https://github.com/hxwork/CHOIR.

Förderung

  • Research Grants Council of the Hong Kong Special Administrative Region, China

Zitieren

APA

Xu, H., Liu, Y., Wang, Y., Fu, C., & Mitra, N. (2026). Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction. ACM Transactions on Graphics, 45(6), 1-12. https://doi.org/10.1145/3842509

Vancouver

Xu H, Liu Y, Wang Y, Fu C, Mitra N. Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction. ACM Transactions on Graphics. 2026;45(6):1-12. doi:10.1145/3842509

BibTeX
@article{Xu2026,
  author = {Xu, Hao and Liu, Yilin and Wang, Yinqiao and Fu, Chi-Wing and Mitra, Niloy},
  title = {Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction},
  journal = {ACM Transactions on Graphics},
  year = {2026},
  volume = {45},
  number = {6},
  pages = {1--12},
  doi = {10.1145/3842509}
}
Kostenlos starten →