In-Hand 3D Object Reconstruction from a Monocular RGB Video
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (3) · 2525-2533
DOI: 10.1609/aaai.v38i3.28029 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Shijian Jiang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref).
Abstract (Originalsprache)
Our work aims to reconstruct a 3D object that is held and rotated by a hand in front of a static RGB camera. Previous methods that use implicit neural representations to recover the geometry of a generic hand-held object from multi-view images achieved compelling results in the visible part of the object. However, these methods falter in accurately capturing the shape within the hand-object contact region due to occlusion. In this paper, we propose a novel method that deals with surface reconstruction under occlusion by incorporating priors of 2D occlusion elucidation and physical contact constraints. For the former, we introduce an object amodal completion network to infer the 2D complete mask of objects under occlusion. To ensure the accuracy and view consistency of the predicted 2D amodal masks, we devise a joint optimization method for both amodal mask refinement and 3D reconstruction. For the latter, we impose penetration and attraction constraints on the local geometry in contact regions. We evaluate our approach on HO3D and HOD datasets and demonstrate that it outperforms the state-of-the-art methods in terms of reconstruction surface quality, with an improvement of 52% on HO3D and 20% on HOD. Project webpage: https://east-j.github.io/ihor.
Zitieren
APA
Jiang, S., Ye, Q., Xie, R., Huo, Y., Li, X., Zhou, Y., & Chen, J. (2024). In-Hand 3D Object Reconstruction from a Monocular RGB Video. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 2525-2533. https://doi.org/10.1609/aaai.v38i3.28029
Vancouver
Jiang S, Ye Q, Xie R, Huo Y, Li X, Zhou Y, et al. In-Hand 3D Object Reconstruction from a Monocular RGB Video. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(3):2525-2533. doi:10.1609/aaai.v38i3.28029
BibTeX
@article{Jiang2024,
author = {Jiang, Shijian and Ye, Qi and Xie, Rengan and Huo, Yuchi and Li, Xiang and Zhou, Yang and Chen, Jiming},
title = {In-Hand 3D Object Reconstruction from a Monocular RGB Video},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {3},
pages = {2525--2533},
doi = {10.1609/aaai.v38i3.28029}
}
EUVIMED