Progressive Learning of 3D Reconstruction Network From 2D GAN Data
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-02 · 46 (2) · 793-804
DOI: 10.1109/tpami.2023.3324806 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-02. Verfasst von Aysegul Dundar und 3 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 59 Literaturangaben.
Abstract (Originalsprache)
This paper presents a method to reconstruct high-quality textured 3D models from single images. Current methods rely on datasets with expensive annotations; multi-view images and their camera parameters. Our method relies on GAN generated multi-view image datasets which have a negligible annotation cost. However, they are not strictly multi-view consistent and sometimes GANs output distorted images. This results in degraded reconstruction qualities. In this work, to overcome these limitations of generated datasets, we have two main contributions which lead us to achieve state-of-the-art results on challenging objects: 1) A robust multi-stage learning scheme that gradually relies more on the models own predictions when calculating losses and 2) A novel adversarial learning pipeline with online pseudo-ground truth generations to achieve fine details. Our work provides a bridge from 2D supervisions of GAN models to 3D reconstruction models and removes the expensive annotation efforts. We show significant improvements over previous methods whether they were trained on GAN generated multi-view images or on real images with expensive annotations.
Zitieren
APA
Dundar, A., Gao, J., Tao, A., & Catanzaro, B. (2024). Progressive Learning of 3D Reconstruction Network From 2D GAN Data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2), 793-804. https://doi.org/10.1109/tpami.2023.3324806
Vancouver
Dundar A, Gao J, Tao A, Catanzaro B. Progressive Learning of 3D Reconstruction Network From 2D GAN Data. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(2):793-804. doi:10.1109/tpami.2023.3324806
BibTeX
@article{Dundar2024,
author = {Dundar, Aysegul and Gao, Jun and Tao, Andrew and Catanzaro, Bryan},
title = {Progressive Learning of 3D Reconstruction Network From 2D GAN Data},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {2},
pages = {793--804},
doi = {10.1109/tpami.2023.3324806}
}
EUVIMED