EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Tele360: Real-Time Feed-forward Human Reconstruction from Sparse Unposed Cameras

Hanzhang Tu, Zhanfeng Liao, Wei Min, Jiajun Zhang, Yebin Liu

ACM Transactions on Graphics · 09.11.2026 · 45 (6) · 1-18

DOI: 10.1145/3842578 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 09.11.2026. Verfasst von Hanzhang Tu und 4 weiteren Autor:innen. Die Arbeit nennt 97 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Live free-viewpoint visualization of real humans is critical for immersive communication and interactive digital experiences. Existing methods either rely on computationally expensive optimization or require calibrated cameras and low-resolution inputs, making real-time high-resolution deployment impractical. In this work, we present Tele360, the first real-time feedforward system for dynamic human reconstruction and live free-viewpoint visualization from sparse, unposed RGB streams. Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance in a single forward pass. To achieve this, we start by designing a lightweight sparsity-aware multi-view transformer backbone that tokenizes foreground human regions while preserving global context through a shared scene token. We then employ a fully transformer-based Gaussian decoder to mitigate convolution-induced over-smoothing while keeping decoding sparse and efficient. In addition, we introduce a hybrid feature pyramid that injects multi-scale appearance cues into geometry prediction. We further introduce a lightweight differentiable Levenberg-Marquardt camera refinement layer to enhance multi-view consistency and geometric alignment. Moreover, to stabilize learning under sparse, unposed inputs, we transfer multi-view geometry priors from a large visual-geometry foundation model via teacher-student distillation. Finally, the predicted Gaussian maps are streamed with video codecs to remote devices for interactive free-viewpoint rendering. Together, these designs form a balanced pipeline between efficiency and fidelity. Extensive experiments show that Tele360 achieves state-of-the-art visual quality on studio benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU. Additional captured sequences illustrate its performance across varied subjects, clothing, and motions under our multi-camera setup. Inference code and pretrained weights will be released.

Zitieren

APA

Tu, H., Liao, Z., Min, W., Zhang, J., & Liu, Y. (2026). Tele360: Real-Time Feed-forward Human Reconstruction from Sparse Unposed Cameras. ACM Transactions on Graphics, 45(6), 1-18. https://doi.org/10.1145/3842578

Vancouver

Tu H, Liao Z, Min W, Zhang J, Liu Y. Tele360: Real-Time Feed-forward Human Reconstruction from Sparse Unposed Cameras. ACM Transactions on Graphics. 2026;45(6):1-18. doi:10.1145/3842578

BibTeX
@article{Tu2026,
  author = {Tu, Hanzhang and Liao, Zhanfeng and Min, Wei and Zhang, Jiajun and Liu, Yebin},
  title = {Tele360: Real-Time Feed-forward Human Reconstruction from Sparse Unposed Cameras},
  journal = {ACM Transactions on Graphics},
  year = {2026},
  volume = {45},
  number = {6},
  pages = {1--18},
  doi = {10.1145/3842578}
}
Kostenlos starten →