EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers

Zongyuan Yang, Mingjing Yi, Wanli Ma, Chenzhuo Fan, Bocheng Li, Baolin Liu, Yuke Lou, Yingde Song, Yongping Xiong, Zhengdong Guo, Shimu Wang

ACM Transactions on Graphics · 09.11.2026 · 45 (6) · 1-24

DOI: 10.1145/3842532 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 09.11.2026. Verfasst von Zongyuan Yang und 10 weiteren Autor:innen. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper addresses the challenge of incorporating 3D mesh directly into the sequence of Multimodal Large Language Models (MLLMs), rather than treating it as an external output. Diffusion-based large reconstruction models decouple semantic understanding from geometric reasoning, operating as stateless reconstructors conditioned on dense 2D pixel priors. Recent MLLM-based methods treat the 3D modality as an external output rather than an integral component of the multimodal sequence, making incremental adaptations without systematic analysis of how geometric manifolds align with MLLM feature spaces. We introduce EVA01, a framework that extends the modality boundary of MLLMs to incorporate 3D mesh understanding, generation, and context-aware editing, processing 3D tokens directly within the shared sequence rather than through rendered 2D intermediaries. Built upon a Mixture-of-Transformers (MoT) architecture, EVA01 decouples the model into a pre-trained Understanding Expert ( E und ) and a structurally mirrored Generation Expert ( E gen ), coupled through shared global self-attention with hard modality routing. This design aligns the semantic latent space of the MLLM backbone with the geometric manifold, enabling direct transfer of multimodal priors without intermediate 2D representations. Results show that EVA01 achieves state-of-the-art text-to-3D generation fidelity and unlocks robust long-context multi-turn geometric editing with identity preservation—a capability fundamentally inaccessible to stateless reconstruction pipelines. Our findings further offer architectural insights for integrating 2D foundation models with 3D tasks, informing the design of multimodal systems that operate directly on 3D. Project page: https://github.com/SeeleAI/OpenEVA.

Zitieren

APA

Yang, Z., Yi, M., Ma, W., Fan, C., Li, B., Liu, B., Lou, Y., Song, Y., Xiong, Y., Guo, Z., & Wang, S. (2026). EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers. ACM Transactions on Graphics, 45(6), 1-24. https://doi.org/10.1145/3842532

Vancouver

Yang Z, Yi M, Ma W, Fan C, Li B, Liu B, et al. EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers. ACM Transactions on Graphics. 2026;45(6):1-24. doi:10.1145/3842532

BibTeX
@article{Yang2026,
  author = {Yang, Zongyuan and Yi, Mingjing and Ma, Wanli and Fan, Chenzhuo and Li, Bocheng and Liu, Baolin and Lou, Yuke and Song, Yingde and Xiong, Yongping and Guo, Zhengdong and Wang, Shimu},
  title = {EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers},
  journal = {ACM Transactions on Graphics},
  year = {2026},
  volume = {45},
  number = {6},
  pages = {1--24},
  doi = {10.1145/3842532}
}
Kostenlos starten →