EUVIMEDEUVIMED entdecken
Crossref

High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning With Gaussian Splatting

Haoyu Zhao, Cheng Zeng, Linghao Zhuang, Yaxi Zhao, Shengke Xue, Hao Wang, Xingyue Zhao, Zhongyu Li, Kehan Li, Siteng Huang, Mingxiu Chen, Xin Li und 2 weitere

IEEE Robotics and Automation Letters · 2026-05 · 11 (5) · 5310-5317

DOI: 10.1109/lra.2026.3671535 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Robotics and Automation Letters, 2026-05. Verfasst von Haoyu Zhao und 13 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 36 Literaturangaben. Gefördert von: Central Guidance for Local Science and Technology Development Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The scalability of robotic learning is fundamentally bottlenecked by the significant cost and labor of real-world data collection. While simulated data offers a scalable alternative, it often fails to generalize to the real world due to significant gaps in visual appearance, physical properties, and object interactions. To address this, we propose RoboSimGS, a novel Real2Sim2Real framework that converts multi-view real-world images into high-fidelity, and physically interactive simulation environments for robotic manipulation. Our approach reconstructs scenes using a hybrid representation: 3D Gaussian Splatting (3DGS) captures the photorealistic appearance of the environment, while mesh primitives for interactive objects ensure accurate physics simulation. Crucially, our framework leverages a Multi-modal Large Language Model (MLLM) to automate the creation of physically plausible, articulated assets from visual data. The MLLM analyzes visual data to infer not only physical properties (e.g., density, stiffness) but also kinematic structures (e.g., hinges, sliding rails) of objects. We demonstrate that policies trained entirely on data generated by RoboSimGS achieve successful zero-shot Sim2Real transfer across a diverse set of real-world manipulation tasks. Furthermore, data from RoboSimGS significantly enhances the performance and generalization capabilities of SOTA methods. Our results validate RoboSimGS as a powerful and scalable solution for bridging the Sim2Real gap.

Förderung

  • Central Guidance for Local Science and Technology Development Fund

Zitieren

APA

Zhao, H., Zeng, C., Zhuang, L., Zhao, Y., Xue, S., Wang, H., Zhao, X., Li, Z., Li, K., Huang, S., Chen, M., Li, X., Zhao, D., & Zou, H. (2026). High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning With Gaussian Splatting. IEEE Robotics and Automation Letters, 11(5), 5310-5317. https://doi.org/10.1109/lra.2026.3671535

Vancouver

Zhao H, Zeng C, Zhuang L, Zhao Y, Xue S, Wang H, et al. High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning With Gaussian Splatting. IEEE Robotics and Automation Letters. 2026;11(5):5310-5317. doi:10.1109/lra.2026.3671535

BibTeX
@article{Zhao2026,
  author = {Zhao, Haoyu and Zeng, Cheng and Zhuang, Linghao and Zhao, Yaxi and Xue, Shengke and Wang, Hao and Zhao, Xingyue and Li, Zhongyu and Li, Kehan and Huang, Siteng and Chen, Mingxiu and Li, Xin and Zhao, Deli and Zou, Hua},
  title = {High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning With Gaussian Splatting},
  journal = {IEEE Robotics and Automation Letters},
  year = {2026},
  volume = {11},
  number = {5},
  pages = {5310--5317},
  doi = {10.1109/lra.2026.3671535}
}
Kostenlos starten →