TactGen: Tactile Sensory Data Generation via Zero-Shot Sim-to-Real Transfer (Abstract Reprint)
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (47) · 39909-39909
DOI: 10.1609/aaai.v40i47.41424 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Shaohong Zhong und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Recent advances in machine learning have driven a step-change in robot perception with modalities such as vision, where large amounts of training data are readily available or cheap to collect. However, in tactile perception, the relatively high cost of data collection still largely impedes the adoption of such data-driven learning solutions. In this article, we introduce TactGen, a novel, cross-modal framework to tackle this challenge. In particular, using a two-step data generation pipeline, we leverage easily accessible vision data to synthesise artificial tactile data for downstream classifier training. Specifically, we use readily collected video data of objects of interest to efficiently learn neural radiance field (NeRF) representations. The NeRF models are then used to render red–green–blue-depth (RGBD) images from any desired vantage points. In the second stage, the RGBD images are translated into corresponding tactile images typically generated by camera-based tactile sensors using a conditional generative adversarial network (cGAN). The cGAN model is itself trained with a large set of visuo-tactile images collected in simulation, and can be transferred into the real world without fine-tuning or additional data collection. We extensively validate this approach in the context of tactile object classification, showing that it effectively reduces data collection time by a factor of 20 while achieving similar performance to training a classifier on manually collected real data.
Zitieren
APA
Zhong, S., Albini, A., Maiolino, P., & Posner, I. (2026). TactGen: Tactile Sensory Data Generation via Zero-Shot Sim-to-Real Transfer (Abstract Reprint). Proceedings of the AAAI Conference on Artificial Intelligence, 40(47), 39909-39909. https://doi.org/10.1609/aaai.v40i47.41424
Vancouver
Zhong S, Albini A, Maiolino P, Posner I. TactGen: Tactile Sensory Data Generation via Zero-Shot Sim-to-Real Transfer (Abstract Reprint). Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(47):39909-39909. doi:10.1609/aaai.v40i47.41424
BibTeX
@article{Zhong2026,
author = {Zhong, Shaohong and Albini, Alessandro and Maiolino, Perla and Posner, Ingmar},
title = {TactGen: Tactile Sensory Data Generation via Zero-Shot Sim-to-Real Transfer (Abstract Reprint)},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {47},
pages = {39909--39909},
doi = {10.1609/aaai.v40i47.41424}
}
EUVIMED