EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Capacitive Tactile Sensor Digital Twin for Real-Time Synthetic Data Generation and Sim-to-Real Transfer in NVIDIA Isaac Sim

Berith Atemoztli De la Cruz Sánchez, Jean-Philippe Roberge

Applied Sciences · 03.08.2026 · 16 (15) · 7708

DOI: 10.3390/app16157708 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 03.08.2026. Verfasst von Berith Atemoztli De la Cruz Sánchez und 1 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Sciences and Engineering Research Council of Canada, Fonds de recherche du Québec—Nature et technologies.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With advances in robotic manipulation in recent years, tactile sensing has become increasingly important in scenarios where visual information is unreliable or insufficient. However, the development of learning-based tactile algorithms and policies is limited by the cost and time required to collect large-scale physical datasets. While robotic simulation offers an alternative for synthetic data generation, current simulation platforms, such as NVIDIA Isaac Sim, lack integrated capacitive tactile sensors. This paper presents a finite element method (FEM)-based digital twin of a capacitive tactile sensor and an extension for NVIDIA Isaac Sim that enables the real-time generation of synthetic tactile data directly within the simulation environment. The proposed system extracts nodal deformations from the Isaac Sim PhysX engine and uses a convolutional neural network (CNN) to predict synthetic tactile maps that replicate the response of the physical sensor. We further demonstrate adaptability by retraining the model from an initial sensor to a second capacitive sensor, the Robotiq TSF-85, operating under the same sensing principle. The complete generation pipeline executes in 8.04 ms per frame on the laptop configuration and 6.71 ms on the workstation, enabling real-time operation at 60 Hz on both, and at 120 Hz on the workstation. The similarity of the generated tactile data for the Robotiq TSF-85 is evaluated using complementary similarity metrics, achieving a mean Structural Similarity Index Measure (SSIM) of 0.727 ± 0.16 and a mean Pearson correlation of 0.87 ± 0.16 against real measurements. To demonstrate the utility of the proposed framework, a shape-recognition task (cylinder, sphere, cube) was performed using only synthetic tactile data and evaluated on real-world sensor data, achieving an accuracy of 69.3% with zero real training labels. By enabling integrated tactile simulation and synthetic data generation within Isaac Sim, this work provides a practical tool for tactile perception using capacitive tactile sensors.

Förderung

  • Natural Sciences and Engineering Research Council of Canada
  • Fonds de recherche du Québec—Nature et technologies

Zitieren

APA

Sánchez, B. A. D. L. C., & Roberge, J. (2026). A Capacitive Tactile Sensor Digital Twin for Real-Time Synthetic Data Generation and Sim-to-Real Transfer in NVIDIA Isaac Sim. Applied Sciences, 16(15), 7708. https://doi.org/10.3390/app16157708

Vancouver

Sánchez BADLC, Roberge J. A Capacitive Tactile Sensor Digital Twin for Real-Time Synthetic Data Generation and Sim-to-Real Transfer in NVIDIA Isaac Sim. Applied Sciences. 2026;16(15):7708. doi:10.3390/app16157708

BibTeX
@article{Snchez2026,
  author = {Sánchez, Berith Atemoztli De la Cruz and Roberge, Jean-Philippe},
  title = {A Capacitive Tactile Sensor Digital Twin for Real-Time Synthetic Data Generation and Sim-to-Real Transfer in NVIDIA Isaac Sim},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {15},
  pages = {7708},
  doi = {10.3390/app16157708}
}
Kostenlos starten →