EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models

Moritz Alexander Messerschmidt, Purnima Kamath, Yadeesha Weerasinghe, Suranga Nanayakkara

ACM Transactions on Computer-Human Interaction · 31.08.2026 · 33 (4) · 1-44

DOI: 10.1145/3819584 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Computer-Human Interaction, online erschienen am 08.08.2026. Verfasst von Moritz Alexander Messerschmidt und 3 weiteren Autor:innen. Die Arbeit nennt 139 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We introduce Prompt-to-Touch, a proof-of-concept of a multi-step pipeline that generates haptic effects based on the textual description of a haptic experience. Our pipeline first translates the haptic effect description to a sound effect description using our Foley-Interpreter component. It then uses a text-to-audio model to generate a sound effect from the sound description. Afterwards, the sound effect is converted into a perceivable haptic effect using our Dynamic-Audio-Processor . Finally, the haptic effect is post-processed to compensate for actuator-specific frequency response characteristics. We validate our concept in two preliminary human evaluation studies (n = 20, n = 10) and a technical analysis. Our results indicate that our pipeline can generate effects to enhance immersive multimedia experiences, abstract desktop/XR interactions, and social communication applications. They also still reveal significant potential for further improvement. Our pipeline could be used to develop future text-driven haptic design and automation tools. We provide open-source code to support future extensions.

Zitieren

APA

Messerschmidt, M. A., Kamath, P., Weerasinghe, Y., & Nanayakkara, S. (2026). Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models. ACM Transactions on Computer-Human Interaction, 33(4), 1-44. https://doi.org/10.1145/3819584

Vancouver

Messerschmidt MA, Kamath P, Weerasinghe Y, Nanayakkara S. Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models. ACM Transactions on Computer-Human Interaction. 2026;33(4):1-44. doi:10.1145/3819584

BibTeX
@article{Messerschmidt2026,
  author = {Messerschmidt, Moritz Alexander and Kamath, Purnima and Weerasinghe, Yadeesha and Nanayakkara, Suranga},
  title = {Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models},
  journal = {ACM Transactions on Computer-Human Interaction},
  year = {2026},
  volume = {33},
  number = {4},
  pages = {1--44},
  doi = {10.1145/3819584}
}
Kostenlos starten →