Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models
ACM Transactions on Computer-Human Interaction · 31.08.2026 · 33 (4) · 1-44
DOI: 10.1145/3819584 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Computer-Human Interaction, online erschienen am 08.08.2026. Verfasst von Moritz Alexander Messerschmidt und 3 weiteren Autor:innen. Die Arbeit nennt 139 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
We introduce Prompt-to-Touch, a proof-of-concept of a multi-step pipeline that generates haptic effects based on the textual description of a haptic experience. Our pipeline first translates the haptic effect description to a sound effect description using our Foley-Interpreter component. It then uses a text-to-audio model to generate a sound effect from the sound description. Afterwards, the sound effect is converted into a perceivable haptic effect using our Dynamic-Audio-Processor . Finally, the haptic effect is post-processed to compensate for actuator-specific frequency response characteristics. We validate our concept in two preliminary human evaluation studies (n = 20, n = 10) and a technical analysis. Our results indicate that our pipeline can generate effects to enhance immersive multimedia experiences, abstract desktop/XR interactions, and social communication applications. They also still reveal significant potential for further improvement. Our pipeline could be used to develop future text-driven haptic design and automation tools. We provide open-source code to support future extensions.
Zitieren
APA
Messerschmidt, M. A., Kamath, P., Weerasinghe, Y., & Nanayakkara, S. (2026). Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models. ACM Transactions on Computer-Human Interaction, 33(4), 1-44. https://doi.org/10.1145/3819584
Vancouver
Messerschmidt MA, Kamath P, Weerasinghe Y, Nanayakkara S. Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models. ACM Transactions on Computer-Human Interaction. 2026;33(4):1-44. doi:10.1145/3819584
BibTeX
@article{Messerschmidt2026,
author = {Messerschmidt, Moritz Alexander and Kamath, Purnima and Weerasinghe, Yadeesha and Nanayakkara, Suranga},
title = {Prompt-to-Touch: Towards Enabling Automatic Haptic Effect Generation from Text Prompts Using Text-to-Audio Models},
journal = {ACM Transactions on Computer-Human Interaction},
year = {2026},
volume = {33},
number = {4},
pages = {1--44},
doi = {10.1145/3819584}
}
EUVIMED