Simultaneous text and gesture generation for social robots with small language models
Frontiers in Robotics and AI · 16.05.2025 · 12
DOI: 10.3389/frobt.2025.1581024 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Robotics and AI, online erschienen am 16.05.2025. Verfasst von Alessio Galatolo und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Information Technology, Uppsala University, Uppsala, Sweden. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
As social robots gain advanced communication capabilities, users increasingly expect coherent verbal and non-verbal behaviours. Recent work has shown that Large Language Models (LLMs) can support autonomous generation of such multimodal behaviours. However, current LLM-based approaches to non-verbal behaviour often involve multi-step reasoning with large, closed-source models-resulting in significant computational overhead and limiting their feasibility in low-resource or privacy-constrained environments.
Methods
To address these limitations, we propose a novel method for simultaneous generation of text and gestures with minimal computational overhead compared to plain text generation. Our system does not produce low-level joint trajectories, but instead predicts high-level communicative intentions, which are mapped to platform-specific expressions. Central to our approach is the introduction of lightweight, robot-specific “gesture heads” derived from the LLM’s architecture, requiring no pose-based datasets and enabling generalisability across platforms.
Results
We evaluate our method on two distinct robot platforms: Furhat (facial expressions) and Pepper (bodily gestures). Experimental results demonstrate that our method maintains behavioural quality while introducing negligible computational and memory overhead. Furthermore, the gesture heads operate in parallel with the language generation component, ensuring scalability and responsiveness even on small or locally deployed models.
Discussion
Our approach supports the use of Small Language Models for multimodal generation, offering an effective alternative to existing high-resource methods. By abstracting gesture generation and eliminating reliance on platform-specific motion data, we enable broader applicability in real-world, low-resource, and privacy-sensitive HRI settings.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Information Technology, Uppsala University, Uppsala, Sweden.
- Department of Women and Children's Health, Uppsala University, Uppsala, Sweden.
Zitieren
APA
Galatolo, A., & Winkle, K. (2025). Simultaneous text and gesture generation for social robots with small language models. Frontiers in Robotics and AI, 12. https://doi.org/10.3389/frobt.2025.1581024
Vancouver
Galatolo A, Winkle K. Simultaneous text and gesture generation for social robots with small language models. Frontiers in Robotics and AI. 2025;12. doi:10.3389/frobt.2025.1581024
BibTeX
@article{Galatolo2025,
author = {Galatolo, Alessio and Winkle, Katie},
title = {Simultaneous text and gesture generation for social robots with small language models},
journal = {Frontiers in Robotics and AI},
year = {2025},
volume = {12},
doi = {10.3389/frobt.2025.1581024}
}
EUVIMED