EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Language-Guided Gesture and Virtual Human Communication through Multimodal Motion and Sign Representation Learning

Lucas Nyström

INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES · 08.08.2026 · 6 (2) · 51-60

DOI: 10.66238/ijrss.v6i2.121 ↗

Kurz gefasst

Fachartikel in der Zeitschrift INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES, online erschienen am 08.08.2026. Verfasst von Lucas Nyström. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The seamless integration of natural language understanding and non-verbal behavior generation remains a formidable challenge in the development of lifelike virtual humans. This paper presents a comprehensive framework for language-guided gesture and virtual human communication, leveraging advanced multimodal motion and sign representation learning. We propose a novel architecture that maps textual and linguistic inputs into a shared latent space aligned with human motion kinematics, enabling the generation of contextually appropriate co-speech gestures and anatomically accurate sign language sequences. By modeling the intricate dependencies between semantic intent and spatial-temporal motion dynamics, the proposed system bridges the gap between discrete linguistic tokens and continuous skeletal articulations. Our approach utilizes a massive corpus of paired text-motion data to train a transformer-based representation model that inherently distinguishes between the syntactic rigor of sign language and the expressive fluidity of conversational gestures. Through extensive experimental validation, including both objective kinematic metrics and subjective perceptual evaluations, we demonstrate that this framework significantly outperforms existing baseline methods in terms of semantic appropriateness, motion naturalness, and temporal synchrony. The findings highlight the potential of unified multimodal representations to facilitate highly immersive and socially intelligent human-computer interactions in virtual environments.

Zitieren

APA

Nyström, L. (2026). Language-Guided Gesture and Virtual Human Communication through Multimodal Motion and Sign Representation Learning. INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES, 6(2), 51-60. https://doi.org/10.66238/ijrss.v6i2.121

Vancouver

Nyström L. Language-Guided Gesture and Virtual Human Communication through Multimodal Motion and Sign Representation Learning. INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES. 2026;6(2):51-60. doi:10.66238/ijrss.v6i2.121

BibTeX
@article{Nystrm2026,
  author = {Nyström, Lucas},
  title = {Language-Guided Gesture and Virtual Human Communication through Multimodal Motion and Sign Representation Learning},
  journal = {INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES},
  year = {2026},
  volume = {6},
  number = {2},
  pages = {51--60},
  doi = {10.66238/ijrss.v6i2.121}
}
Kostenlos starten →