EUVIMEDEUVIMED entdecken
Crossref

Development of compositionality through interactive learning of language and action of robots

Prasanna Vijayaraghavan iD, Jeffrey Frederic Queißer iD, Sergio Verduzco Flores iD, Jun Tani iD

Science Robotics · 22.01.2025 · 10 (98)

DOI: 10.1126/scirobotics.adp0751 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science Robotics, erschienen am 22.01.2025. Verfasst von Prasanna Vijayaraghavan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Okinawa Institute of Science and Technology, Okinawa, Japan. Hauptthemen laut MEDLINE: Robotics, Language, Learning. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 66 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Humans excel at applying learned behavior to unlearned situations. A crucial component of this generalization behavior is our ability to compose/decompose a whole into reusable parts, an attribute known as compositionality. One of the fundamental questions in robotics concerns this characteristic: How can linguistic compositionality be developed concomitantly with sensorimotor skills through associative learning, particularly when individuals only learn partial linguistic compositions and their corresponding sensorimotor patterns? To address this question, we propose a brain-inspired neural network model that integrates vision, proprioception, and language into a framework of predictive coding and active inference on the basis of the free-energy principle. The effectiveness and capabilities of this model were assessed through various simulation experiments conducted with a robot arm. Our results show that generalization in learning to unlearned verb-noun compositions is significantly enhanced when training variations of task composition are increased. We attribute this to self-organized compositional structures in linguistic latent state space being influenced substantially by sensorimotor learning. Ablation studies show that visual attention and working memory are essential to accurately generate visuomotor sequences to achieve linguistically represented goals. These insights advance our understanding of mechanisms underlying development of compositionality through interactions of linguistic and sensorimotor experience.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RoboticsLanguageLearningHumansNeural Networks, ComputerComputer SimulationProprioceptionLinguisticsBrainMemory, Short-Term

Autor:innen und Institutionen

  • Okinawa Institute of Science and Technology, Okinawa, Japan.

ORCID-Profile: Prasanna Vijayaraghavan, Jeffrey Frederic Queißer, Sergio Verduzco Flores, Jun Tani

Zitieren

APA

Vijayaraghavan, P., Queißer, J. F., Flores, S. V., & Tani, J. (2025). Development of compositionality through interactive learning of language and action of robots. Science Robotics, 10(98). https://doi.org/10.1126/scirobotics.adp0751

Vancouver

Vijayaraghavan P, Queißer JF, Flores SV, Tani J. Development of compositionality through interactive learning of language and action of robots. Science Robotics. 2025;10(98). doi:10.1126/scirobotics.adp0751

BibTeX
@article{Vijayaraghavan2025,
  author = {Vijayaraghavan, Prasanna and Queißer, Jeffrey Frederic and Flores, Sergio Verduzco and Tani, Jun},
  title = {Development of compositionality through interactive learning of language and action of robots},
  journal = {Science Robotics},
  year = {2025},
  volume = {10},
  number = {98},
  doi = {10.1126/scirobotics.adp0751}
}
Kostenlos starten →