EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model

Eva Portelance iD, Michael C Frank iD, Dan Jurafsky iD

Cognitive Science · 2024

DOI: 10.1111/cogs.13448 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cognitive Science, 2024. Verfasst von Eva Portelance und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Linguistics, McGill University. Hauptthemen laut MEDLINE: Language, Learning. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 102 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Social Sciences and Humanities Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Interpreting a seemingly simple function word like “or,” “behind,” or “more” can require logical, numerical, and relational reasoning. How are such words learned by children? Prior acquisition theories have often relied on positing a foundation of innate knowledge. Yet recent neural‐network‐based visual question answering models apparently can learn to use function words as part of answering questions about complex visual scenes. In this paper, we study what these models learn about function words, in the hope of better understanding how the meanings of these words can be learned by both models and children. We show that recurrent models trained on visually grounded language learn gradient semantics for function words requiring spatial and numerical reasoning. Furthermore, we find that these models can learn the meanings of logical connectives and and or without any prior knowledge of logical reasoning as well as early evidence that they are sensitive to alternative expressions when interpreting language. Finally, we show that word learning difficulty is dependent on the frequency of models' input. Our findings offer proof‐of‐concept evidence that it is possible to learn the nuanced interpretations of function words in a visually grounded context by using non‐symbolic general statistical learning algorithms, without any prior knowledge of linguistic meaning.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LanguageLearningHumansSemanticsLanguage DevelopmentNeural Networks, ComputerChildLogic

Stichwörter der Autor:innen

Function word acquisitionLogical reasoningMultimodal statistical learningNeural network modelsProof of conceptReasoning skillsVisual question answeringVisually‐grounded language

Autor:innen und Institutionen

  • Department of Linguistics, McGill University.
  • Mila - Quebec Artificial Intelligence Institute.
  • Department of Psychology, Stanford University.
  • Department of Linguistics, Stanford University.

ORCID-Profile: Eva Portelance, Michael C Frank, Dan Jurafsky

Förderung

  • Social Sciences and Humanities Research Council

Zitieren

APA

Portelance, E., Frank, M. C., & Jurafsky, D. (2024). Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model. Cognitive Science. https://doi.org/10.1111/cogs.13448

Vancouver

Portelance E, Frank MC, Jurafsky D. Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model. Cognitive Science. 2024. doi:10.1111/cogs.13448

BibTeX
@article{Portelance2024,
  author = {Portelance, Eva and Frank, Michael C and Jurafsky, Dan},
  title = {Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model},
  journal = {Cognitive Science},
  year = {2024},
  doi = {10.1111/cogs.13448}
}
Kostenlos starten →