EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Language Models Can Resolve Reference Compositionally, But It’s Not Their Native Strength: The Case of the Personal Relation Task

Bart Evelo, Meaghan Fowlie, Denis Paperno

Transactions of the Association for Computational Linguistics · 01.07.2026 · 14 · 1584-1597

DOI: 10.1162/tacl.a.742 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 01.07.2026. Verfasst von Bart Evelo und 2 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Do neural models, such as Large Language Models, genuinely acquire compositional abilities for interpretation of natural language? When we talk about semantic interpretation, we can distinguish two complementary aspects: establishing what an expression refers to in the world (which we call the Extensional task) and representing its sense in a structured way (which we call the Intensional task). We evaluate LLMs and humans on both tasks in the setting of the Personal Relation Task (Paperno, 2022) in which, given a universe of people and their relationships with each other, one is asked to interpret a noun phrase such as Amber’s parent’s friend. Here, for the Intensional task, the answer is the formula friend(parent(amber)), and for the Extensional task, the person. We find that humans and LLMs show opposite strengths: humans perform better on Extensional than Intensional tasks, and LLMs vice versa. Our methodology brings greater nuance to the understanding of compositional abilities in modern machine learning models. Our results support the notion that the lack of referential grounding in LLM training is a crucial missing component in mimicking humanlike language understanding.

Zitieren

APA

Evelo, B., Fowlie, M., & Paperno, D. (2026). Language Models Can Resolve Reference Compositionally, But It’s Not Their Native Strength: The Case of the Personal Relation Task. Transactions of the Association for Computational Linguistics, 14, 1584-1597. https://doi.org/10.1162/tacl.a.742

Vancouver

Evelo B, Fowlie M, Paperno D. Language Models Can Resolve Reference Compositionally, But It’s Not Their Native Strength: The Case of the Personal Relation Task. Transactions of the Association for Computational Linguistics. 2026;14:1584-1597. doi:10.1162/tacl.a.742

BibTeX
@article{Evelo2026,
  author = {Evelo, Bart and Fowlie, Meaghan and Paperno, Denis},
  title = {Language Models Can Resolve Reference Compositionally, But It’s Not Their Native Strength: The Case of the Personal Relation Task},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  volume = {14},
  pages = {1584--1597},
  doi = {10.1162/tacl.a.742}
}
Kostenlos starten →