Conceptual Combination in Large Language Models: Uncovering Implicit Relational Interpretations in Compound Words With Contextualized Word Embeddings
Cognitive Science · 2025
DOI: 10.1111/cogs.70048 ↗At a glance
Journal article in Cognitive Science, 2025. Written by Marco Ciapparelli and 2 co-authors. Institution: Department of Psychology, University of Milano-Bicocca. Major MEDLINE topics: Semantics, Psycholinguistics, Concept Formation, Comprehension. Cited 2 times so far (Crossref); 89 references. Funded by: European Research Council.
Abstract (original language)
Large language models (LLMs) have been proposed as candidate models of human semantics, and as such, they must be able to account for conceptual combination. This work explores the ability of two LLMs, namely, BERT‐base and Llama‐2‐13b, to reveal the implicit meaning of existing and novel compound words. According to psycholinguistic theories, understanding the meaning of a compound (e.g., “snowman”) involves its automatic decomposition into constituent meanings (“snow,” “man”), which are then connected by an implicit semantic relation selected from a set of possible competitors (FOR, MADE OF , BY, …) to obtain a plausible interpretation (“man MADE OF snow”). Here, we leverage the flexibility of LLMs to obtain contextualized representations for both target compounds (e.g., “snowman”) and their implicit interpretations (e.g., “man MADE OF snow”). We demonstrate that replacing a compound with a paraphrased version leads to changes to the embeddings that are inversely proportional to the paraphrase's plausibility, estimated by human raters. While this relation holds for both existing and novel compounds, results obtained for novel compounds are substantially weaker, and older distributional models outperform LLMs. Nonetheless, the present results show that LLMs can offer a valid approximation of the internal structure of compound words posited by cognitive theories, thus representing a promising tool to model word senses that are at once implicit and possible.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Psychology, University of Milano-Bicocca.
- Artefact, Utrecht.
- NeuroMI, Milan Center for Neuroscience.
ORCID profiles: Marco Ciapparelli
Funding
- European Research Council
Cite
APA
Ciapparelli, M., Zarbo, C., & Marelli, M. (2025). Conceptual Combination in Large Language Models: Uncovering Implicit Relational Interpretations in Compound Words With Contextualized Word Embeddings. Cognitive Science. https://doi.org/10.1111/cogs.70048
Vancouver
Ciapparelli M, Zarbo C, Marelli M. Conceptual Combination in Large Language Models: Uncovering Implicit Relational Interpretations in Compound Words With Contextualized Word Embeddings. Cognitive Science. 2025. doi:10.1111/cogs.70048
BibTeX
@article{Ciapparelli2025,
author = {Ciapparelli, Marco and Zarbo, Calogero and Marelli, Marco},
title = {Conceptual Combination in Large Language Models: Uncovering Implicit Relational Interpretations in Compound Words With Contextualized Word Embeddings},
journal = {Cognitive Science},
year = {2025},
doi = {10.1111/cogs.70048}
}
EUVIMED