Evidence Against Syntactic Encapsulation in Large Language Models
Cognitive Science · 2026
DOI: 10.1111/cogs.70187 ↗Kurz gefasst
Fachartikel in der Zeitschrift Cognitive Science, 2026. Verfasst von Thomas A McGee und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, University of California, Los Angeles. Hauptthemen laut MEDLINE: Large Language Models, Semantics, Attention, Linguistics. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 97 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: University of California.
Abstract (Originalsprache)
Transformer‐based large language models (LLMs) have recently demonstrated exceptional performance in a variety of linguistic tasks. LLMs primarily combine information across words in a sentence using the attention mechanism, implemented by “attention heads:” these components assign numerical weights linking different words in the input to one another, capturing different relationships between these words. Some attention heads automatically learn to assign weights that accurately encode meaningful linguistic features including, importantly, heads that appear specialized for identifying particular syntactic dependencies. Are syntactic computations in such heads “encapsulated”, i.e., impenetrable to the influence of non‐syntactic information? Such encapsulated computations would be strikingly different from those of the human mind, where non‐syntactic information sources (e.g., semantics) influence parsing from the earliest moments of online processing, and where syntax and semantics are tightly linked in the mental lexicon. Here, we tested whether the activity of “syntax‐specialized” attention heads in transformer‐based LLMs is modulated by one type of semantic information: plausibility. In each of three LLMs (BERT, GPT‐2, and Llama 2), we first identified attention heads specialized for various dependency types; in nearly all cases tested, we then found that implausible semantic information reduces attention between the words that constitute the dependency for which a head is specialized. These results demonstrate that, even in attention heads that are the best a‐priori candidates for syntactic encapsulation, syntactic information is penetrable to semantics. These data are broadly consistent with the integration of syntax and semantics in human minds.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Psychology, University of California, Los Angeles.
- Department of Linguistics, University of California, Los Angeles.
ORCID-Profile: Thomas A McGee, Idan A Blank
Förderung
- University of California
Zitieren
APA
McGee, T. A., Zhang, Y., & Blank, I. A. (2026). Evidence Against Syntactic Encapsulation in Large Language Models. Cognitive Science. https://doi.org/10.1111/cogs.70187
Vancouver
McGee TA, Zhang Y, Blank IA. Evidence Against Syntactic Encapsulation in Large Language Models. Cognitive Science. 2026. doi:10.1111/cogs.70187
BibTeX
@article{McGee2026,
author = {McGee, Thomas A and Zhang, Yiyang and Blank, Idan A},
title = {Evidence Against Syntactic Encapsulation in Large Language Models},
journal = {Cognitive Science},
year = {2026},
doi = {10.1111/cogs.70187}
}
EUVIMED