EUVIMEDOdkrijte EUVIMED
Open access · CC BY 4.0

Evidence Against Syntactic Encapsulation in Large Language Models

Thomas A McGee iD, Yiyang Zhang, Idan A Blank iD

Cognitive Science · 2026

DOI: 10.1111/cogs.70187 ↗

At a glance

Journal article in Cognitive Science, 2026. Written by Thomas A McGee and 2 co-authors. Institution: Department of Psychology, University of California, Los Angeles. Major MEDLINE topics: Large Language Models, Semantics, Attention, Linguistics. Cited 3 times so far (Crossref); 97 references. The full text is freely available (open access, CC BY 4.0). Funded by: University of California.

PovzetekSubjectsBibliografski podatkiCiteRelated studies

Abstract (original language)

Transformer‐based large language models (LLMs) have recently demonstrated exceptional performance in a variety of linguistic tasks. LLMs primarily combine information across words in a sentence using the attention mechanism, implemented by “attention heads:” these components assign numerical weights linking different words in the input to one another, capturing different relationships between these words. Some attention heads automatically learn to assign weights that accurately encode meaningful linguistic features including, importantly, heads that appear specialized for identifying particular syntactic dependencies. Are syntactic computations in such heads “encapsulated”, i.e., impenetrable to the influence of non‐syntactic information? Such encapsulated computations would be strikingly different from those of the human mind, where non‐syntactic information sources (e.g., semantics) influence parsing from the earliest moments of online processing, and where syntax and semantics are tightly linked in the mental lexicon. Here, we tested whether the activity of “syntax‐specialized” attention heads in transformer‐based LLMs is modulated by one type of semantic information: plausibility. In each of three LLMs (BERT, GPT‐2, and Llama 2), we first identified attention heads specialized for various dependency types; in nearly all cases tested, we then found that implausible semantic information reduces attention between the words that constitute the dependency for which a head is specialized. These results demonstrate that, even in attention heads that are the best a‐priori candidates for syntactic encapsulation, syntactic information is penetrable to semantics. These data are broadly consistent with the integration of syntax and semantics in human minds.

Subjects

MeSH terms (MEDLINE)

Large Language ModelsSemanticsAttentionLinguisticsHumans

Author keywords

Large language modelsModularityNeural networksSemanticsSyntax

Authors and institutions

  • Department of Psychology, University of California, Los Angeles.
  • Department of Linguistics, University of California, Los Angeles.

ORCID profiles: Thomas A McGee, Idan A Blank

Funding

  • University of California

Cite

APA

McGee, T. A., Zhang, Y., & Blank, I. A. (2026). Evidence Against Syntactic Encapsulation in Large Language Models. Cognitive Science. https://doi.org/10.1111/cogs.70187

Vancouver

McGee TA, Zhang Y, Blank IA. Evidence Against Syntactic Encapsulation in Large Language Models. Cognitive Science. 2026. doi:10.1111/cogs.70187

BibTeX
@article{McGee2026,
  author = {McGee, Thomas A and Zhang, Yiyang and Blank, Idan A},
  title = {Evidence Against Syntactic Encapsulation in Large Language Models},
  journal = {Cognitive Science},
  year = {2026},
  doi = {10.1111/cogs.70187}
}
Start for free →