Large language model developments
Royal Society Open Science · 04.11.2026 · 13 (11)
DOI: 10.1098/rsos.252099 ↗Kurz gefasst
Übersichtsarbeit in der Zeitschrift Royal Society Open Science, online erschienen am 04.11.2026. Verfasst von Philip Treleaven und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, University College London, London, UK. Die Arbeit nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large language models (LLMs) are generative artificial intelligence (AI) models trained with vast amounts of data and context, and excel at processing, understanding and generating human language, using transformer models to learn statistical relationships between tokens. LLMs are a subset of transformer models, and hence are closely related but not interchangeable. Given the ‘explosion’ of LLM research, the challenge is tracking emerging developments and matching LLMs to applications. This paper reviews LLM developments with the aim of understanding the underlying models’ computational strengths and weaknesses, building on previous work focusing on LLM optimization (Sharkey & Treleaven 2025 Optimising large language models: taxonomy and techniques. SSRN Scholarly Paper no. 5278456. Rochester, NY: SSRN). We find that despite their power, LLMs (transformers) face several key challenges. These include: (i) computational complexity and resource requirements—training and deploying large models is extremely costly and slow, and energy consumption is a concern; (ii) understanding long sequences and dependencies—even with enhancements, models can struggle with very long-range dependencies or context lengths beyond their window (e.g. maintaining coherence in a long document), which reflects the more fundamental absence of continual learning; (iii) interpretability and bias—it remains difficult to interpret why a model produces a given output, and models can inadvertently amplify biases present in training data; and (iv) optimization challenges, including fine-tuning for specific tasks without overfitting or losing generality and ensuring efficient inference (latency, the delay before a response is completed) for real-time applications. In summary, this review focuses on architectural, optimization and interpretability developments in LLMs. It does not aim to provide a systematic benchmark comparison, exhaustive implementation details or empirical performance evaluation, which are covered extensively elsewhere. The authors are committed to updating this review aligned to constant changes in the technology; please reach out with any additions.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science, University College London, London, UK.
ORCID-Profile: Philip Treleaven, Edward Sharkey
Zitieren
APA
Treleaven, P., Sharkey, E., & Craciun, S. (2026). Large language model developments. Royal Society Open Science, 13(11). https://doi.org/10.1098/rsos.252099
Vancouver
Treleaven P, Sharkey E, Craciun S. Large language model developments. Royal Society Open Science. 2026;13(11). doi:10.1098/rsos.252099
BibTeX
@article{Treleaven2026,
author = {Treleaven, Philip and Sharkey, Edward and Craciun, Sara},
title = {Large language model developments},
journal = {Royal Society Open Science},
year = {2026},
volume = {13},
number = {11},
doi = {10.1098/rsos.252099}
}
EUVIMED