Transformers as the Foundation of Large Language Models: A Comprehensive Review
International Journal of Innovations in Science and Technology · 08.11.2025 · 2705
DOI: 10.33411/ijist/1632 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Innovations in Science and Technology, online erschienen am 08.11.2025. Verfasst von Muhammad Rayan Shaikh und 4 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The transformation of Transformer architecture has led the way into a new era for NLP, as it broke the traditional RNNs, LSTMs, Seq2Seq models, etc. As their main feature, the Revolution of Transformers was the hybridization of self-attention and multiheaded attention, which allowed the models to learn dependencies across time spans of any length through positioning methods. This resulted in a quick and efficient process for training large-scale Language Models (LLMs) that could handle the data very well and simultaneously learn the long-term dependencies. This paper is titled "Transformers as the Foundation of Large Language Models: A Comprehensive Review", and it not only reflects but also presents a critically reviewed path taken by LLMs from BERT to GPT-4 and beyond, along with the better reasoning, arithmetic, and instruction following attributed to the scaling up of architecture. The review further indicates and discusses the current concerns regarding efficiency, bias, interpretability, and domain specialization, and warns that settling these issues might dictate the fate of T-bases improvements. The authors aim through this project to provide an exhaustive comprehension of the setting in which Transformers enabled LLMs and actively directed the development of contemporary AI research.
Zitieren
APA
Shaikh, M. R., Shahryar, N., Mahboob, K., Naiyar, Q., & Talha, M. (2025). Transformers as the Foundation of Large Language Models: A Comprehensive Review. International Journal of Innovations in Science and Technology, 2705. https://doi.org/10.33411/ijist/1632
Vancouver
Shaikh MR, Shahryar N, Mahboob K, Naiyar Q, Talha M. Transformers as the Foundation of Large Language Models: A Comprehensive Review. International Journal of Innovations in Science and Technology. 2025:2705. doi:10.33411/ijist/1632
BibTeX
@article{Shaikh2025,
author = {Shaikh, Muhammad Rayan and Shahryar, Nida and Mahboob, Khalid and Naiyar, Qurrat-ul-Ain and Talha, Muhammad},
title = {Transformers as the Foundation of Large Language Models: A Comprehensive Review},
journal = {International Journal of Innovations in Science and Technology},
year = {2025},
pages = {2705},
doi = {10.33411/ijist/1632}
}
EUVIMED