A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Applied Sciences · 01.03.2024 · 14 (5) · 2074
DOI: 10.3390/app14052074 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 01.03.2024. Verfasst von Rajvardhan Patil und 1 weiteren Autor:innen. Die Arbeit wurde bisher 261-mal zitiert (Crossref) und nennt 106 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Natural language processing (NLP) has significantly transformed in the last decade, especially in the field of language modeling. Large language models (LLMs) have achieved SOTA performances on natural language understanding (NLU) and natural language generation (NLG) tasks by learning language representation in self-supervised ways. This paper provides a comprehensive survey to capture the progression of advances in language models. In this paper, we examine the different aspects of language models, which started with a few million parameters but have reached the size of a trillion in a very short time. We also look at how these LLMs transitioned from task-specific to task-independent to task-and-language-independent architectures. This paper extensively discusses different pretraining objectives, benchmarks, and transfer learning methods used in LLMs. It also examines different finetuning and in-context learning techniques used in downstream tasks. Moreover, it explores how LLMs can perform well across many domains and datasets if sufficiently trained on a large and diverse dataset. Next, it discusses how, over time, the availability of cheap computational power and large datasets have improved LLM’s capabilities and raised new challenges. As part of our study, we also inspect LLMs from the perspective of scalability to see how their performance is affected by the model’s depth, width, and data size. Lastly, we provide an empirical comparison of existing trends and techniques and a comprehensive analysis of where the field of LLM currently stands.
Zitieren
APA
Patil, R., & Gudivada, V. (2024). A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs). Applied Sciences, 14(5), 2074. https://doi.org/10.3390/app14052074
Vancouver
Patil R, Gudivada V. A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs). Applied Sciences. 2024;14(5):2074. doi:10.3390/app14052074
BibTeX
@article{Patil2024,
author = {Patil, Rajvardhan and Gudivada, Venkat},
title = {A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {5},
pages = {2074},
doi = {10.3390/app14052074}
}
EUVIMED