EUVIMEDEUVIMED entdecken
Crossref

Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey

Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, Dan Roth

ACM Computing Surveys · 29.02.2024 · 56 (2) · 1-40

DOI: 10.1145/3605943 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 14.09.2023. Verfasst von Bonan Min und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1225-mal zitiert (Crossref) und nennt 227 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large, pre-trained language models (PLMs) such as BERT and GPT have drastically changed the Natural Language Processing (NLP) field. For numerous NLP tasks, approaches leveraging PLMs have achieved state-of-the-art performance. The key idea is to learn a generic, latent representation of language from a generic task once, then share it across disparate NLP tasks. Language modeling serves as the generic task, one with abundant self-supervised text available for extensive training. This article presents the key fundamental concepts of PLM architectures and a comprehensive view of the shift to PLM-driven NLP techniques. It surveys work applying the pre-training then fine-tuning, prompting, and text generation approaches. In addition, it discusses PLM limitations and suggested directions for future research.

Zitieren

APA

Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024). Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys, 56(2), 1-40. https://doi.org/10.1145/3605943

Vancouver

Min B, Ross H, Sulem E, Veyseh APB, Nguyen TH, Sainz O, et al. Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys. 2024;56(2):1-40. doi:10.1145/3605943

BibTeX
@article{Min2024,
  author = {Min, Bonan and Ross, Hayley and Sulem, Elior and Veyseh, Amir Pouran Ben and Nguyen, Thien Huu and Sainz, Oscar and Agirre, Eneko and Heintz, Ilana and Roth, Dan},
  title = {Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey},
  journal = {ACM Computing Surveys},
  year = {2024},
  volume = {56},
  number = {2},
  pages = {1--40},
  doi = {10.1145/3605943}
}
Kostenlos starten →