EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

End-to-End Transformer-Based Models in Textual-Based NLP

Abir Rahali, Moulay A. Akhloufi

AI · 05.01.2023 · 4 (1) · 54-110

DOI: 10.3390/ai4010004 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI, online erschienen am 05.01.2023. Verfasst von Abir Rahali und 1 weiteren Autor:innen. Die Arbeit wurde bisher 166-mal zitiert (Crossref) und nennt 412 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Sciences and Engineering Research Council of Canada (NSERC).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformer architectures are highly expressive because they use self-attention mechanisms to encode long-range dependencies in the input sequences. In this paper, we present a literature review on Transformer-based (TB) models, providing a detailed overview of each model in comparison to the Transformer’s standard architecture. This survey focuses on TB models used in the field of Natural Language Processing (NLP) for textual-based tasks. We begin with an overview of the fundamental concepts at the heart of the success of these models. Then, we classify them based on their architecture and training mode. We compare the advantages and disadvantages of popular techniques in terms of architectural design and experimental value. Finally, we discuss open research, directions, and potential future work to help solve current TB application challenges in NLP.

Förderung

  • Natural Sciences and Engineering Research Council of Canada (NSERC)

Zitieren

APA

Rahali, A., & Akhloufi, M. A. (2023). End-to-End Transformer-Based Models in Textual-Based NLP. AI, 4(1), 54-110. https://doi.org/10.3390/ai4010004

Vancouver

Rahali A, Akhloufi MA. End-to-End Transformer-Based Models in Textual-Based NLP. AI. 2023;4(1):54-110. doi:10.3390/ai4010004

BibTeX
@article{Rahali2023,
  author = {Rahali, Abir and Akhloufi, Moulay A.},
  title = {End-to-End Transformer-Based Models in Textual-Based NLP},
  journal = {AI},
  year = {2023},
  volume = {4},
  number = {1},
  pages = {54--110},
  doi = {10.3390/ai4010004}
}
Kostenlos starten →