End-to-End Transformer-Based Models in Textual-Based NLP
AI · 05.01.2023 · 4 (1) · 54-110
DOI: 10.3390/ai4010004 ↗Kurz gefasst
Fachartikel in der Zeitschrift AI, online erschienen am 05.01.2023. Verfasst von Abir Rahali und 1 weiteren Autor:innen. Die Arbeit wurde bisher 166-mal zitiert (Crossref) und nennt 412 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Sciences and Engineering Research Council of Canada (NSERC).
Abstract (Originalsprache)
Transformer architectures are highly expressive because they use self-attention mechanisms to encode long-range dependencies in the input sequences. In this paper, we present a literature review on Transformer-based (TB) models, providing a detailed overview of each model in comparison to the Transformer’s standard architecture. This survey focuses on TB models used in the field of Natural Language Processing (NLP) for textual-based tasks. We begin with an overview of the fundamental concepts at the heart of the success of these models. Then, we classify them based on their architecture and training mode. We compare the advantages and disadvantages of popular techniques in terms of architectural design and experimental value. Finally, we discuss open research, directions, and potential future work to help solve current TB application challenges in NLP.
Förderung
- Natural Sciences and Engineering Research Council of Canada (NSERC)
Zitieren
APA
Rahali, A., & Akhloufi, M. A. (2023). End-to-End Transformer-Based Models in Textual-Based NLP. AI, 4(1), 54-110. https://doi.org/10.3390/ai4010004
Vancouver
Rahali A, Akhloufi MA. End-to-End Transformer-Based Models in Textual-Based NLP. AI. 2023;4(1):54-110. doi:10.3390/ai4010004
BibTeX
@article{Rahali2023,
author = {Rahali, Abir and Akhloufi, Moulay A.},
title = {End-to-End Transformer-Based Models in Textual-Based NLP},
journal = {AI},
year = {2023},
volume = {4},
number = {1},
pages = {54--110},
doi = {10.3390/ai4010004}
}
EUVIMED