EUVIMEDEUVIMED entdecken
Open Access · CC BY-ND 4.0

Efficient Transformers: A Survey

Yi Tay, Mostafa Dehghani, Dara Bahri, Donald Metzler

ACM Computing Surveys · 30.06.2023 · 55 (6) · 1-28

DOI: 10.1145/3530811 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 07.12.2022. Verfasst von Yi Tay und 3 weiteren Autor:innen. Die Arbeit wurde bisher 917-mal zitiert (Crossref) und nennt 96 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision, and reinforcement learning. In the field of natural language processing for example, Transformers have become an indispensable staple in the modern deep learning stack. Recently, a dizzying number of “X-former” models have been proposed—Reformer, Linformer, Performer, Longformer, to name a few—which improve upon the original Transformer architecture, many of which make improvements around computational and memory efficiency . With the aim of helping the avid researcher navigate this flurry, this article characterizes a large and thoughtful selection of recent efficiency-flavored “X-former” models, providing an organized and comprehensive overview of existing work and models across multiple domains.

Zitieren

APA

Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023). Efficient Transformers: A Survey. ACM Computing Surveys, 55(6), 1-28. https://doi.org/10.1145/3530811

Vancouver

Tay Y, Dehghani M, Bahri D, Metzler D. Efficient Transformers: A Survey. ACM Computing Surveys. 2023;55(6):1-28. doi:10.1145/3530811

BibTeX
@article{Tay2023,
  author = {Tay, Yi and Dehghani, Mostafa and Bahri, Dara and Metzler, Donald},
  title = {Efficient Transformers: A Survey},
  journal = {ACM Computing Surveys},
  year = {2023},
  volume = {55},
  number = {6},
  pages = {1--28},
  doi = {10.1145/3530811}
}
Kostenlos starten →