EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

mGPT: Few-Shot Learners Go Multilingual

Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova, Anastasia Kozlova, Vladislav Mikhailov, Tatiana Shavrina

Transactions of the Association for Computational Linguistics · 31.01.2024 · 12 · 58-79

DOI: 10.1162/tacl_a_00633 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 31.01.2024. Verfasst von Oleh Shliazhko und 5 weiteren Autor:innen. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 87 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper introduces mGPT, a multilingual variant of GPT-3, pretrained on 61 languages from 25 linguistically diverse language families using Wikipedia and the C4 Corpus. We detail the design and pretraining procedure. The models undergo an intrinsic and extrinsic evaluation: language modeling in all languages, downstream evaluation on cross-lingual NLU datasets and benchmarks in 33 languages, and world knowledge probing in 23 languages. The in-context learning abilities are on par with the contemporaneous language models while covering a larger number of languages, including underrepresented and low-resource languages of the Commonwealth of Independent States and the indigenous peoples in Russia. The source code and the language models are publicly available under the MIT license.

Zitieren

APA

Shliazhko, O., Fenogenova, A., Tikhonova, M., Kozlova, A., Mikhailov, V., & Shavrina, T. (2024). mGPT: Few-Shot Learners Go Multilingual. Transactions of the Association for Computational Linguistics, 12, 58-79. https://doi.org/10.1162/tacl_a_00633

Vancouver

Shliazhko O, Fenogenova A, Tikhonova M, Kozlova A, Mikhailov V, Shavrina T. mGPT: Few-Shot Learners Go Multilingual. Transactions of the Association for Computational Linguistics. 2024;12:58-79. doi:10.1162/tacl_a_00633

BibTeX
@article{Shliazhko2024,
  author = {Shliazhko, Oleh and Fenogenova, Alena and Tikhonova, Maria and Kozlova, Anastasia and Mikhailov, Vladislav and Shavrina, Tatiana},
  title = {mGPT: Few-Shot Learners Go Multilingual},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  volume = {12},
  pages = {58--79},
  doi = {10.1162/tacl_a_00633}
}
Kostenlos starten →