EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages

Atabay Ziyaden, Amir Yelenov iD, Fuad Hajiyev, Samir Rustamov, Alexandr Pak

PeerJ Computer Science · 29.03.2024 · 10 · e1974

DOI: 10.7717/peerj-cs.1974 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 29.03.2024. Verfasst von Atabay Ziyaden und 4 weiteren Autor:innen. Beteiligte Einrichtung: Kazakh-British Technical University, Almaty, Kazakhstan. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Ministry of Education and Sciences of the Republic of Kazakhstan.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

In the domain of natural language processing (NLP), the development and success of advanced language models are predominantly anchored in the richness of available linguistic resources. Languages such as Azerbaijani, which is classified as a low-resource, often face challenges arising from limited labeled datasets, consequently hindering effective model training. Methodology The primary objective of this study was to enhance the effectiveness and generalization capabilities of news text classification models using text augmentation techniques. In this study, we solve the problem of working with low-resource languages using translations using the Facebook mBart50 model, as well as the Google Translate API and a combination of mBart50 and Google Translate thus expanding the capabilities when working with text.

Results

The experimental outcomes reveal a promising uptick in classification performance when models are trained on the augmented dataset compared with their counterparts using the original data. This investigation underscores the immense potential of combined data augmentation strategies to bolster the NLP capabilities of underrepresented languages. As a result of our research, we have published our labeled text classification dataset and pre-trained RoBERTa model for the Azerbaijani language.

Schlagwörter

Stichwörter der Autor:innen

Azerbaijani languageDeep learningLow-resource languageMachine learningNatural language processingText augmentationText classification

Autor:innen und Institutionen

  • Kazakh-British Technical University, Almaty, Kazakhstan.
  • Institute of Information and Computational Technologies, Almaty, Kazakhstan.
  • Nazarbayev University, Astana, Kazakhstan.
  • School of Information Technologies and Engineering, ADA University, Baku, Azerbaijan.

ORCID-Profile: Amir Yelenov

Förderung

  • Ministry of Education and Sciences of the Republic of Kazakhstan

Zitieren

APA

Ziyaden, A., Yelenov, A., Hajiyev, F., Rustamov, S., & Pak, A. (2024). Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages. PeerJ Computer Science, 10, e1974. https://doi.org/10.7717/peerj-cs.1974

Vancouver

Ziyaden A, Yelenov A, Hajiyev F, Rustamov S, Pak A. Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages. PeerJ Computer Science. 2024;10:e1974. doi:10.7717/peerj-cs.1974

BibTeX
@article{Ziyaden2024,
  author = {Ziyaden, Atabay and Yelenov, Amir and Hajiyev, Fuad and Rustamov, Samir and Pak, Alexandr},
  title = {Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages},
  journal = {PeerJ Computer Science},
  year = {2024},
  volume = {10},
  pages = {e1974},
  doi = {10.7717/peerj-cs.1974}
}
Kostenlos starten →