Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages
PeerJ Computer Science · 29.03.2024 · 10 · e1974
DOI: 10.7717/peerj-cs.1974 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 29.03.2024. Verfasst von Atabay Ziyaden und 4 weiteren Autor:innen. Beteiligte Einrichtung: Kazakh-British Technical University, Almaty, Kazakhstan. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Ministry of Education and Sciences of the Republic of Kazakhstan.
Abstract (Originalsprache)
Background
In the domain of natural language processing (NLP), the development and success of advanced language models are predominantly anchored in the richness of available linguistic resources. Languages such as Azerbaijani, which is classified as a low-resource, often face challenges arising from limited labeled datasets, consequently hindering effective model training. Methodology The primary objective of this study was to enhance the effectiveness and generalization capabilities of news text classification models using text augmentation techniques. In this study, we solve the problem of working with low-resource languages using translations using the Facebook mBart50 model, as well as the Google Translate API and a combination of mBart50 and Google Translate thus expanding the capabilities when working with text.
Results
The experimental outcomes reveal a promising uptick in classification performance when models are trained on the augmented dataset compared with their counterparts using the original data. This investigation underscores the immense potential of combined data augmentation strategies to bolster the NLP capabilities of underrepresented languages. As a result of our research, we have published our labeled text classification dataset and pre-trained RoBERTa model for the Azerbaijani language.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Kazakh-British Technical University, Almaty, Kazakhstan.
- Institute of Information and Computational Technologies, Almaty, Kazakhstan.
- Nazarbayev University, Astana, Kazakhstan.
- School of Information Technologies and Engineering, ADA University, Baku, Azerbaijan.
ORCID-Profile: Amir Yelenov
Förderung
- Ministry of Education and Sciences of the Republic of Kazakhstan
Zitieren
APA
Ziyaden, A., Yelenov, A., Hajiyev, F., Rustamov, S., & Pak, A. (2024). Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages. PeerJ Computer Science, 10, e1974. https://doi.org/10.7717/peerj-cs.1974
Vancouver
Ziyaden A, Yelenov A, Hajiyev F, Rustamov S, Pak A. Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages. PeerJ Computer Science. 2024;10:e1974. doi:10.7717/peerj-cs.1974
BibTeX
@article{Ziyaden2024,
author = {Ziyaden, Atabay and Yelenov, Amir and Hajiyev, Fuad and Rustamov, Samir and Pak, Alexandr},
title = {Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages},
journal = {PeerJ Computer Science},
year = {2024},
volume = {10},
pages = {e1974},
doi = {10.7717/peerj-cs.1974}
}
EUVIMED