Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0301738 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift PLOS ONE, 2024. Verfasst von Olesya Razuvayevskaya und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom. Hauptthemen laut MEDLINE: Multilingualism. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Innovate UK, European Digital Media Observatory, SoBigData++: European Integrated Infrastructure for Social Mining and BigData Analytics.
Abstract (Originalsprache)
Adapters and Low-Rank Adaptation (LoRA) are parameter-efficient fine-tuning techniques designed to make the training of language models more efficient. Previous results demonstrated that these methods can even improve performance on some classification tasks. This paper complements existing research by investigating how these techniques influence classification performance and computation costs compared to full fine-tuning. We focus specifically on multilingual text classification tasks (genre, framing, and persuasion techniques detection; with different input lengths, number of predicted classes and classification difficulty), some of which have limited training data. In addition, we conduct in-depth analyses of their efficacy across different training scenarios (training on the original multilingual data; on the translations into English; and on a subset of English-only data) and different languages. Our findings provide valuable insights into the applicability of parameter-efficient fine-tuning techniques, particularly for multilabel classification and non-parallel multilingual tasks which are aimed at analysing input texts of varying length.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom.
- Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia.
ORCID-Profile: Olesya Razuvayevskaya, Freddy Heppell
Förderung
- Innovate UK
- European Digital Media Observatory
- SoBigData++: European Integrated Infrastructure for Social Mining and BigData Analytics
- University of Sheffield Faculty of Engineering PGR Prize Scholarship.
- University of Sheffield Engineering & Physical Sciences Research Council (EPSRC) Doctoral Training Partnership Scholarship.
Zitieren
APA
Razuvayevskaya, O., Wu, B., Leite, J. A., Heppell, F., Srba, I., Scarton, C., Bontcheva, K., & Song, X. (2024). Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification. PLOS ONE. https://doi.org/10.1371/journal.pone.0301738
Vancouver
Razuvayevskaya O, Wu B, Leite JA, Heppell F, Srba I, Scarton C, et al. Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification. PLOS ONE. 2024. doi:10.1371/journal.pone.0301738
BibTeX
@article{Razuvayevskaya2024,
author = {Razuvayevskaya, Olesya and Wu, Ben and Leite, João A and Heppell, Freddy and Srba, Ivan and Scarton, Carolina and Bontcheva, Kalina and Song, Xingyi},
title = {Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0301738}
}
EUVIMED