EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification

Olesya Razuvayevskaya iD, Ben Wu, João A Leite, Freddy Heppell iD, Ivan Srba, Carolina Scarton, Kalina Bontcheva, Xingyi Song

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0301738 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS ONE, 2024. Verfasst von Olesya Razuvayevskaya und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom. Hauptthemen laut MEDLINE: Multilingualism. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Innovate UK, European Digital Media Observatory, SoBigData++: European Integrated Infrastructure for Social Mining and BigData Analytics.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Adapters and Low-Rank Adaptation (LoRA) are parameter-efficient fine-tuning techniques designed to make the training of language models more efficient. Previous results demonstrated that these methods can even improve performance on some classification tasks. This paper complements existing research by investigating how these techniques influence classification performance and computation costs compared to full fine-tuning. We focus specifically on multilingual text classification tasks (genre, framing, and persuasion techniques detection; with different input lengths, number of predicted classes and classification difficulty), some of which have limited training data. In addition, we conduct in-depth analyses of their efficacy across different training scenarios (training on the original multilingual data; on the translations into English; and on a subset of English-only data) and different languages. Our findings provide valuable insights into the applicability of parameter-efficient fine-tuning techniques, particularly for multilabel classification and non-parallel multilingual tasks which are aimed at analysing input texts of varying length.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

MultilingualismHumansLanguageAlgorithms

Autor:innen und Institutionen

  • Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom.
  • Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia.

ORCID-Profile: Olesya Razuvayevskaya, Freddy Heppell

Förderung

  • Innovate UK
  • European Digital Media Observatory
  • SoBigData++: European Integrated Infrastructure for Social Mining and BigData Analytics
  • University of Sheffield Faculty of Engineering PGR Prize Scholarship.
  • University of Sheffield Engineering & Physical Sciences Research Council (EPSRC) Doctoral Training Partnership Scholarship.

Zitieren

APA

Razuvayevskaya, O., Wu, B., Leite, J. A., Heppell, F., Srba, I., Scarton, C., Bontcheva, K., & Song, X. (2024). Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification. PLOS ONE. https://doi.org/10.1371/journal.pone.0301738

Vancouver

Razuvayevskaya O, Wu B, Leite JA, Heppell F, Srba I, Scarton C, et al. Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification. PLOS ONE. 2024. doi:10.1371/journal.pone.0301738

BibTeX
@article{Razuvayevskaya2024,
  author = {Razuvayevskaya, Olesya and Wu, Ben and Leite, João A and Heppell, Freddy and Srba, Ivan and Scarton, Carolina and Bontcheva, Kalina and Song, Xingyi},
  title = {Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0301738}
}
Kostenlos starten →