EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA3, and ReFT

Steve Nwaiwu

Frontiers in Big Data · 02.12.2025 · 8

DOI: 10.3389/fdata.2025.1677331 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Big Data, online erschienen am 02.12.2025. Verfasst von Steve Nwaiwu. Beteiligte Einrichtung: Data and Information Science, Faculty of Science and Technology, Rajamangala University of Technology, Pathum Thani, Thailand. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 18 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The successful application of large-scale transformer models in Natural Language Processing (NLP) is often hindered by the substantial computational cost and data requirements of full fine-tuning. This challenge is particularly acute in low-resource settings, where standard fine-tuning can lead to catastrophic overfitting and model collapse. To address this, Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a promising solution. However, a direct comparative analysis of their trade-offs under unified low-resource conditions is lacking. This study provides a rigorous empirical evaluation of three prominent PEFT methods: Low-Rank Adaptation (LoRA), Infused Adapter by Inhibiting and Amplifying Inner Activations (IA 3 ), and a Representation Fine-Tuning (ReFT) strategy. Using a DistilBERT base model on low-resource versions of the AG News and Amazon Reviews datasets, the present work compares these methods against a full fine-tuning baseline across accuracy, F1 score, trainable parameters, and GPU memory usage. The findings reveal that while all PEFT methods dramatically outperform the baseline, LoRA consistently achieves the highest F1 scores (0.909 on Amazon Reviews). Critically, ReFT delivers nearly identical performance (~98% of LoRA's F1 score) while training only ~3% of the parameters, establishing it as the most efficient method. This research demonstrates that PEFT is not merely an efficiency optimization, but a necessary tool for robust generalization in data-scarce environments, providing practitioners with a clear guide to navigate the performance—efficiency trade-off. By unifying these evaluations under controlled conditions, this study advances beyond fragmented prior research and offers a systematic framework for selecting PEFT strategies.

Schlagwörter

Stichwörter der Autor:innen

deep learningmachine learningmulti-modal learningsingle modaltext processing

Autor:innen und Institutionen

  • Data and Information Science, Faculty of Science and Technology, Rajamangala University of Technology, Pathum Thani, Thailand.

Zitieren

APA

Nwaiwu, S. (2025). Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA3, and ReFT. Frontiers in Big Data, 8. https://doi.org/10.3389/fdata.2025.1677331

Vancouver

Nwaiwu S. Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA3, and ReFT. Frontiers in Big Data. 2025;8. doi:10.3389/fdata.2025.1677331

BibTeX
@article{Nwaiwu2025,
  author = {Nwaiwu, Steve},
  title = {Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA3, and ReFT},
  journal = {Frontiers in Big Data},
  year = {2025},
  volume = {8},
  doi = {10.3389/fdata.2025.1677331}
}
Kostenlos starten →