Low-Resource Speech Recognition by Fine-Tuning Whisper with Optuna-LoRA
Applied Sciences · 12.12.2025 · 15 (24) · 13090
DOI: 10.3390/app152413090 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 12.12.2025. Verfasst von Huan Wang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Mianyang Science Foundation, the National Natural Science Foundation of China.
Abstract (Originalsprache)
In low-resource speech recognition, the performance of the Whisper model is often limited by the size of the available training data. To address this challenge, this paper proposes a training optimization method for the Whisper model that integrates Low-Rank Adaptation (LoRA) with the Optuna hyperparameter optimization framework. This combined approach enables efficient fine-tuning and enhances model performance. A dual-metric early stopping strategy, based on validation loss and relative word error rate improvement, is introduced to ensure robust convergence during training. Experimental data were collected from three low-resource languages in Xinjiang, China: Uyghur, Kazakh, and Kyrgyz. Compared to baseline LoRA fine-tuning, the proposed optimization method reduces WER by 20.98%, 6.46%, and 8.72%, respectively, across the three languages. The dual-metric early stopping strategy effectively shortens optimization time while preventing overfitting. Overall, these results demonstrate that the proposed method significantly reduces both WER and computational costs, making it highly effective for low-resource speech recognition tasks.
Förderung
- Mianyang Science Foundation
- the National Natural Science Foundation of China
Zitieren
APA
Wang, H., Bin, J., Gou, C., Yang, L., Hou, B., & Qin, M. (2025). Low-Resource Speech Recognition by Fine-Tuning Whisper with Optuna-LoRA. Applied Sciences, 15(24), 13090. https://doi.org/10.3390/app152413090
Vancouver
Wang H, Bin J, Gou C, Yang L, Hou B, Qin M. Low-Resource Speech Recognition by Fine-Tuning Whisper with Optuna-LoRA. Applied Sciences. 2025;15(24):13090. doi:10.3390/app152413090
BibTeX
@article{Wang2025,
author = {Wang, Huan and Bin, Jie and Gou, Chunyan and Yang, Lian and Hou, Baolin and Qin, Mingwei},
title = {Low-Resource Speech Recognition by Fine-Tuning Whisper with Optuna-LoRA},
journal = {Applied Sciences},
year = {2025},
volume = {15},
number = {24},
pages = {13090},
doi = {10.3390/app152413090}
}
EUVIMED