EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

LoRA-INT8 Whisper: A Low-Cost Cantonese Speech Recognition Framework for Edge Devices

Lusheng Zhang, Shie Wu, Zhongxun Wang

Sensors · 01.09.2025 · 25 (17) · 5404

DOI: 10.3390/s25175404 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 01.09.2025. Verfasst von Lusheng Zhang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Physics and Electronic Information, Yantai University, Yantai 264005, China. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: This work was supported in part by Yantai City 2023 School-Land Integration Development Project Fund under Grant 2323013-2023XDRH001, he Science and Technology-Based Small and Medium-Sized Enterprise Innovation Capacity Enhancement Project of Shandong Pro: 2323013-2023XDRH001;2023TSGC0823;, This research was funded by Chinese National Nature Science Foundationand the Natural Science Foundation of Shandong Province: 62201491;ZR2021QF097.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To address the triple bottlenecks of data scarcity, oversized models, and slow inference that hinder Cantonese automatic speech recognition (ASR) in low-resource and edge-deployment settings, this study proposes a cost-effective Cantonese ASR system based on LoRA fine-tuning and INT8 quantization. First, Whisper-tiny is parameter-efficiently fine-tuned on the Common Voice zh-HK training set using LoRA with rank = 8. Only 1.6% of the original weights are updated, reducing the character error rate (CER) from 49.5% to 11.1%, a performance close to full fine-tuning (10.3%), while cutting the training memory footprint and computational cost by approximately one order of magnitude. Next, the fine-tuned model is compressed into a 60 MB INT8 checkpoint via dynamic quantization in ONNX Runtime. On a MacBook Pro M1 Max CPU, the quantized model achieves an RTF = 0.20 (offline inference 5 × real-time) and 43% lower latency than the FP16 baseline; on an NVIDIA A10 GPU, it reaches RTF = 0.06, meeting the requirements of high-concurrency cloud services. Ablation studies confirm that the LoRA-INT8 configuration offers the best trade-off among accuracy, speed, and model size. Limitations include the absence of spontaneous-speech noise data, extreme-hardware validation, and adaptive LoRA structure optimization. Future work will incorporate large-scale self-supervised pre-training, tone-aware loss functions, AdaLoRA architecture search, and INT4/NPU quantization, and will establish an mJ/char energy–accuracy curve. The ultimate goal is to achieve CER ≤ 8%, RTF < 0.1, and mJ/char < 1 for low-power real-time Cantonese ASR in practical IoT scenarios.

Schlagwörter

Stichwörter der Autor:innen

Cantonese speech recognitionINT8 quantizationLoRAWhisperedge computingparameter-efficient fine-tuning

Autor:innen und Institutionen

  • School of Physics and Electronic Information, Yantai University, Yantai 264005, China.
  • Shandong Data Open Innovation Application Laboratory of Smart Grid Advanced Technology, Yantai University, Yantai 264005, China.

Förderung

  • This work was supported in part by Yantai City 2023 School-Land Integration Development Project Fund under Grant 2323013-2023XDRH001
  • he Science and Technology-Based Small and Medium-Sized Enterprise Innovation Capacity Enhancement Project of Shandong Pro: 2323013-2023XDRH001;2023TSGC0823;
  • This research was funded by Chinese National Nature Science Foundationand the Natural Science Foundation of Shandong Province: 62201491;ZR2021QF097
  • National Natural Science Foundation of China
  • Natural Science Foundation of Shandong Province
  • Yantai City 2023 School-Land Integration Development Project Fund
  • Science and Technology-Based Small and Medium-Sized Enterprise Innovation Capacity Enhancement Project of Shandong Province

Zitieren

APA

Zhang, L., Wu, S., & Wang, Z. (2025). LoRA-INT8 Whisper: A Low-Cost Cantonese Speech Recognition Framework for Edge Devices. Sensors, 25(17), 5404. https://doi.org/10.3390/s25175404

Vancouver

Zhang L, Wu S, Wang Z. LoRA-INT8 Whisper: A Low-Cost Cantonese Speech Recognition Framework for Edge Devices. Sensors. 2025;25(17):5404. doi:10.3390/s25175404

BibTeX
@article{Zhang2025,
  author = {Zhang, Lusheng and Wu, Shie and Wang, Zhongxun},
  title = {LoRA-INT8 Whisper: A Low-Cost Cantonese Speech Recognition Framework for Edge Devices},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {17},
  pages = {5404},
  doi = {10.3390/s25175404}
}
Kostenlos starten →