Phoneme-Aware Hierarchical Augmentation and Semantic-Aware SpecAugment for Low-Resource Cantonese Speech Recognition
Sensors · 2025
DOI: 10.3390/s25144288 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Lusheng Zhang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Physics and Electronic Information, Yantai University, Yantai 264005, China. Hauptthemen laut MEDLINE: Speech Recognition Software, Phonetics, Speech. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: This work was supported in part by Yantai City 2023 School-Land Integration Development Project Fund under Grant 2323013-2023XDRH001,the Science and Technology…, This research was funded by Chinese National Nature Science Foundationand the Natural Science Foundation of Shandong Province, National Natural Science Foundation of China.
Abstract (Originalsprache)
Cantonese Automatic Speech Recognition (ASR) is hindered by tonal complexity, acoustic diversity, and a lack of labelled data. This study proposes a phoneme-aware hierarchical augmentation framework that enhances performance without additional annotation. A Phoneme Substitution Matrix (PSM), built from Montreal Forced Aligner alignments and Tacotron-2 synthesis, injects adversarial phoneme variants into both transcripts and their aligned audio segments, enlarging pronunciation diversity. Concurrently, a semantic-aware SpecAugment scheme exploits wav2vec 2.0 attention heat maps and keyword boundaries to adaptively mask informative time–frequency regions; a reinforcement-learning controller tunes the masking schedule online, forcing the model to rely on a wider context. On the Common Voice Cantonese 50 h subset, the combined strategy reduces the character error rate (CER) from 26.17% to 16.88% with wav2vec 2.0 and from 38.83% to 23.55% with Zipformer. At 100 h, the CER further drops to 4.27% and 2.32%, yielding relative gains of 32–44%. Ablation studies confirm that phoneme-level and masking components provide complementary benefits. The framework offers a practical, model-independent path toward accurate ASR for Cantonese and other low-resource tonal languages. This paper presents an intelligent sensing-oriented modeling framework for speech signals, which is suitable for deployment on edge or embedded systems to process input from audio sensors (e.g., microphones) and shows promising potential for voice-interactive terminal applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Physics and Electronic Information, Yantai University, Yantai 264005, China.
- Shandong Data Open Innovation Application Laboratory of Smart Grid Advanced Technology, Yantai University, Yantai 264005, China.
Förderung
- This work was supported in part by Yantai City 2023 School-Land Integration Development Project Fund under Grant 2323013-2023XDRH001,the Science and Technology…
- This research was funded by Chinese National Nature Science Foundationand the Natural Science Foundation of Shandong Province
- National Natural Science Foundation of China
- Natural Science Foundation of Shandong Province
- Yantai City 2023 School-Land Integration Development Project Fund
- Science and Technology-Based Small and Medium-Sized Enterprise Innovation Capacity Enhancement Project of Shandong Province
Zitieren
APA
Zhang, L., Wu, S., & Wang, Z. (2025). Phoneme-Aware Hierarchical Augmentation and Semantic-Aware SpecAugment for Low-Resource Cantonese Speech Recognition. Sensors. https://doi.org/10.3390/s25144288
Vancouver
Zhang L, Wu S, Wang Z. Phoneme-Aware Hierarchical Augmentation and Semantic-Aware SpecAugment for Low-Resource Cantonese Speech Recognition. Sensors. 2025. doi:10.3390/s25144288
BibTeX
@article{Zhang2025,
author = {Zhang, Lusheng and Wu, Shie and Wang, Zhongxun},
title = {Phoneme-Aware Hierarchical Augmentation and Semantic-Aware SpecAugment for Low-Resource Cantonese Speech Recognition},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25144288}
}
EUVIMED