EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-SA 4.0

Acoustic and Tonal Modeling of the tpuri Language through a Multi-Modular Hybrid Approach

Jules Paulin Bayang Souloukna, Patrick Nounamou Dabou, Paul Dayang, Kolyang

EAI Endorsed Transactions on Intelligent Systems and Machine Learning Applications · 2026

DOI: 10.4108/eetismla.11285 ↗

Kurz gefasst

Fachartikel in der Zeitschrift EAI Endorsed Transactions on Intelligent Systems and Machine Learning Applications, 2026. Verfasst von Jules Paulin Bayang Souloukna und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automatic speech recognition (ASR) for tonal low-resource languages remains challenging due to the scarcity of labelled data and the need to model complex prosodic systems. This paper presents a hybrid multimodular ASR architecture for tpuri, a Mboum-Day Niger-Congo language spoken in Cameroon and Chad that exhibits contrastive lexical tone, vowel length and nasalisation. The system combines a self-supervised Wav2Vec 2.0 acoustic encoder with a tonal processing module based on YIN pitch estimation and STFTderived spectral features, and an adaptive fusion mechanism that integrates acoustic and tonal representations before decoding. We pretrain the acoustic encoder on 45 hours of read and spontaneous speech and finetune it on 19h35 of scripted speech. On the scripted test set, our best configuration reaches a word error rate (WER) of 10.4%, a phone error rate (PER) of 8.7% and a tone error rate (TER) of 6.1%. Ablation experiments show that removing the tonal module (+1.5 WER, +2.3 TER) or self-supervised pretraining (+3.4 WER) substantially degrades performance, while adaptive fusion and tone-aware data augmentation yield smaller but consistent gains. A fine-grained error analysis across tonal, grammatical, syllabic and morphological dimensions indicates that the architecture is particularly effective at modelling lexical tone and clause-level syntax, but still struggles with complex syllable structures and rich morphology. Overall, the results demonstrate that competitive ASR is attainable for under-resourced tonal languages such as tpuri by tightly coupling self-supervised acoustic modelling with explicit tonal representations, and provide a reusable blueprint for extending ASR to other Niger-Congo languages.

Zitieren

APA

Souloukna, J. P. B., Dabou, P. N., Dayang, P., & Kolyang (2026). Acoustic and Tonal Modeling of the tpuri Language through a Multi-Modular Hybrid Approach. EAI Endorsed Transactions on Intelligent Systems and Machine Learning Applications. https://doi.org/10.4108/eetismla.11285

Vancouver

Souloukna JPB, Dabou PN, Dayang P, Kolyang. Acoustic and Tonal Modeling of the tpuri Language through a Multi-Modular Hybrid Approach. EAI Endorsed Transactions on Intelligent Systems and Machine Learning Applications. 2026. doi:10.4108/eetismla.11285

BibTeX
@article{Souloukna2026,
  author = {Souloukna, Jules Paulin Bayang and Dabou, Patrick Nounamou and Dayang, Paul and Kolyang},
  title = {Acoustic and Tonal Modeling of the tpuri Language through a Multi-Modular Hybrid Approach},
  journal = {EAI Endorsed Transactions on Intelligent Systems and Machine Learning Applications},
  year = {2026},
  doi = {10.4108/eetismla.11285}
}
Kostenlos starten →