Automatic Fluency Assessment Method for Spontaneous Speech without Reference Text
Electronics · 09.04.2023 · 12 (8) · 1775
DOI: 10.3390/electronics12081775 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 09.04.2023. Verfasst von Jiajun Liu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation of China, Basic Research Program of Tianshan Talent Plan of Xinjiang, China.
Abstract (Originalsprache)
The automatic fluency assessment of spontaneous speech without reference text is a challenging task that heavily depends on the accuracy of automatic speech recognition (ASR). Considering this scenario, it is necessary to explore an assessment method that combines ASR. This is mainly due to the fact that in addition to acoustic features being essential for assessment, the text features output by ASR may also contain potentially fluency information. However, most existing studies on automatic fluency assessment of spontaneous speech are based solely on audio features, without utilizing textual information, which may lead to a limited understanding of fluency features. To address this, we propose a multimodal automatic speech fluency assessment method that combines ASR output. Specifically, we first explore the relevance of the fluency assessment task to the ASR task and fine-tune the Wav2Vec2.0 model using multi-task learning to jointly optimize the ASR task and fluency assessment task, resulting in both the fluency assessment results and the ASR output. Then, the text features and audio features obtained from the fine-tuned model are fed into the multimodal fluency assessment model, using attention mechanisms to obtain more reliable assessment results. Finally, experiments on the PSCPSF and Speechocean762 dataset suggest that our proposed method performs well in different assessment scenarios.
Förderung
- National Science Foundation of China
- Basic Research Program of Tianshan Talent Plan of Xinjiang, China
Zitieren
APA
Liu, J., Wumaier, A., Fan, C., & Guo, S. (2023). Automatic Fluency Assessment Method for Spontaneous Speech without Reference Text. Electronics, 12(8), 1775. https://doi.org/10.3390/electronics12081775
Vancouver
Liu J, Wumaier A, Fan C, Guo S. Automatic Fluency Assessment Method for Spontaneous Speech without Reference Text. Electronics. 2023;12(8):1775. doi:10.3390/electronics12081775
BibTeX
@article{Liu2023,
author = {Liu, Jiajun and Wumaier, Aishan and Fan, Cong and Guo, Shen},
title = {Automatic Fluency Assessment Method for Spontaneous Speech without Reference Text},
journal = {Electronics},
year = {2023},
volume = {12},
number = {8},
pages = {1775},
doi = {10.3390/electronics12081775}
}
EUVIMED