Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task
Sensors · 2023
DOI: 10.3390/s23073437 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von Denis Shitov und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Engineering, RMIT University, Melbourne 3000, Australia. Hauptthemen laut MEDLINE: Speech Acoustics, Phonetics. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Articulatory synthesis is one of the approaches used for modeling human speech production. In this study, we propose a model-based algorithm for learning the policy to control the vocal tract of the articulatory synthesizer in a vowel-to-vowel imitation task. Our method does not require external training data, since the policy is learned through interactions with the vocal tract model. To improve the sample efficiency of the learning, we trained the model of speech production dynamics simultaneously with the policy. The policy was trained in a supervised way using predictions of the model of speech production dynamics. To stabilize the training, early stopping was incorporated into the algorithm. Additionally, we extracted acoustic features using an acoustic word embedding (AWE) model. This model was trained to discriminate between different words and to enable compact encoding of acoustics while preserving contextual information of the input. Our preliminary experiments showed that introducing this AWE model was crucial to guide the policy toward a near-optimal solution. The acoustic embeddings, obtained using the proposed approach, were revealed to be useful when applied as inputs to the policy and the model of speech production dynamics.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Engineering, RMIT University, Melbourne 3000, Australia.
- Department of Electrical and Computer Engineering, University of Nebraska-Lincoln, Lincoln, NE 68588, USA.
- Faculty of Telecommunications, Computer Science and Electrical Engineering, Bydgoszcz University of Science and Technology, 85-796 Bydgoszcz, Poland.
ORCID-Profile: Denis Shitov, Elena Pirogova, Tadeusz A Wysocki, Margaret Lech
Zitieren
APA
Shitov, D., Pirogova, E., Wysocki, T. A., & Lech, M. (2023). Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task. Sensors. https://doi.org/10.3390/s23073437
Vancouver
Shitov D, Pirogova E, Wysocki TA, Lech M. Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task. Sensors. 2023. doi:10.3390/s23073437
BibTeX
@article{Shitov2023,
author = {Shitov, Denis and Pirogova, Elena and Wysocki, Tadeusz A and Lech, Margaret},
title = {Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task},
journal = {Sensors},
year = {2023},
doi = {10.3390/s23073437}
}
EUVIMED