EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task

Denis Shitov iD, Elena Pirogova iD, Tadeusz A Wysocki iD, Margaret Lech iD

Sensors · 2023

DOI: 10.3390/s23073437 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von Denis Shitov und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Engineering, RMIT University, Melbourne 3000, Australia. Hauptthemen laut MEDLINE: Speech Acoustics, Phonetics. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Articulatory synthesis is one of the approaches used for modeling human speech production. In this study, we propose a model-based algorithm for learning the policy to control the vocal tract of the articulatory synthesizer in a vowel-to-vowel imitation task. Our method does not require external training data, since the policy is learned through interactions with the vocal tract model. To improve the sample efficiency of the learning, we trained the model of speech production dynamics simultaneously with the policy. The policy was trained in a supervised way using predictions of the model of speech production dynamics. To stabilize the training, early stopping was incorporated into the algorithm. Additionally, we extracted acoustic features using an acoustic word embedding (AWE) model. This model was trained to discriminate between different words and to enable compact encoding of acoustics while preserving contextual information of the input. Our preliminary experiments showed that introducing this AWE model was crucial to guide the policy toward a near-optimal solution. The acoustic embeddings, obtained using the proposed approach, were revealed to be useful when applied as inputs to the policy and the model of speech production dynamics.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Speech AcousticsPhoneticsHumansImitative BehaviorSpeechLearningSpeech Production Measurement

Stichwörter der Autor:innen

reinforcement learningspeech modelingspeech synthesis

Autor:innen und Institutionen

  • School of Engineering, RMIT University, Melbourne 3000, Australia.
  • Department of Electrical and Computer Engineering, University of Nebraska-Lincoln, Lincoln, NE 68588, USA.
  • Faculty of Telecommunications, Computer Science and Electrical Engineering, Bydgoszcz University of Science and Technology, 85-796 Bydgoszcz, Poland.

ORCID-Profile: Denis Shitov, Elena Pirogova, Tadeusz A Wysocki, Margaret Lech

Zitieren

APA

Shitov, D., Pirogova, E., Wysocki, T. A., & Lech, M. (2023). Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task. Sensors. https://doi.org/10.3390/s23073437

Vancouver

Shitov D, Pirogova E, Wysocki TA, Lech M. Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task. Sensors. 2023. doi:10.3390/s23073437

BibTeX
@article{Shitov2023,
  author = {Shitov, Denis and Pirogova, Elena and Wysocki, Tadeusz A and Lech, Margaret},
  title = {Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23073437}
}
Kostenlos starten →