EUVIMEDEUVIMED entdecken
Crossref

Speech synthesis via vocal tract shape extracted from real-time MRI

Sumisa Shinkai, Mako Wakita, Hironori Takemoto, Hiroyuki Hirai, Kikuo Maekawa

The Journal of the Acoustical Society of America · 01.10.2025 · 158 (4_Supplement) · A464-A464

DOI: 10.1121/10.0041537 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Journal of the Acoustical Society of America, erschienen am 01.10.2025. Verfasst von Sumisa Shinkai und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We propose an integrated model that synthesizes speech by predicting vocal tract shapes from linguistic features as an intermediate representation, followed by the prediction of acoustic features, using articulatory contours extracted from real-time MRI (rtMRI). Previously, we constructed Model 1, which predicts vocal tract shapes from linguistic features, and Model 2, which predicts acoustic features from the shapes captured by rtMRI. However, when these models were cascaded (Model 1 + 2), the synthesized speech showed substantial quality degradation compared to reference speech re-synthesized using the WORLD vocoder. To overcome this, we integrated both tasks into a single model. Objective evaluation showed that the integrated model predicted vocal tract shapes with accuracy comparable to Model 1 + 2, achieving a root mean square error (RMSE) of 1.14 pixels. For acoustic features, it achieved equal or significantly better accuracy for all features except mel-generalized cepstrum (mgc). A Degradation Mean Opinion Score (DMOS) test with 16 participants showed that the integrated model received a significantly higher score (3.31) than Model 1 + 2. We also analyzed the vocal tract regions contributing to F0 prediction, which was possible from mid-sagittal MRI images. These results demonstrate the effectiveness of the proposed model in producing higher-quality speech than the conventional cascade approach.

Zitieren

APA

Shinkai, S., Wakita, M., Takemoto, H., Hirai, H., & Maekawa, K. (2025). Speech synthesis via vocal tract shape extracted from real-time MRI. The Journal of the Acoustical Society of America, 158(4_Supplement), A464-A464. https://doi.org/10.1121/10.0041537

Vancouver

Shinkai S, Wakita M, Takemoto H, Hirai H, Maekawa K. Speech synthesis via vocal tract shape extracted from real-time MRI. The Journal of the Acoustical Society of America. 2025;158(4_Supplement):A464-A464. doi:10.1121/10.0041537

BibTeX
@article{Shinkai2025,
  author = {Shinkai, Sumisa and Wakita, Mako and Takemoto, Hironori and Hirai, Hiroyuki and Maekawa, Kikuo},
  title = {Speech synthesis via vocal tract shape extracted from real-time MRI},
  journal = {The Journal of the Acoustical Society of America},
  year = {2025},
  volume = {158},
  number = {4_Supplement},
  pages = {A464--A464},
  doi = {10.1121/10.0041537}
}
Kostenlos starten →