EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results

Aidana Karibayeva, Oleg Myssov, Balzhan Abduali, Dina Amirova, Diana Rakhimova

Computers · 01.11.2026 · 15 (11) · 775

DOI: 10.3390/computers15110775 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers, online erschienen am 01.11.2026. Verfasst von Aidana Karibayeva und 4 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and Higher Education of the Republic of Kazakhstan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Speech-to-speech translation (S2ST) among Turkic languages is rarely studied due to the scarcity of parallel speech corpora and the complex morphology. In this work, we develop low-resource S2ST systems translating from Kazakh to Turkish, Tatar, and Uzbek, and examine which architectural choices proved ineffective and why. Three architectures were trained on a synthetic parallel speech corpus: (i) a scratch-trained, morphology-aware Conformer encoder with a morpheme-level connectionist temporal classification (CTC) loss, a vector-quantized (VQ) codebook, and a semantic anchor; (ii) a pretrained Whisper encoder with a text decoder conditioned on the target language by feature-wise linear modulation (FiLM); and (iii) a text-mediated two-pass model that predicts Turkic speech units from the decoder states and converts them to speech with a Turkic unit vocoder. Text output was scored with BLEU and speech output with ASR-BLEU, that is, BLEU on automatic transcripts. The scratch-trained model and about ten variants scored ASR-BLEU 0. The Whisper-based speech-to-text model reached 19.49 BLEU on synthetic test speech and 3.56 on natural speech. The final speech-to-speech model reached 3.25 ASR-BLEU on about 1500 synthetic test utterances per language, 5.1 times the English-unit baseline, and 0.86 ASR-BLEU on natural speech. These scores show feasibility, not usable quality.

Förderung

  • Ministry of Science and Higher Education of the Republic of Kazakhstan

Zitieren

APA

Karibayeva, A., Myssov, O., Abduali, B., Amirova, D., & Rakhimova, D. (2026). Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results. Computers, 15(11), 775. https://doi.org/10.3390/computers15110775

Vancouver

Karibayeva A, Myssov O, Abduali B, Amirova D, Rakhimova D. Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results. Computers. 2026;15(11):775. doi:10.3390/computers15110775

BibTeX
@article{Karibayeva2026,
  author = {Karibayeva, Aidana and Myssov, Oleg and Abduali, Balzhan and Amirova, Dina and Rakhimova, Diana},
  title = {Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {11},
  pages = {775},
  doi = {10.3390/computers15110775}
}
Kostenlos starten →