Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results
Computers · 01.11.2026 · 15 (11) · 775
DOI: 10.3390/computers15110775 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computers, online erschienen am 01.11.2026. Verfasst von Aidana Karibayeva und 4 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and Higher Education of the Republic of Kazakhstan.
Abstract (Originalsprache)
Speech-to-speech translation (S2ST) among Turkic languages is rarely studied due to the scarcity of parallel speech corpora and the complex morphology. In this work, we develop low-resource S2ST systems translating from Kazakh to Turkish, Tatar, and Uzbek, and examine which architectural choices proved ineffective and why. Three architectures were trained on a synthetic parallel speech corpus: (i) a scratch-trained, morphology-aware Conformer encoder with a morpheme-level connectionist temporal classification (CTC) loss, a vector-quantized (VQ) codebook, and a semantic anchor; (ii) a pretrained Whisper encoder with a text decoder conditioned on the target language by feature-wise linear modulation (FiLM); and (iii) a text-mediated two-pass model that predicts Turkic speech units from the decoder states and converts them to speech with a Turkic unit vocoder. Text output was scored with BLEU and speech output with ASR-BLEU, that is, BLEU on automatic transcripts. The scratch-trained model and about ten variants scored ASR-BLEU 0. The Whisper-based speech-to-text model reached 19.49 BLEU on synthetic test speech and 3.56 on natural speech. The final speech-to-speech model reached 3.25 ASR-BLEU on about 1500 synthetic test utterances per language, 5.1 times the English-unit baseline, and 0.86 ASR-BLEU on natural speech. These scores show feasibility, not usable quality.
Förderung
- Ministry of Science and Higher Education of the Republic of Kazakhstan
Zitieren
APA
Karibayeva, A., Myssov, O., Abduali, B., Amirova, D., & Rakhimova, D. (2026). Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results. Computers, 15(11), 775. https://doi.org/10.3390/computers15110775
Vancouver
Karibayeva A, Myssov O, Abduali B, Amirova D, Rakhimova D. Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results. Computers. 2026;15(11):775. doi:10.3390/computers15110775
BibTeX
@article{Karibayeva2026,
author = {Karibayeva, Aidana and Myssov, Oleg and Abduali, Balzhan and Amirova, Dina and Rakhimova, Diana},
title = {Speech-to-Speech Translation from Kazakh into Turkic Languages: Pretrained Speech Encoders, Turkic Speech Units, and Lessons from Negative Results},
journal = {Computers},
year = {2026},
volume = {15},
number = {11},
pages = {775},
doi = {10.3390/computers15110775}
}
EUVIMED