EUVIMEDUpptäck EUVIMED
Open access · CC BY 4.0

Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh–English Overlapping Speech

Rakhat Meiramov, Tomiris Rakhimzhanova, Adil Taibassarov, Zhanat Makhataeva, Huseyin Atakan Varol

Machine Learning and Knowledge Extraction · 2026-08-14 · 8 (8) · 246

DOI: 10.3390/make8080246 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-08-14. Written by Rakhat Meiramov and 4 co-authors. 40 references. The full text is freely available (open access, CC BY 4.0). Funded by: Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan.

AbstractBibliografiska uppgifterCiteRelated studies

Abstract (original language)

Target-speaker automatic speech recognition (TS-ASR) enables transcription of a specific speaker in multi-talker environments, yet remains largely unexplored for multilingual, low-resource languages. Existing TS-ASR systems predominantly target monolingual English using diarization-based or speaker-embedding approaches, leaving a critical gap for languages such as Kazakh, where code-switching with Russian and English is commonplace. We propose Persona-ASR, a modular two-stage architecture. The first stage is an explicit target-presence gate that verifies whether the enrolled speaker appears in the mixture and emits a token to suppress transcription when the speaker is absent, directly addressing the acoustic-hallucination failure mode of prior systems. The second stage performs enrollment-conditioned recognition: a 192-dimensional ECAPA-TDNN speaker embedding modulates a WavLM-Base-Plus encoder through feature-wise linear modulation (FiLM), while language-specific CTC heads enable joint Kazakh and English decoding without forcing Latin and Cyrillic symbols to compete in a single output space. To evaluate the system, we introduce KazMix3, a Kazakh overlap dataset for TS-ASR training, and PersonaMix, a controlled bilingual benchmark spanning same- and cross-language enrollment across varying interferer counts (1–3) and signal-to-noise ratios (−3 to +3 dB). Persona-ASR outperforms a strong off-the-shelf cascade baseline by 13.3 WER points on English and 24.6 on Kazakh, and matches a published monolingual English baseline. On PersonaMix, speaker conditioning reduces relative word error rate by 40.7% on English and 59.3% on Kazakh mixtures over an unconditioned variant of the same model, and cross-language enrollment (unseen during training) remains effective, increasing average raw WER by only 4.1 points (English) and 2.2 points (Kazakh) relative to same-language enrollment. To our knowledge, Persona-ASR is the first TS-ASR system for the Kazakh language, establishing a foundation for multilingual personalized ASR in low-resource settings.

Funding

  • Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan

Cite

APA

Meiramov, R., Rakhimzhanova, T., Taibassarov, A., Makhataeva, Z., & Varol, H. A. (2026). Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh–English Overlapping Speech. Machine Learning and Knowledge Extraction, 8(8), 246. https://doi.org/10.3390/make8080246

Vancouver

Meiramov R, Rakhimzhanova T, Taibassarov A, Makhataeva Z, Varol HA. Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh–English Overlapping Speech. Machine Learning and Knowledge Extraction. 2026;8(8):246. doi:10.3390/make8080246

BibTeX
@article{Meiramov2026,
  author = {Meiramov, Rakhat and Rakhimzhanova, Tomiris and Taibassarov, Adil and Makhataeva, Zhanat and Varol, Huseyin Atakan},
  title = {Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh–English Overlapping Speech},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {8},
  pages = {246},
  doi = {10.3390/make8080246}
}
Start for free →