From Monologue to Dialogue: A Generative AI Framework for MultiSpeaker, Code-Switched Multilingual Video Dubbing
International Journal of Drug Delivery Technology · 11.06.2026 · 16 (52s)
DOI: 10.25258/ijddt.16.52s.13 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Drug Delivery Technology, online erschienen am 11.06.2026. Verfasst von Badal Singh und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Automated multilingual video dubbing has advanced considerably with generative AI; however, existing systems remain architecturally constrained to single-speaker, monolingual audio streams. Real-world media content including panel discussions, interviews, multilingual films, and social media dialogue routinely involves concurrent speaker activity and intra-utterance language switching, commonly termed code-switching. This paper introduces DialogueDub, an end-to-end generative AI framework that addresses both challenges simultaneously through a tightly coupled pipeline of neural speaker diarization, sub-sentence language identification, dynamic speaker embedding assignment, and identity-preserving text-to-speech synthesis. The proposed system is evaluated on three established benchmarks: the AMI Meeting Corpus for multi-speaker diarization, the SEAME dataset for Mandarin-English code-switching, and the Multilingual LibriSpeech (MLS) corpus for cross-lingual TTS fidelity. Experimental results demonstrate a Diarization Error Rate (DER) of 8.3%, a code-switching boundary detection F1-score of 0.847, a speaker similarity score of 0.912, and a Mean Opinion Score (MOS) of 4.1 on naturalness across target languages including Hindi, Spanish, and Mandarin. The framework outperforms singlespeaker baselines by 34.2% in synchronization accuracy and reduces speaker identity drift by 41.7% over conventional TTS pipelines. These findings establish DialogueDub as a meaningful step toward production-grade, multi-speaker multilingual dubbing systems.
Zitieren
APA
Singh, B., & Sirohi, P. (2026). From Monologue to Dialogue: A Generative AI Framework for MultiSpeaker, Code-Switched Multilingual Video Dubbing. International Journal of Drug Delivery Technology, 16(52s). https://doi.org/10.25258/ijddt.16.52s.13
Vancouver
Singh B, Sirohi P. From Monologue to Dialogue: A Generative AI Framework for MultiSpeaker, Code-Switched Multilingual Video Dubbing. International Journal of Drug Delivery Technology. 2026;16(52s). doi:10.25258/ijddt.16.52s.13
BibTeX
@article{Singh2026,
author = {Singh, Badal and Sirohi, Priyank},
title = {From Monologue to Dialogue: A Generative AI Framework for MultiSpeaker, Code-Switched Multilingual Video Dubbing},
journal = {International Journal of Drug Delivery Technology},
year = {2026},
volume = {16},
number = {52s},
doi = {10.25258/ijddt.16.52s.13}
}
EUVIMED