Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language
Big Data and Cognitive Computing · 15.06.2026 · 10 (6) · 192
DOI: 10.3390/bdcc10060192 ↗Kurz gefasst
Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 15.06.2026. Verfasst von Batyr Arystanbekov und 3 weiteren Autor:innen. Die Arbeit nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan.
Abstract (Originalsprache)
Recent advances in multi-modal large language models have enabled systems to jointly process text, images, and audio. However, these developments have primarily benefited high-resource languages, leaving many low-resource communities underserved. In response, we introduce Qolda-AVL, a compact five-billion-parameter audio–vision–language model tailored for Kazakh. Qolda-AVL extends our previous Qolda vision–language model by adding a dedicated audio perception branch while maintaining strong visual and linguistic performance. Built on the Qwen3-VL-Thinking backbone, we incorporate Audio DeepStack, which transfers features from three intermediate Whisper encoder layers into the first three layers of the language model using dedicated projections and residual connections. The model is trained through a four-stage pipeline: adapting the Whisper encoder and language model to Kazakh, aligning the new audio branch to the language backbone, and jointly fine-tuning all modules on chain-of-thought reasoning tasks across audio, image, and text. All audio, vision, and language capabilities are evaluated using the model’s native reasoning mode, and a chain-of-thought trace is generated before each final answer during the performance assessment. To facilitate further research, we open-source the model along with the adapted Kazakh versions of four audio benchmarks, covering spoken attribute reasoning, spoken mathematical question answering, and audio captioning with question answering.
Förderung
- Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan
Zitieren
APA
Arystanbekov, B., Maxutov, A., Nurimanov, A., & Varol, H. A. (2026). Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language. Big Data and Cognitive Computing, 10(6), 192. https://doi.org/10.3390/bdcc10060192
Vancouver
Arystanbekov B, Maxutov A, Nurimanov A, Varol HA. Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language. Big Data and Cognitive Computing. 2026;10(6):192. doi:10.3390/bdcc10060192
BibTeX
@article{Arystanbekov2026,
author = {Arystanbekov, Batyr and Maxutov, Akylbek and Nurimanov, Aspandiyar and Varol, Huseyin Atakan},
title = {Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language},
journal = {Big Data and Cognitive Computing},
year = {2026},
volume = {10},
number = {6},
pages = {192},
doi = {10.3390/bdcc10060192}
}
EUVIMED