EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language

Batyr Arystanbekov, Akylbek Maxutov, Aspandiyar Nurimanov, Huseyin Atakan Varol

Big Data and Cognitive Computing · 15.06.2026 · 10 (6) · 192

DOI: 10.3390/bdcc10060192 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 15.06.2026. Verfasst von Batyr Arystanbekov und 3 weiteren Autor:innen. Die Arbeit nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances in multi-modal large language models have enabled systems to jointly process text, images, and audio. However, these developments have primarily benefited high-resource languages, leaving many low-resource communities underserved. In response, we introduce Qolda-AVL, a compact five-billion-parameter audio–vision–language model tailored for Kazakh. Qolda-AVL extends our previous Qolda vision–language model by adding a dedicated audio perception branch while maintaining strong visual and linguistic performance. Built on the Qwen3-VL-Thinking backbone, we incorporate Audio DeepStack, which transfers features from three intermediate Whisper encoder layers into the first three layers of the language model using dedicated projections and residual connections. The model is trained through a four-stage pipeline: adapting the Whisper encoder and language model to Kazakh, aligning the new audio branch to the language backbone, and jointly fine-tuning all modules on chain-of-thought reasoning tasks across audio, image, and text. All audio, vision, and language capabilities are evaluated using the model’s native reasoning mode, and a chain-of-thought trace is generated before each final answer during the performance assessment. To facilitate further research, we open-source the model along with the adapted Kazakh versions of four audio benchmarks, covering spoken attribute reasoning, spoken mathematical question answering, and audio captioning with question answering.

Förderung

  • Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan

Zitieren

APA

Arystanbekov, B., Maxutov, A., Nurimanov, A., & Varol, H. A. (2026). Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language. Big Data and Cognitive Computing, 10(6), 192. https://doi.org/10.3390/bdcc10060192

Vancouver

Arystanbekov B, Maxutov A, Nurimanov A, Varol HA. Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language. Big Data and Cognitive Computing. 2026;10(6):192. doi:10.3390/bdcc10060192

BibTeX
@article{Arystanbekov2026,
  author = {Arystanbekov, Batyr and Maxutov, Akylbek and Nurimanov, Aspandiyar and Varol, Huseyin Atakan},
  title = {Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language},
  journal = {Big Data and Cognitive Computing},
  year = {2026},
  volume = {10},
  number = {6},
  pages = {192},
  doi = {10.3390/bdcc10060192}
}
Kostenlos starten →