EUVIMEDEUVIMED entdecken
Crossref

Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation

Julian Spravil, Sebastian Houben, Sven Behnke

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25599-25607

DOI: 10.1609/aaai.v40i30.39756 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Julian Spravil und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Cross-lingual, cross-task transfer is challenged by task-specific data scarcity, which becomes more severe as language support grows and is further amplified in vision-language models (VLMs). We investigate multilingual generalization in encoder-decoder transformer VLMs to enable zero-shot image captioning in languages encountered only in the translation task. In this setting, the encoder must learn to generate generalizable, task-aware latent vision representations to instruct the decoder via inserted cross-attention layers. To analyze scaling behavior, we train Florence-2 based and Gemma-2 based models (0.4B to 11.2B parameters) on a synthetic dataset using varying compute budgets. While all languages in the dataset have image-aligned translations, only a subset of them include image captions. Notably, we show that captioning can emerge using a language prefix, even when this language only appears in the translation task. We find that indirect learning of unseen task-language pairs adheres to scaling laws that are governed by the multilinguality of the model, model size, and seen training samples. Finally, we demonstrate that the scaling laws extend to downstream tasks, achieving competitive performance through fine-tuning in multimodal machine translation (Multi30K, CoMMuTE), lexical disambiguation (CoMMuTE), and image captioning (Multi30K, XM3600, COCO Karpathy).

Zitieren

APA

Spravil, J., Houben, S., & Behnke, S. (2026). Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25599-25607. https://doi.org/10.1609/aaai.v40i30.39756

Vancouver

Spravil J, Houben S, Behnke S. Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25599-25607. doi:10.1609/aaai.v40i30.39756

BibTeX
@article{Spravil2026,
  author = {Spravil, Julian and Houben, Sebastian and Behnke, Sven},
  title = {Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {30},
  pages = {25599--25607},
  doi = {10.1609/aaai.v40i30.39756}
}
Kostenlos starten →