Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa
Jurnal Ilmiah Teknologi dan Rekayasa · 03.08.2026 · 31 (2) · 110-127
DOI: 10.35760/tr.2026.v31i2.152 ↗Kurz gefasst
Fachartikel in der Zeitschrift Jurnal Ilmiah Teknologi dan Rekayasa, online erschienen am 03.08.2026. Verfasst von Rizka Oktaviani und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Visual literacy is the ability to understand and interpret information from visual representations, which involves linguistic processing. In the context of elementary education, this ability is crucial for helping students associate visual objects with bilingual vocabulary. However, most image captioning research remains monolingual, few studies accommodate Indonesian, and none have integrated audio representations into a single multimodal learning system. This study develops a bilingual image captioning system based on BLIP-2 and MarianMT, integrated with Text-to-Speech (TTS) within a FastAPI-based web application. English captions are generated using a pretrained BLIP-2 model, then translated into Indonesian using MarianMT, and converted to audio using gTTS. Additionally, a QLoRA fine-tuning experiment was conducted to compare model performance. The dataset consists of 6400 animal images relevant to the elementary school learning context, with caption quality evaluated using the METEOR metric. The results show that the pretrained BLIP-2 model delivers relatively stable performance with a METEOR score of 0.3765 for English and 0.3295 for Indonesian. These scores indicate that the generated captions are semantically relevant, although the overall performance is still relatively moderate compared to advanced image captioning systems. Functional testing of the prototype involving five elementary school students showed that the system is capable of generating bilingual captions and audio in real time and is easy to use. This multimodal integration supports the visual–verbal association process and has the potential to enrich students’ bilingual vocabulary, although no controlled experimental testing has yet been conducted to quantitatively measure improvements in literacy.
Zitieren
APA
Oktaviani, R., & Fauziah, F. (2026). Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa. Jurnal Ilmiah Teknologi dan Rekayasa, 31(2), 110-127. https://doi.org/10.35760/tr.2026.v31i2.152
Vancouver
Oktaviani R, Fauziah F. Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa. Jurnal Ilmiah Teknologi dan Rekayasa. 2026;31(2):110-127. doi:10.35760/tr.2026.v31i2.152
BibTeX
@article{Oktaviani2026,
author = {Oktaviani, Rizka and Fauziah, Fauziah},
title = {Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa},
journal = {Jurnal Ilmiah Teknologi dan Rekayasa},
year = {2026},
volume = {31},
number = {2},
pages = {110--127},
doi = {10.35760/tr.2026.v31i2.152}
}
EUVIMED