EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa

Rizka Oktaviani, Fauziah Fauziah

Jurnal Ilmiah Teknologi dan Rekayasa · 03.08.2026 · 31 (2) · 110-127

DOI: 10.35760/tr.2026.v31i2.152 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Jurnal Ilmiah Teknologi dan Rekayasa, online erschienen am 03.08.2026. Verfasst von Rizka Oktaviani und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual literacy is the ability to understand and interpret information from visual representations, which involves linguistic processing. In the context of elementary education, this ability is crucial for helping students associate visual objects with bilingual vocabulary. However, most image captioning research remains monolingual, few studies accommodate Indonesian, and none have integrated audio representations into a single multimodal learning system. This study develops a bilingual image captioning system based on BLIP-2 and MarianMT, integrated with Text-to-Speech (TTS) within a FastAPI-based web application. English captions are generated using a pretrained BLIP-2 model, then translated into Indonesian using MarianMT, and converted to audio using gTTS. Additionally, a QLoRA fine-tuning experiment was conducted to compare model performance. The dataset consists of 6400 animal images relevant to the elementary school learning context, with caption quality evaluated using the METEOR metric. The results show that the pretrained BLIP-2 model delivers relatively stable performance with a METEOR score of 0.3765 for English and 0.3295 for Indonesian. These scores indicate that the generated captions are semantically relevant, although the overall performance is still relatively moderate compared to advanced image captioning systems. Functional testing of the prototype involving five elementary school students showed that the system is capable of generating bilingual captions and audio in real time and is easy to use. This multimodal integration supports the visual–verbal association process and has the potential to enrich students’ bilingual vocabulary, although no controlled experimental testing has yet been conducted to quantitatively measure improvements in literacy.

Zitieren

APA

Oktaviani, R., & Fauziah, F. (2026). Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa. Jurnal Ilmiah Teknologi dan Rekayasa, 31(2), 110-127. https://doi.org/10.35760/tr.2026.v31i2.152

Vancouver

Oktaviani R, Fauziah F. Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa. Jurnal Ilmiah Teknologi dan Rekayasa. 2026;31(2):110-127. doi:10.35760/tr.2026.v31i2.152

BibTeX
@article{Oktaviani2026,
  author = {Oktaviani, Rizka and Fauziah, Fauziah},
  title = {Image captioning bilingual berbasis BLIP-2 dan MarianMT dengan integrasi text-to-speech untuk literasi visual siswa},
  journal = {Jurnal Ilmiah Teknologi dan Rekayasa},
  year = {2026},
  volume = {31},
  number = {2},
  pages = {110--127},
  doi = {10.35760/tr.2026.v31i2.152}
}
Kostenlos starten →