EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Evaluating Machine Translation Models and LLMs for Indonesian–Javanese Translation Across Speech Levels

Mahendra Bayu Prayoga, Bagas Restya Ermawan, Akmal Rafi Fadhillah, Mohammad Nizar Farizi, Ema Utami

Journal of Applied Informatics and Computing · 17.04.2026 · 10 (2) · 1549-1560

DOI: 10.30871/jaic.v10i2.12326 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Applied Informatics and Computing, online erschienen am 17.04.2026. Verfasst von Mahendra Bayu Prayoga und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite being one of the most widely spoken regional languages in Indonesia, Javanese remains underrepresented in modern machine translation systems, particularly with respect to its hierarchical speech-level system. This study presents a comprehensive benchmarking of machine translation approaches for low-resource Indonesian-to-Javanese translation with explicit consideration of Javanese speech-level registers, namely Ngoko, Krama, and Krama Alus. We evaluate the effectiveness of two multilingual neural machine translation models, NLLB-200 and M2M100, under both zero-shot and supervised fine-tuning settings using a parallel corpus of approximately 4,000 sentence pairs from the Unggah-Ungguh dataset. Translation quality is assessed using BLEU, chrF++, METEOR, and BERTScore on both register-specific and overall test sets constructed from a balanced evaluation set of 1,500 sentence pairs (500 per register). Experimental results show that supervised fine-tuning substantially improves translation performance, with fine-tuned M2M100 achieving the strongest results among neural machine translation models. In addition, instruction-based translation using the Gemini large language model demonstrates superior overall performance, particularly in semantic-oriented metrics, highlighting its effectiveness under controlled instruction-based conditions within the scope of this experimental configuration. Overall, this study provides a reproducible and extensible evaluation framework for sociolinguistically informed machine translation of regional languages.

Zitieren

APA

Prayoga, M. B., Ermawan, B. R., Fadhillah, A. R., Farizi, M. N., & Utami, E. (2026). Evaluating Machine Translation Models and LLMs for Indonesian–Javanese Translation Across Speech Levels. Journal of Applied Informatics and Computing, 10(2), 1549-1560. https://doi.org/10.30871/jaic.v10i2.12326

Vancouver

Prayoga MB, Ermawan BR, Fadhillah AR, Farizi MN, Utami E. Evaluating Machine Translation Models and LLMs for Indonesian–Javanese Translation Across Speech Levels. Journal of Applied Informatics and Computing. 2026;10(2):1549-1560. doi:10.30871/jaic.v10i2.12326

BibTeX
@article{Prayoga2026,
  author = {Prayoga, Mahendra Bayu and Ermawan, Bagas Restya and Fadhillah, Akmal Rafi and Farizi, Mohammad Nizar and Utami, Ema},
  title = {Evaluating Machine Translation Models and LLMs for Indonesian–Javanese Translation Across Speech Levels},
  journal = {Journal of Applied Informatics and Computing},
  year = {2026},
  volume = {10},
  number = {2},
  pages = {1549--1560},
  doi = {10.30871/jaic.v10i2.12326}
}
Kostenlos starten →