Regional LLM versus Dedicated Neural MT for Waray–English: A Bidirectional FLORES-200 Evaluation of a Low-Resource Philippine Language
Computación y Sistemas · 29.09.2026 · 30 (3)
DOI: 10.13053/cys-30-3-6497 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computación y Sistemas, online erschienen am 29.09.2026. Verfasst von Richelle R. Esporas und 1 weiteren Autor:innen.
Abstract (Originalsprache)
. Waray (Winaray), an Austronesian language of the eastern Philippines with about three million speakers, is computationally low-resourced and rarely evaluated in machine translation (MT). This study compares two MT approaches on Waray–English using FLORES-200: NLLB-200, a dedicated multilingual neural model that supports Waray, and Gemma-SEA-LION-v4-27B-IT, a Southeast Asian regional large language model (LLM) for which Waray is unsupported. Both were evaluated bidirectionally with spBLEU and chrF++, and a blind two-rater expert evaluation assessed the into-Waray direction. A directional asymmetry emerged: the LLM exceeded the dedicated model by 16.5 chrF++ into English but trailed by 4.5 into Waray. Yet two native experts found no significant difference in the systems’ Waray output, suggesting the metric gap may largely reflect single-reference divergence rather than quality. A consistent domain split favored the LLM on news and the dedicated model on instructional text. The findings caution against surface-metric-only evaluation for low-resource generation.
Zitieren
APA
Esporas, R. R., & Sedanza, M. A. C. (2026). Regional LLM versus Dedicated Neural MT for Waray–English: A Bidirectional FLORES-200 Evaluation of a Low-Resource Philippine Language. Computación y Sistemas, 30(3). https://doi.org/10.13053/cys-30-3-6497
Vancouver
Esporas RR, Sedanza MAC. Regional LLM versus Dedicated Neural MT for Waray–English: A Bidirectional FLORES-200 Evaluation of a Low-Resource Philippine Language. Computación y Sistemas. 2026;30(3). doi:10.13053/cys-30-3-6497
BibTeX
@article{Esporas2026,
author = {Esporas, Richelle R. and Sedanza, Mary Anne C.},
title = {Regional LLM versus Dedicated Neural MT for Waray–English: A Bidirectional FLORES-200 Evaluation of a Low-Resource Philippine Language},
journal = {Computación y Sistemas},
year = {2026},
volume = {30},
number = {3},
doi = {10.13053/cys-30-3-6497}
}
EUVIMED