EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Evaluating the Potential of Open LLMs in Multilingual and On-Device Environments

Youngho Lee

TEM Journal · 27.05.2026 · 15 (2) · 1493

DOI: 10.18421/tem152-45 ↗

Kurz gefasst

Fachartikel in der Zeitschrift TEM Journal, online erschienen am 27.05.2026. Verfasst von Youngho Lee. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing literature highlights Socratic questioning as a key method for fostering critical thinking, yet its scalability is limited by teacher workload. While Large Language Models (LLMs) offer automation potential, recent studies point to performance disparities between Open and Closed models, particularly in multilingual and resourceconstrained environments. This study addresses two primary research questions: whether Open LLMs can perform comparably to Closed LLMs in generating Socratic questions for Korean students, and if on-device models are practically feasible. It was hypothesized that optimized Open LLMs would achieve semantic parity with proprietary Closed models and that quantized ondevice versions would maintain sufficient quality for offline educational use. To test this, a comparative analysis was conducted using a dataset of 2,400 argumentative essays written by South Korean elementary (grades 4–6), middle, and high school students. Four models—Phi-4 (14B), Phi-4 GGUF, Llama-3.1 (8B), and GPT-o3-mini were evaluated using BLEU, ROUGE-L, Cosine Similarity, and BERTScore. The results demonstrate that Phi-4 (14B) achieved performance comparable to the Closed model benchmark, and the on-device Phi-4 GGUF proved viable for privacy-conscious environments. Notably, question quality was higher for elementary-level essays and consistent across varying student performance levels, as long as logical structure was maintained in the input. These findings support the practical application of open and lightweight LLMs to deliver scalable and equitable AI-based feedback.

Zitieren

APA

Lee, Y. (2026). Evaluating the Potential of Open LLMs in Multilingual and On-Device Environments. TEM Journal, 15(2), 1493. https://doi.org/10.18421/tem152-45

Vancouver

Lee Y. Evaluating the Potential of Open LLMs in Multilingual and On-Device Environments. TEM Journal. 2026;15(2):1493. doi:10.18421/tem152-45

BibTeX
@article{Lee2026,
  author = {Lee, Youngho},
  title = {Evaluating the Potential of Open LLMs in Multilingual and On-Device Environments},
  journal = {TEM Journal},
  year = {2026},
  volume = {15},
  number = {2},
  pages = {1493},
  doi = {10.18421/tem152-45}
}
Kostenlos starten →