EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Local language models for Spanish L2 writing assessment: human baselines, RAG, and fine-tuning

Giovanni Zimotti

Journal of Computers in Education · 30.06.2026

DOI: 10.1007/s40692-026-00399-w ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Computers in Education, online erschienen am 30.06.2026. Verfasst von Giovanni Zimotti. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study investigates the use of local Large Language Models (LLMs): Gemma, LLama, Mistral Small, and Phi, in assessing Spanish second language (L2) writing. While commercial models like ChatGPT have already been used for this task, they present privacy risks and reproducibility challenges. To evaluate local alternatives to ChatGPT, human raters were used to assess a third-semester assignment, evaluating 100 paragraphs to provide a baseline of comparison for the local LLMs’ assessment. Results showed varying performance: while some models approached the human baseline; others were inaccurate and inconsistent. On average, they performed better using a holistic rubric than an analytic one. A third experiment, averaging the top 3 models: Llama3.3:70b, Gemma2:9b, Mistral-Small:22b, achieved the highest agreement ( r = 0.88), surpassing inter-human rating. A separate experiment testing RAG and fine-tuning techniques on the smallest model: Llama3.1:8b, showed no significant improvements with RAG but substantial gains in accuracy and consistency with fine tuning. Findings suggest that local LLMs show promise for Spanish L2 writing assessment. Local LLMs are preferable to proprietary ones because they protect privacy and reduce risks of data exposure, they may better support replicable workflows and may offer practical advantages for long term adoption.

Zitieren

APA

Zimotti, G. (2026). Local language models for Spanish L2 writing assessment: human baselines, RAG, and fine-tuning. Journal of Computers in Education. https://doi.org/10.1007/s40692-026-00399-w

Vancouver

Zimotti G. Local language models for Spanish L2 writing assessment: human baselines, RAG, and fine-tuning. Journal of Computers in Education. 2026. doi:10.1007/s40692-026-00399-w

BibTeX
@article{Zimotti2026,
  author = {Zimotti, Giovanni},
  title = {Local language models for Spanish L2 writing assessment: human baselines, RAG, and fine-tuning},
  journal = {Journal of Computers in Education},
  year = {2026},
  doi = {10.1007/s40692-026-00399-w}
}
Kostenlos starten →