LARGE LANGUAGE MODELS AND BULGARIAN LITERATURE (AN EXPERIMENTAL STUDY ON 7TH-GRADE NATIONAL EXTERNAL ASSESSMENT TASKS)
Ezikov Svyat (Orbis Linguarum) · 30.10.2026 · 24 (3) · 175-184
DOI: 10.37708/ezs.swu.bg.v24i3.11 ↗Kurz gefasst
Fachartikel in der Zeitschrift Ezikov Svyat (Orbis Linguarum), online erschienen am 26.10.2026. Verfasst von Vladislav Marinov.
Abstract (Originalsprache)
This study compares the performance of BgGPT, Gemini, Claude, and ChatGPT on standardized tasks concerning Bulgarian literature. The corpus consists of literature tasks 18–24 from the Bulgarian national external assessment for 7th grade over five consecutive years (2022–2026). Each test was solved three times by each system in a new session, producing 60 sessions in total. Multiple-choice tasks were scored against the official answer keys, while the four short answers in task 24 were scored separately and normalized so that each test task had equal weight. The results do not support the working hypothesis that a model adapted to Bulgarian would outperform general multilingual systems. Gemini achieved the highest overall score (88.1%), followed by ChatGPT (86.0%), BgGPT (68.8%), and Claude (61.4%). All four systems performed less accurately on the short free-response task than on the multiple-choice tasks, although the present design does not isolate response format from content. Performance also varied across types of literary work and across cumulative year samples. The comparison is limited to the freely available versions of the four systems accessed in July 2026 and does not isolate the effects of their technical characteristics. The findings indicate that language adaptation alone is insufficient to predict performance on subject-specific and culturally localized content and support the use of authentic local educational tasks when evaluating language models for educational contexts.
Zitieren
APA
Marinov, V. (2026). LARGE LANGUAGE MODELS AND BULGARIAN LITERATURE (AN EXPERIMENTAL STUDY ON 7TH-GRADE NATIONAL EXTERNAL ASSESSMENT TASKS). Ezikov Svyat (Orbis Linguarum), 24(3), 175-184. https://doi.org/10.37708/ezs.swu.bg.v24i3.11
Vancouver
Marinov V. LARGE LANGUAGE MODELS AND BULGARIAN LITERATURE (AN EXPERIMENTAL STUDY ON 7TH-GRADE NATIONAL EXTERNAL ASSESSMENT TASKS). Ezikov Svyat (Orbis Linguarum). 2026;24(3):175-184. doi:10.37708/ezs.swu.bg.v24i3.11
BibTeX
@article{Marinov2026,
author = {Marinov, Vladislav},
title = {LARGE LANGUAGE MODELS AND BULGARIAN LITERATURE (AN EXPERIMENTAL STUDY ON 7TH-GRADE NATIONAL EXTERNAL ASSESSMENT TASKS)},
journal = {Ezikov Svyat (Orbis Linguarum)},
year = {2026},
volume = {24},
number = {3},
pages = {175--184},
doi = {10.37708/ezs.swu.bg.v24i3.11}
}
EUVIMED