Does ChatGPT Adapt Itself to the Language Used and the Audience It Implies?
Computational Linguistics in Bulgaria · 2025
DOI: 10.47810/jclib.1.2025.02 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computational Linguistics in Bulgaria, 2025. Verfasst von Iglika Nikolova Stoupak und 2 weiteren Autor:innen.
Abstract (Originalsprache)
This paper seeks to quantify and analyse the progress that ChatGPT has made from its GPT-3.5 (2022) to its GPT-4.5 (2025) version when it comes to answering prompts in a selection of differently-resourced languages: English, Bulgarian, Greek, French, Hebrew, Japanese and Russian. Factual correctness, textual quality and an answer’s linguistic and cultural independence from an English baseline are evaluated in the process. Each response is marked positively or negatively for each of the three metrics based on a set of defined criteria and careful human-based analysis. In addition, three categories of questions are experimented with: general (e.g. communication assistance or request for jokes), perception-related (e.g. creative writing or explanation of physical processes) and geography-/culture-sensitive (questions in a specific language that address a particular, slightly sensitive topic related to the implied audience e.g. ‘Why do French people eat snails?’). As hypothesised, the recent GPT-4.5 version demonstrates significant progress in all evaluated categories, thereby resolving past issues such as decreased textual quality of low-resourced languages and, notably, very limited variety in answers to the same question across languages. The metric ‘Independence from the (English) Baseline’ receives 80.95% of positive marks in the GPT-4.5 version as opposed to 26.19% for GPT-3.5. Lingering problems include ChatGPT’s incomplete ability to generate relevant and culturally-sensitive jokes and poems.
Zitieren
APA
Stoupak, I. N., Lejeune, G., & Schaeffer-Lacroix, E. (2025). Does ChatGPT Adapt Itself to the Language Used and the Audience It Implies?. Computational Linguistics in Bulgaria. https://doi.org/10.47810/jclib.1.2025.02
Vancouver
Stoupak IN, Lejeune G, Schaeffer-Lacroix E. Does ChatGPT Adapt Itself to the Language Used and the Audience It Implies?. Computational Linguistics in Bulgaria. 2025. doi:10.47810/jclib.1.2025.02
BibTeX
@article{Stoupak2025,
author = {Stoupak, Iglika Nikolova and Lejeune, Gaël and Schaeffer-Lacroix, Eva},
title = {Does ChatGPT Adapt Itself to the Language Used and the Audience It Implies?},
journal = {Computational Linguistics in Bulgaria},
year = {2025},
doi = {10.47810/jclib.1.2025.02}
}
EUVIMED