EUVIMEDEUVIMED entdecken
Crossref

Cross-Platform GenAI Evaluation for ESP Learning Scenarios: A Computational Analysis of Linguistic and Cultural Appropriateness

Andreea Nechifor, Alina Turculeț, Anca Vasilescu

Proceedings of the ... International Conference "Linguistic Resources and Tools for Natural Language Processing" · 31.12.2025 · 162

DOI: 10.47743/consilr2025.12 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ... International Conference "Linguistic Resources and Tools for Natural Language Processing", online erschienen am 31.12.2025. Verfasst von Andreea Nechifor und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The present study is based on the evaluation of over 180 mobility scenarios AIgenerated by five platforms (ChatGPT, Claude, Copilot, DeepSeek, and Gemini) by means of an ample framework that integrates perspectives from applied linguistics and computational linguistics. Based on Hutchinson and Waters’ (1987) ESP framework, Byram’s (1997) intercultural competence model, and Thomas’s (1983) pragmalinguistic distinctions, the research employs five evaluation criteria: pedagogical authenticity, linguistic appropriateness, cultural sensitivity, discourse coherence, and pragmatic authenticity. Results disclose platform variations, pragmatic authenticity averaging around a solid 4.03, cultural sensitivity (4.55) and pedagogical value (4.16) denoting advanced AI competencies, while discourse coherence (3.92) and linguistic appropriateness (3.88) still presenting challenges. Gemini performed the best across most dimensions, while linguistic register management and conversational naturalness demonstrated to be problematic across all platforms. A validation study was also undertaken to compare the automated estimates generated by Llama 3.1 with human specialist ratings, which produced a moderate positive correlation (ρ = 0.40, p = .51), thus indicating partial alignment between computational and expert findings, even if statistical significance was not achieved considering the limited sample size (N = 5). The results demonstrate that current GenAI platforms possess significant ESP content generation potential, at the same time requiring careful evaluation structures supported by corpus-based analysis (Swales, 1990; Charles, 2012) and applied linguistics perspectives (Canagarajah, 2013; Pennycook, 2001) to ensure culturally sensitive and pedagogically authentic professional language education materials, with human supervision remaining essential for quality assurance.

Zitieren

APA

Nechifor, A., Turculeț, A., & Vasilescu, A. (2025). Cross-Platform GenAI Evaluation for ESP Learning Scenarios: A Computational Analysis of Linguistic and Cultural Appropriateness. Proceedings of the ... International Conference "Linguistic Resources and Tools for Natural Language Processing", 162. https://doi.org/10.47743/consilr2025.12

Vancouver

Nechifor A, Turculeț A, Vasilescu A. Cross-Platform GenAI Evaluation for ESP Learning Scenarios: A Computational Analysis of Linguistic and Cultural Appropriateness. Proceedings of the ... International Conference "Linguistic Resources and Tools for Natural Language Processing". 2025:162. doi:10.47743/consilr2025.12

BibTeX
@article{Nechifor2025,
  author = {Nechifor, Andreea and Turculeț, Alina and Vasilescu, Anca},
  title = {Cross-Platform GenAI Evaluation for ESP Learning Scenarios: A Computational Analysis of Linguistic and Cultural Appropriateness},
  journal = {Proceedings of the ... International Conference "Linguistic Resources and Tools for Natural Language Processing"},
  year = {2025},
  pages = {162},
  doi = {10.47743/consilr2025.12}
}
Kostenlos starten →