Assessing Interpreting Performance Through Human and AI Evaluation: Validity, Reliability, and Pedagogical Implications
International Journal of Language, Translation and Intercultural Communication · 12.05.2026 · 11 · 114-153
DOI: 10.12681/ijltic.45426 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Language, Translation and Intercultural Communication, online erschienen am 12.05.2026. Verfasst von Effrossyni Fragkou. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).
Abstract (Originalsprache)
This paper examines the use of artificial intelligence (AI) in assessing student interpreting performance in examination settings. It draws on a corpus of 30 audio recordings produced by six students in a first-year healthcare interpreting course within an MA in Conference Interpreting in Canada. The tasks include sight translation (ENFR), consecutive interpreting (ENFR), and bidirectional medical dialogue in healthcare settings. Student renditions are compared with original source texts, both audio and written, and evaluated against a pre-established assessment grid. The study compares human instructor assessment with AI-based assessment at two points: December 2024-January 2025, during the mid-term examination period, and February 2026, introducing a longitudinal dimension. Using a mixed-methods comparative design, it combines quantitative analysis of scoring patterns with qualitative analysis of convergences and divergences, focusing on accuracy, omissions, additions, distortions, and related assessment criteria. Findings suggest that human assessment better captures prosodic and interactional features, including pronunciation, intonation, rhythm, pausing, speaker attitude, pragmatic force, and hesitation. AI assessment appears relatively stronger in evaluating linguistic and textual dimensions, including content transfer, completeness, grammar, terminology, coherence, and cohesion. The paper also addresses anonymization, voice identifiability, AI use, validity, reliability, and bias.
Zitieren
APA
Fragkou, E. (2026). Assessing Interpreting Performance Through Human and AI Evaluation: Validity, Reliability, and Pedagogical Implications. International Journal of Language, Translation and Intercultural Communication, 11, 114-153. https://doi.org/10.12681/ijltic.45426
Vancouver
Fragkou E. Assessing Interpreting Performance Through Human and AI Evaluation: Validity, Reliability, and Pedagogical Implications. International Journal of Language, Translation and Intercultural Communication. 2026;11:114-153. doi:10.12681/ijltic.45426
BibTeX
@article{Fragkou2026,
author = {Fragkou, Effrossyni},
title = {Assessing Interpreting Performance Through Human and AI Evaluation: Validity, Reliability, and Pedagogical Implications},
journal = {International Journal of Language, Translation and Intercultural Communication},
year = {2026},
volume = {11},
pages = {114--153},
doi = {10.12681/ijltic.45426}
}
EUVIMED