EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Pronunciation assessment in foreign language learning: Reliability and scoring bias in human–generative AI evaluation

Whyunyoung Choi iD

PLOS One · 29.07.2026 · 21 (7) · e0354603

DOI: 10.1371/journal.pone.0354603 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 29.07.2026. Verfasst von Whyunyoung Choi. Beteiligte Einrichtung: Department of Educational Research, County South, Lancaster University, Lancaster, United Kingdom. Hauptthemen laut MEDLINE: Language, Learning, Multilingualism. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Lancaster University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study examines the reliability and scoring bias of generative AI (Gen-AI)-based pronunciation assessment compared with human raters, addressing whether AI-generated scores can be trusted in real educational settings. Sixty students participated in a 12-week program. A total of 180 pronunciation samples were evaluated across eight subcomponents (individual phonemes, stress, rhythm, intonation, linking, reduction, fluency, and clarity) by three standardized human raters and Gen-AI using the same 7-point rubric. Quantitative analyses (intraclass correlation coefficients, paired-samples t-tests, and Pearson correlations) assessed reliability and bias, while semi-structured interviews with raters provided explanatory qualitative insights. Gen-AI demonstrated moderate reliability with human raters across most components, showing the highest agreement in fluency and the weakest in individual phonemes. However, Gen-AI consistently assigned significantly higher scores than human raters across all subcomponents. Qualitative findings revealed that discrepancies originated from Gen-AI’s limited discriminative power, systematic flaws in handling missing data, decontextualized scoring approach, lack of sensitivity to L1 interference, and inability to interpret pragmatic context. While Gen-AI cannot fully replace human expertise, it can function as a complementary tool for formative assessment and autonomous practice. A hybrid assessment model integrating Gen-AI’s efficiency with human raters’ contextual and interpretive insights is recommended for effective foreign language pronunciation education.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LanguageLearningMultilingualismHumansGenerative Artificial IntelligenceReproducibility of ResultsFemaleMalePhonetics

Autor:innen und Institutionen

  • Department of Educational Research, County South, Lancaster University, Lancaster, United Kingdom.

ORCID-Profile: Whyunyoung Choi

Förderung

  • Lancaster University

Zitieren

APA

Choi, W. (2026). Pronunciation assessment in foreign language learning: Reliability and scoring bias in human–generative AI evaluation. PLOS One, 21(7), e0354603. https://doi.org/10.1371/journal.pone.0354603

Vancouver

Choi W. Pronunciation assessment in foreign language learning: Reliability and scoring bias in human–generative AI evaluation. PLOS One. 2026;21(7):e0354603. doi:10.1371/journal.pone.0354603

BibTeX
@article{Choi2026,
  author = {Choi, Whyunyoung},
  title = {Pronunciation assessment in foreign language learning: Reliability and scoring bias in human–generative AI evaluation},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {7},
  pages = {e0354603},
  doi = {10.1371/journal.pone.0354603}
}
Kostenlos starten →