An exploratory comparison of ChatGPT-generated and human-made Japanese reading comprehension test items
Language Education & Assessment · 24.04.2026 · 9 · 104152
DOI: 10.29140/lea.2026.104152 ↗Kurz gefasst
Fachartikel in der Zeitschrift Language Education & Assessment, online erschienen am 24.04.2026. Verfasst von Gilbert Dizon und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0).
Abstract (Originalsprache)
While existing research involving generative artificial intelligence (GenAI) has focused primarily on English, little is known about the technology’s impact on other languages, particularly in the context of second language (L2) assessment. Thus, this study examined the potential of GenAI to support non-English language assessment by comparing expert-developed reading comprehension test items and ChatGPT-generated test materials based on the Japanese Language Proficiency Test (JLPT). Japanese university language instructors (N = 16) blindly evaluated both sets of materials in terms of naturalness, attractiveness, and overall quality. Descriptive statistics and the Mann-Whitney U test revealed no significant differences between the two types of reading assessment materials. Qualitative responses indicated some concerns regarding the naturalness of the expressions, cohesion, and answer choice quality, but no major perceived differences overall. These findings suggest that GenAI may serve as a useful supplementary tool for developing JLPT preparation materials, potentially reducing time and cost burdens. However, human expertise remains necessary to refine and ensure the quality of GenAI-generated assessment materials.
Zitieren
APA
Dizon, G., & Kurose, R. (2026). An exploratory comparison of ChatGPT-generated and human-made Japanese reading comprehension test items. Language Education & Assessment, 9, 104152. https://doi.org/10.29140/lea.2026.104152
Vancouver
Dizon G, Kurose R. An exploratory comparison of ChatGPT-generated and human-made Japanese reading comprehension test items. Language Education & Assessment. 2026;9:104152. doi:10.29140/lea.2026.104152
BibTeX
@article{Dizon2026,
author = {Dizon, Gilbert and Kurose, Ryo},
title = {An exploratory comparison of ChatGPT-generated and human-made Japanese reading comprehension test items},
journal = {Language Education & Assessment},
year = {2026},
volume = {9},
pages = {104152},
doi = {10.29140/lea.2026.104152}
}
EUVIMED