Conventionally Developed Versus Large Language Model-Generated Exam Tasks in Higher Education: A Comparative Analysis
Journal of Digital Educational Technology · 29.10.2026 · 6 (2) · ep2622
DOI: 10.29333/jdet/19699 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Digital Educational Technology, erschienen am 29.10.2026. Verfasst von Niklas Lettow und 2 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben.
Abstract (Originalsprache)
This exploratory field study investigates how human-screened exam tasks generated by the large language model (LLM) GPT-4o perform in comparison with conventionally developed exam tasks with regard to selected quality criteria. Using two business administration modules at a German distance-learning university, human-screened LLM-generated tasks were integrated into submission assignments (SA) alongside instructor-created tasks. Four selected task-quality indicators were examined: objective difficulty, subjective difficulty, context relevance, and item discrimination. The analysis combines descriptive statistics with paired-sample t-tests and Wilcoxon signed-rank tests, applied at three aggregation levels: SA-specific, module-level, and cross-module. The results revealed mixed and task-specific patterns. While selected indicators were numerically similar for some task pairs, statistically significant and, in one comparison, substantial differences were observed in others, particularly in objective difficulty. Given that the study examined only four task pairs that were not systematically matched and that prior human screening was conducted, the findings establish neither equivalence nor general comparability. However, the study provides initial exploratory insights from two business administration modules that can serve as a foundation for further research.
Zitieren
APA
Lettow, N., Pantani, K., & Littkemann, J. (2026). Conventionally Developed Versus Large Language Model-Generated Exam Tasks in Higher Education: A Comparative Analysis. Journal of Digital Educational Technology, 6(2), ep2622. https://doi.org/10.29333/jdet/19699
Vancouver
Lettow N, Pantani K, Littkemann J. Conventionally Developed Versus Large Language Model-Generated Exam Tasks in Higher Education: A Comparative Analysis. Journal of Digital Educational Technology. 2026;6(2):ep2622. doi:10.29333/jdet/19699
BibTeX
@article{Lettow2026,
author = {Lettow, Niklas and Pantani, Kristopher and Littkemann, Jörn},
title = {Conventionally Developed Versus Large Language Model-Generated Exam Tasks in Higher Education: A Comparative Analysis},
journal = {Journal of Digital Educational Technology},
year = {2026},
volume = {6},
number = {2},
pages = {ep2622},
doi = {10.29333/jdet/19699}
}
EUVIMED