EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An Effectiveness Study of Generative Artificial Intelligence Tools Used to Develop Multiple-Choice Test Items

Toni A. May, Yiyun Kate Fan, Gregory E. Stone, Kristin L. K. Koskey, Connor J. Sondergeld, Timothy D. Folger, James N. Archer, Kathleen Provinzano, Carla C. Johnson

Education Sciences · 24.01.2025 · 15 (2) · 144

DOI: 10.3390/educsci15020144 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Education Sciences, online erschienen am 24.01.2025. Verfasst von Toni A. May und 8 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Generative artificial intelligence (GenAI) tools developed to support teaching and learning are widely available. Trustworthiness concerns, however, have prompted calls for researchers to study their effectiveness and for educators and educational researchers to be involved in their creation and piloting processes. This study investigated one type of GenAI created to support educators: multiple-choice question generators (MCQ GenAI). Among the nine MCQ GenAI tools investigated, a variety of useful options were available, but only one indicated teacher involvement and none mentioned testing experts in development processes. MCQ GenAI-created items (n = 270) were coded based on MCQ quality item-writing guidelines. Results showed 80.00% of items (n = 216) violated at least one guideline, with 73.70% (n = 199) likely to produce major measurement error (should not use without revision), 6.30% (n = 17) likely to elicit minor measurement error (consider modifying), and 20.00% (n = 54) acceptable (usable as created). Implications suggest multidisciplinary teams are needed in educational GenAI tool development.

Zitieren

APA

May, T. A., Fan, Y. K., Stone, G. E., Koskey, K. L. K., Sondergeld, C. J., Folger, T. D., Archer, J. N., Provinzano, K., & Johnson, C. C. (2025). An Effectiveness Study of Generative Artificial Intelligence Tools Used to Develop Multiple-Choice Test Items. Education Sciences, 15(2), 144. https://doi.org/10.3390/educsci15020144

Vancouver

May TA, Fan YK, Stone GE, Koskey KLK, Sondergeld CJ, Folger TD, et al. An Effectiveness Study of Generative Artificial Intelligence Tools Used to Develop Multiple-Choice Test Items. Education Sciences. 2025;15(2):144. doi:10.3390/educsci15020144

BibTeX
@article{May2025,
  author = {May, Toni A. and Fan, Yiyun Kate and Stone, Gregory E. and Koskey, Kristin L. K. and Sondergeld, Connor J. and Folger, Timothy D. and Archer, James N. and Provinzano, Kathleen and Johnson, Carla C.},
  title = {An Effectiveness Study of Generative Artificial Intelligence Tools Used to Develop Multiple-Choice Test Items},
  journal = {Education Sciences},
  year = {2025},
  volume = {15},
  number = {2},
  pages = {144},
  doi = {10.3390/educsci15020144}
}
Kostenlos starten →