EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Comparison of Automatic Item Generation Methods in the Assessment of Clinical Reasoning Skills

Emre Emekli, Betül Nalan Karahan

Revista Española de Educación Médica · 25.11.2024 · 6 (1)

DOI: 10.6018/edumed.637221 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Revista Española de Educación Médica, online erschienen am 25.11.2024. Verfasst von Emre Emekli und 1 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The use of automatic item generation (AIG) methods offers potential for assessing clinical reasoning (CR) skills in medical education, a critical skill combining intuitive and analytical thinking. In preclinical education, these skills are commonly evaluated through written exams and case-based multiple-choice questions (MCQs), which are widely used due to the high number of students, ease of standardization, and quick evaluation. This research generated CR-focused questions for medical exams using two primary AIG methods: template-based and non-template-based (using AI tools like ChatGPT for a flexible approach). A total of 18 questions were produced on ordering radiologic investigations for abdominal emergencies, alongside faculty-developed questions used in medical exams for comparison. Experienced radiologists evaluated the questions based on clarity, clinical relevance, and effectiveness in measuring CR skills. Results showed that ChatGPT-generated questions measured CR skills with an 84.52% success rate, faculty-developed questions with 82.14%, and template-based questions with 78.57%, indicating that both AIG methods are effective in CR assessment, with ChatGPT performing slightly better. Both AIG methods received high ratings for clarity and clinical suitability, showing promise in producing effective CR-assessing questions comparable to, and in some cases surpassing, faculty-developed questions. While template-based AIG is effective, it requires more time and effort, suggesting that both methods may offer time-saving potential in exam preparation for educators.

Zitieren

APA

Emekli, E., & Karahan, B. N. (2024). Comparison of Automatic Item Generation Methods in the Assessment of Clinical Reasoning Skills. Revista Española de Educación Médica, 6(1). https://doi.org/10.6018/edumed.637221

Vancouver

Emekli E, Karahan BN. Comparison of Automatic Item Generation Methods in the Assessment of Clinical Reasoning Skills. Revista Española de Educación Médica. 2024;6(1). doi:10.6018/edumed.637221

BibTeX
@article{Emekli2024,
  author = {Emekli, Emre and Karahan, Betül Nalan},
  title = {Comparison of Automatic Item Generation Methods in the Assessment of Clinical Reasoning Skills},
  journal = {Revista Española de Educación Médica},
  year = {2024},
  volume = {6},
  number = {1},
  doi = {10.6018/edumed.637221}
}
Kostenlos starten →