EUVIMEDEUVIMED entdecken
Crossref

A tool for the rapid postexamination analysis of examination data using classical test theory, generalizability theory, and the Rasch model

Mohsen Tavakol, Claire Stewart

Academic Medicine · 2025

DOI: 10.1093/acamed/wvaf086 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Academic Medicine, 2025. Verfasst von Mohsen Tavakol und 1 weiteren Autor:innen. Beteiligte Einrichtung: Education Centre, School of Medicine, University of Nottingham, Nottingham, United Kingdom. Hauptthemen laut MEDLINE: Psychometrics, Educational Measurement, Education, Medical. Die Arbeit nennt 7 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

PROBLEM: Effective postexamination psychometric analysis plays an important role in supporting medical education assessments and is critical for ensuring reliability, dependability, validity, and fairness, which ultimately contribute to patient safety. However, many medical educational settings face significant barriers to achieving this, including expensive or complex software, limited psychometric expertise or programming skills, and increasing budgetary constraints regarding the need for rapid score release and quality control. This report describes a free, user-friendly tool that enables health education professions to conduct rapid and accurate postexamination analysis using classical test theory (CTT), generalizability theory (GT), and the Rasch model (RM). APPROACH: The web-based tool, developed and tested in 2025 at the University of Nottingham, removes the need for complex postexamination data processing and provides real-time item performance metrics, reliability coefficients, and diagnostic reports, enabling educators to rapidly compare 3 measurement theories simultaneously for a comprehensive postexamination review. OUTCOMES: A dataset from a high-stakes final medical assessment (489 students, 200 items) was uploaded. The tool rapidly produced CTT measures, GT variance components, and a decision study, including G-coefficient and phi-coefficient, as well as the absolute standard error of measurement and RM calibrations. Under CTT, the tool also outputs individual outcomes, histograms of marks, the true score and 68% or 95% bounds, rank scores, z scores, percentiles, and deciles. The tool also identified misfit items and persons, item-person maps, separation coefficients, and Rasch reliability. Additionally, the tool produced item characteristics curves, test characteristic curves, and test information function. NEXT STEPS: Future work includes expanding the tool by adding categorical responses to fully evaluate the quality of single-best-answer questions or multiple-choice question distractors. This aim will be achieved by analyzing plausible and implausible options and displaying the correlation between the total scores and the occurrence of each answer option for each question.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PsychometricsEducational MeasurementEducation, MedicalHumansReproducibility of Results

Stichwörter der Autor:innen

assessmentmeasurement theoriespsychometric analysisrapid analysis

Autor:innen und Institutionen

  • Education Centre, School of Medicine, University of Nottingham, Nottingham, United Kingdom.

Zitieren

APA

Tavakol, M., & Stewart, C. (2025). A tool for the rapid postexamination analysis of examination data using classical test theory, generalizability theory, and the Rasch model. Academic Medicine. https://doi.org/10.1093/acamed/wvaf086

Vancouver

Tavakol M, Stewart C. A tool for the rapid postexamination analysis of examination data using classical test theory, generalizability theory, and the Rasch model. Academic Medicine. 2025. doi:10.1093/acamed/wvaf086

BibTeX
@article{Tavakol2025,
  author = {Tavakol, Mohsen and Stewart, Claire},
  title = {A tool for the rapid postexamination analysis of examination data using classical test theory, generalizability theory, and the Rasch model},
  journal = {Academic Medicine},
  year = {2025},
  doi = {10.1093/acamed/wvaf086}
}
Kostenlos starten →