EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study

Dong Gi Seo, Jeongwook Choi, Jinha Kim

Journal of Educational Evaluation for Health Professions · 2024

DOI: 10.3352/jeehp.2024.21.18 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Educational Evaluation for Health Professions, 2024. Verfasst von Dong Gi Seo und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, Hallym Applied Psychology Institute, College of Social Science, Hallym University, Chuncheon, Korea. Hauptthemen laut MEDLINE: Educational Measurement, Psychometrics, Students, Medical. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 10 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Hallym University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study aimed to compare and evaluate the efficiency and accuracy of computerized adaptive testing (CAT) under 2 stopping rules (standard error of measurement [SEM]=0.3 and 0.25) using both real and simulated data in medical examinations in Korea.

Methods

This study employed post-hoc simulation and real data analysis to explore the optimal stopping rule for CAT in medical examinations. The real data were obtained from the responses of 3rd-year medical students during examinations in 2020 at Hallym University College of Medicine. Simulated data were generated using estimated parameters from a real item bank in R. Outcome variables included the number of examinees’ passing or failing with SEM values of 0.25 and 0.30, the number of items administered, and the correlation. The consistency of real CAT result was evaluated by examining consistency of pass or fail based on a cut score of 0.0. The efficiency of all CAT designs was assessed by comparing the average number of items administered under both stopping rules.

Results

Both SEM 0.25 and SEM 0.30 provided a good balance between accuracy and efficiency in CAT. The real data showed minimal differences in pass/fail outcomes between the 2 SEM conditions, with a high correlation (r=0.99) between ability estimates. The simulation results confirmed these findings, indicating similar average item numbers between real and simulated data.

Conclusion

The findings suggest that both SEM 0.25 and 0.30 are effective termination criteria in the context of the Rasch model, balancing accuracy and efficiency in CAT.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Educational MeasurementPsychometricsStudents, MedicalHumansRepublic of KoreaComputer SimulationData AnalysisEducation, Medical, UndergraduateMaleFemale

Stichwörter der Autor:innen

Computerized adaptive testingPsychometricsRepublic of KoreaMedical studentsStatistical models

Autor:innen und Institutionen

  • Department of Psychology, Hallym Applied Psychology Institute, College of Social Science, Hallym University, Chuncheon, Korea.
  • The CAT Korea Company, Chuncheon, Korea.

Förderung

  • Hallym University

Zitieren

APA

Seo, D. G., Choi, J., & Kim, J. (2024). Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study. Journal of Educational Evaluation for Health Professions. https://doi.org/10.3352/jeehp.2024.21.18

Vancouver

Seo DG, Choi J, Kim J. Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study. Journal of Educational Evaluation for Health Professions. 2024. doi:10.3352/jeehp.2024.21.18

BibTeX
@article{Seo2024,
  author = {Seo, Dong Gi and Choi, Jeongwook and Kim, Jinha},
  title = {Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study},
  journal = {Journal of Educational Evaluation for Health Professions},
  year = {2024},
  doi = {10.3352/jeehp.2024.21.18}
}
Kostenlos starten →