A practical guide to automated test assembly for educational evaluation in health professions: a demonstration using the R package ROI
Journal of Educational Evaluation for Health Professions · 2026
DOI: 10.3352/jeehp.2026.23.31 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Educational Evaluation for Health Professions, 2026. Verfasst von Hwanggyu Lim und 2 weiteren Autor:innen. Beteiligte Einrichtung: College of Education, Inha University, Incheon, Korea. hlim@inha.ac.kr. Hauptthemen laut MEDLINE: Educational Measurement, Health Occupations.
Abstract (Originalsprache)
Automated test assembly (ATA) using mixed-integer programming (MIP) is standard practice in international testing organizations, but detailed and accessible practical guidelines remain scarce for health professions assessment. This study introduces the mathematical foundations of MIP-based ATA and provides a step-by-step tutorial demonstrating its implementation with the open-source R package ROI and the HiGHS solver. We constructed a modified 928-item pool with empirical item parameters from the Clinical Medical Science Comprehensive Examination, establishing 31 content categories and 18 enemy-item groups. Using the ROI package and the HiGHS solver, we modeled and solved 3 assembly scenarios: a single 80-item form under a relative target, 2 parallel 80-item forms under a relative target, and 2 parallel 80-item forms under an absolute target combined with additional mean item difficulty constraints. All 3 optimization scenarios satisfied the test-length, content-blueprint, and enemy-item constraints. The relative-target forms attained observed-to-target information ratios within rounding error of the targets, producing closely equivalent parallel forms. The absolute-target forms closely approximated the specified information curve, with a maximum deviation of 0.27 information units, and their mean difficulties fell within the targeted range of 0.2 to 0.4. All formulated MIP models converged to optimal solutions within a few minutes. The practical combination of the ROI package and the HiGHS solver offers an efficient, reproducible, and cost-effective method for test assembly. This approach may enable Korean health professions examination programs to construct psychometrically equivalent and content-balanced test forms, facilitating the transition toward item-bank-based testing systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Education, Inha University, Incheon, Korea. hlim@inha.ac.kr.
- Department of Surgery, Yonsei University College of Medicine, Seoul, Korea.
- The CAT Korea Company, Chuncheon, Korea.
Zitieren
APA
Lim, H., Kim, K. S., & Choi, J. (2026). A practical guide to automated test assembly for educational evaluation in health professions: a demonstration using the R package ROI. Journal of Educational Evaluation for Health Professions. https://doi.org/10.3352/jeehp.2026.23.31
Vancouver
Lim H, Kim KS, Choi J. A practical guide to automated test assembly for educational evaluation in health professions: a demonstration using the R package ROI. Journal of Educational Evaluation for Health Professions. 2026. doi:10.3352/jeehp.2026.23.31
BibTeX
@article{Lim2026,
author = {Lim, Hwanggyu and Kim, Kyung Sik and Choi, Jeongwook},
title = {A practical guide to automated test assembly for educational evaluation in health professions: a demonstration using the R package ROI},
journal = {Journal of Educational Evaluation for Health Professions},
year = {2026},
doi = {10.3352/jeehp.2026.23.31}
}
EUVIMED