Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination
PLOS ONE · 06.09.2024 · 19 (9) · e0309887
DOI: 10.1371/journal.pone.0309887 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 06.09.2024. Verfasst von Masaki Uto und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer and Network Engineering, The University of Electro-Communications, Chofu, Tokyo, Japan. Hauptthemen laut MEDLINE: Educational Measurement. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.
Abstract (Originalsprache)
Objective structured clinical examinations (OSCEs) are a widely used performance assessment for medical and dental students. A common limitation of OSCEs is that the evaluation results depend on the characteristics of raters and a scoring rubric. To overcome this limitation, item response theory (IRT) models such as the many-facet Rasch model have been proposed to estimate examinee abilities while taking into account the characteristics of raters and evaluation items in a rubric. However, conventional IRT models have two impractical assumptions: constant rater severity across all evaluation items in a rubric and an equal interval rating scale among evaluation items, which can decrease model fitting and ability measurement accuracy. To resolve this problem, we propose a new IRT model that introduces two parameters: (1) a rater–item interaction parameter representing the rater severity for each evaluation item and (2) an item-specific step-difficulty parameter representing the difference in rating scales among evaluation items. We demonstrate the effectiveness of the proposed model by applying it to actual data collected from a medical interview test conducted at Tokyo Medical and Dental University as part of a post-clinical clerkship OSCE. The experimental results showed that the proposed model was well-fitted to our OSCE data and measured ability accurately. Furthermore, it provided abundant information on rater and item characteristics that conventional models cannot, helping us to better understand rater and item properties.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Computer and Network Engineering, The University of Electro-Communications, Chofu, Tokyo, Japan.
- Institute of Education, Tokyo Medical and Dental University, Bunkyo-ku, Tokyo, Japan.
- Educational System in Dentistry, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Bunkyo-ku, Tokyo, Japan.
ORCID-Profile: Masaki Uto
Förderung
- Japan Society for the Promotion of Science
Zitieren
APA
Uto, M., Tsuruta, J., Araki, K., & Ueno, M. (2024). Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination. PLOS ONE, 19(9), e0309887. https://doi.org/10.1371/journal.pone.0309887
Vancouver
Uto M, Tsuruta J, Araki K, Ueno M. Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination. PLOS ONE. 2024;19(9):e0309887. doi:10.1371/journal.pone.0309887
BibTeX
@article{Uto2024,
author = {Uto, Masaki and Tsuruta, Jun and Araki, Kouji and Ueno, Maomi},
title = {Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination},
journal = {PLOS ONE},
year = {2024},
volume = {19},
number = {9},
pages = {e0309887},
doi = {10.1371/journal.pone.0309887}
}
EUVIMED