EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination

Masaki Uto iD, Jun Tsuruta, Kouji Araki, Maomi Ueno

PLOS ONE · 06.09.2024 · 19 (9) · e0309887

DOI: 10.1371/journal.pone.0309887 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 06.09.2024. Verfasst von Masaki Uto und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer and Network Engineering, The University of Electro-Communications, Chofu, Tokyo, Japan. Hauptthemen laut MEDLINE: Educational Measurement. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective structured clinical examinations (OSCEs) are a widely used performance assessment for medical and dental students. A common limitation of OSCEs is that the evaluation results depend on the characteristics of raters and a scoring rubric. To overcome this limitation, item response theory (IRT) models such as the many-facet Rasch model have been proposed to estimate examinee abilities while taking into account the characteristics of raters and evaluation items in a rubric. However, conventional IRT models have two impractical assumptions: constant rater severity across all evaluation items in a rubric and an equal interval rating scale among evaluation items, which can decrease model fitting and ability measurement accuracy. To resolve this problem, we propose a new IRT model that introduces two parameters: (1) a rater–item interaction parameter representing the rater severity for each evaluation item and (2) an item-specific step-difficulty parameter representing the difference in rating scales among evaluation items. We demonstrate the effectiveness of the proposed model by applying it to actual data collected from a medical interview test conducted at Tokyo Medical and Dental University as part of a post-clinical clerkship OSCE. The experimental results showed that the proposed model was well-fitted to our OSCE data and measured ability accurately. Furthermore, it provided abundant information on rater and item characteristics that conventional models cannot, helping us to better understand rater and item properties.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Educational MeasurementHumansClinical CompetenceStudents, MedicalModels, TheoreticalStudents, Dental

Autor:innen und Institutionen

  • Department of Computer and Network Engineering, The University of Electro-Communications, Chofu, Tokyo, Japan.
  • Institute of Education, Tokyo Medical and Dental University, Bunkyo-ku, Tokyo, Japan.
  • Educational System in Dentistry, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Bunkyo-ku, Tokyo, Japan.

ORCID-Profile: Masaki Uto

Förderung

  • Japan Society for the Promotion of Science

Zitieren

APA

Uto, M., Tsuruta, J., Araki, K., & Ueno, M. (2024). Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination. PLOS ONE, 19(9), e0309887. https://doi.org/10.1371/journal.pone.0309887

Vancouver

Uto M, Tsuruta J, Araki K, Ueno M. Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination. PLOS ONE. 2024;19(9):e0309887. doi:10.1371/journal.pone.0309887

BibTeX
@article{Uto2024,
  author = {Uto, Masaki and Tsuruta, Jun and Araki, Kouji and Ueno, Maomi},
  title = {Item response theory model highlighting rating scale of a rubric and rater–rubric interaction in objective structured clinical examination},
  journal = {PLOS ONE},
  year = {2024},
  volume = {19},
  number = {9},
  pages = {e0309887},
  doi = {10.1371/journal.pone.0309887}
}
Kostenlos starten →