Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery
Surgical Endoscopy · 2024
DOI: 10.1007/s00464-024-11024-1 ↗Kurz gefasst
Validierungsstudie in der Zeitschrift Surgical Endoscopy, 2024. Verfasst von Conor Toale und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Surgical Affairs, Royal College of Surgeons in Ireland, 121 St. Stephen's Green, Dublin, Ireland. conortoale@rcsi.com. Hauptthemen laut MEDLINE: Clinical Competence, General Surgery, Simulation Training, Educational Measurement. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Royal College of Surgeons in Ireland/ Hermitage Medical Clinic Strategic Academic Recruitment (StAR MD) programme, Royal College of Surgeons in Ireland.
Abstract (Originalsprache)
Background
Simulation is increasingly being explored as an assessment modality. This study sought to develop and collate validity evidence for a novel simulation-based assessment of operative competence. We describe the approach to assessment design, development, pilot testing, and validity investigation.
Methods
Eight procedural stations were generated using both virtual reality and bio-hybrid models. Content was identified from a previously conducted Delphi consensus study of trainers. Trainee performance was scored using an equally weighted Objective Structured Assessment of Technical Skills (OSATS) tool and a modified Procedure-Based Assessment (PBA) tool. Validity evidence was analyzed in accordance with Messick’s validity framework. Both ‘junior’ (ST2–ST4) and ‘senior’ trainees (ST 5–ST8) were included to allow for comparative analysis.
Results
Thirteen trainees were assessed by ten assessors across eight stations. Inter-station reliability was high ( α = 0.81), and inter-rater reliability was acceptable (inter-class correlation coefficient 0.77). A significant difference in mean station score was observed between junior and senior trainees (44.82 vs 58.18, p = .004), while overall mean scores were moderately correlated with increasing training year (rs = .74, p = .004, Kendall’s tau-b .57, p = 0.009). A pass-fail score generated using borderline regression methodology resulted in all ‘senior’ trainees passing and 4/6 of junior trainees failing the assessment.
Conclusion
This study reports validity evidence for a novel simulation-based assessment, designed to assess the operative competence of higher specialist trainees in general surgery. Graphical abstract
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, 121 St. Stephen's Green, Dublin, Ireland. conortoale@rcsi.com.
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, 121 St. Stephen's Green, Dublin, Ireland.
- SIM Centre for Simulation Education and Research, Royal College of Surgeons in Ireland, 123 St. Stephen's Green, Dublin, Ireland.
ORCID-Profile: Conor Toale
Förderung
- Royal College of Surgeons in Ireland/ Hermitage Medical Clinic Strategic Academic Recruitment (StAR MD) programme
- Royal College of Surgeons in Ireland
Zitieren
APA
Toale, C., Morris, M., Roche, A., Voborsky, M., Traynor, O., & Kavanagh, D. (2024). Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery. Surgical Endoscopy. https://doi.org/10.1007/s00464-024-11024-1
Vancouver
Toale C, Morris M, Roche A, Voborsky M, Traynor O, Kavanagh D. Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery. Surgical Endoscopy. 2024. doi:10.1007/s00464-024-11024-1
BibTeX
@article{Toale2024,
author = {Toale, Conor and Morris, Marie and Roche, Adam and Voborsky, Miroslav and Traynor, Oscar and Kavanagh, Dara},
title = {Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery},
journal = {Surgical Endoscopy},
year = {2024},
doi = {10.1007/s00464-024-11024-1}
}
EUVIMED