There is no “too small” for frequent workplace-based assessment: Differences between large and small residency programs in anesthesia when using a mobile application to assess EPAs
GMS Journal for Medical Education · 15.11.2024 · 41 (5) · Doc54
DOI: 10.3205/zma001709 ↗Kurz gefasst
Fachartikel in der Zeitschrift GMS Journal for Medical Education, erschienen am 15.11.2024. Verfasst von Tobias Tessmann und 4 weiteren Autor:innen. Beteiligte Einrichtung: University Hospital Zurich, Institute of Anaesthesiology, Zurich, Switzerland. Hauptthemen laut MEDLINE: Internship and Residency, Mobile Applications, Clinical Competence, Educational Measurement. Die Arbeit wurde bisher 3-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
A competency-based education approach calls for frequent workplace-based assessments (WBA) of Entrustable Professional Activities (EPAs). While mobile applications increase the efficiency, it is not known how many assessments are required for reliable ratings and whether the concept can be implemented in all sizes of residency programs.
Methods
Over 5 months, a mobile app was used to assess 10 different EPAs in daily clinical routine in Swiss anesthesia departments. The data from large residency programs was compared to those from smaller ones. We applied generalizability theory and decision studies to estimate the minimum number of assessments needed for reliable assessments.
Results
From 28 residency programs, we included 3936 assessments by 306 supervisors for 295 residents. The median number of assessments per trainee was 8, with a median of 4 different EPAs assessed by 3 different supervisors. We found no statistically significant differences between large and small programs in the number of assessments per trainee, per supervisor, per EPA, the agreement between supervisors and trainees, and the number of feedback processes stimulated. The average “level of supervision” (LoS, scale from 1 to 5) recorded in larger programs was 3.2 (SD 0.5) compared to 2.7 (SD 0.4) (p 0.7, at least a random set of 3 different EPAs needed to be assessed, with each EPA rated at least 4 times by 4 different supervisors, resulting in a total of 12 assessments.
Conclusion
Frequent WBAs of EPAs were feasible in large and small residency programs. We found no significant differences in the number of assessments performed. The minimum number of assessments required for a g-coefficient >0.7 was attainable in large and small residency programs.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University Hospital Zurich, Institute of Anaesthesiology, Zurich, Switzerland.
- University Hospital Balgrist, Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Zurich, Switzerland.
- University of Bern, Institute for Medical Education, Bern, Switzerland.
Zitieren
APA
Tessmann, T., Marty, A. P., Stricker, D., Huwendiek, S., & Breckwoldt, J. (2024). There is no “too small” for frequent workplace-based assessment: Differences between large and small residency programs in anesthesia when using a mobile application to assess EPAs. GMS Journal for Medical Education, 41(5), Doc54. https://doi.org/10.3205/zma001709
Vancouver
Tessmann T, Marty AP, Stricker D, Huwendiek S, Breckwoldt J. There is no “too small” for frequent workplace-based assessment: Differences between large and small residency programs in anesthesia when using a mobile application to assess EPAs. GMS Journal for Medical Education. 2024;41(5):Doc54. doi:10.3205/zma001709
BibTeX
@article{Tessmann2024,
author = {Tessmann, Tobias and Marty, Adrian P and Stricker, Daniel and Huwendiek, Sören and Breckwoldt, Jan},
title = {There is no “too small” for frequent workplace-based assessment: Differences between large and small residency programs in anesthesia when using a mobile application to assess EPAs},
journal = {GMS Journal for Medical Education},
year = {2024},
volume = {41},
number = {5},
pages = {Doc54},
doi = {10.3205/zma001709}
}
EUVIMED