Validity of constructed‐response situational judgement tests in health professions education: A systematic review and meta‐analysis
Medical Education · 2026-12 · 60 (12) · 1319-1331
DOI: 10.1111/medu.70245 ↗Kurz gefasst
Systematische Übersicht in der Zeitschrift Medical Education, online erschienen am 04.06.2026. Verfasst von Alexander MacIntosh und 7 weiteren Autor:innen. Beteiligte Einrichtung: Research, Acuity Insights Inc., Toronto, Ontario, Canada. Hauptthemen laut MEDLINE: Judgment, School Admission Criteria, Health Occupations, Educational Measurement. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Constructed‐response situational judgement tests (CR‐SJTs) are used internationally to assess personal and professional attributes in health professions admissions, with over one million applicants to more than 500 programs having used them in the last decade. Despite this, a synthesis of their validity is lacking. This study aimed to quantify the association between CR‐SJT scores and measures of personal, interpersonal and professional performance in health professions education and to determine how moderating factors influence this relationship.
Methods
MEDLINE, EMBASE, CINAHL, ERIC, SCOPUS, Web of Science and grey literature sources (ProQuest Dissertations & Theses, EThOS and OpenGrey) were searched from inception to 7 April 2025. Search was supplemented by contacting experts and admissions directors to include unpublished quality assurance studies. Eligible studies evaluated a CR‐SJT for applicants or trainees in health professions programmes and reported a quantitative relationship with a non‐academic outcome (e.g., professionalism and communication). Of 463 full‐texts reviewed, 27 met inclusion criteria. Following the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines, two reviewers independently extracted data and assessed risk of bias using the Quality In Prognosis Studies (QUIPS) tool. A multilevel random‐effects meta‐analysis was used to pool Fisher z‐transformed correlation coefficients. Meta‐regressions tested moderator effects, and sensitivity analyses examined bias impact. The primary outcome was the correlation between CR‐SJT scores and measures of interpersonal or professional skills. Moderators included the construct congruence between the CR‐SJT and the outcome measure, outcome type, publication type, and outcome assessment stage.
Results
The 27 studies yielded 100 unique effect sizes and were judged, in total, to be at moderate risk of bias. The pooled correlation between CR‐SJT scores and outcomes was z = 0.22 (95% CI, 0.16–0.28; p < 0.001). Construct congruence was the only significant moderator; more congruent outcomes showed z = 0.32 (95% CI, 0.25–0.37), compared to that of less congruent outcomes ( z = 0.17; 95% CI, 0.12–0.18). Publication bias was insignificant (Egger's test, p = 0.73).
Discussion
The use of CR‐SJTs in health professions selection is supported by the evidence. Validity depends substantially on construct congruence between the CR‐SJT and the outcome measure. Programs should consider CR‐SJTs within their operational context and with deliberate attention to downstream evaluation alignment.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Research, Acuity Insights Inc., Toronto, Ontario, Canada.
- Applied Modelling and Quantitative Methods, Trent University, Peterborough, Ontario, Canada.
- Department of Psychology, Emotion and Health Research Laboratory, Trent University, Peterborough, Ontario, Canada.
- Department of Physician Assistant Studies, University of Wisconsin-La Crosse, La Crosse, Wisconsin, USA.
- Admissions and Academic Practice Partnerships, Physician Assistant Program, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA.
- School of Medicine, Imperial College London, London, UK.
ORCID-Profile: Alexander MacIntosh, Colin Henning, Safea Altef, Ivy Erickson, Sobia Shariff Hussaini, Elizabeth Morgan, Yoon Soo Park, Amir H Sam
Zitieren
APA
MacIntosh, A., Henning, C., Altef, S., Erickson, I., Hussaini, S. S., Morgan, E., Park, Y. S., & Sam, A. H. (2026). Validity of constructed‐response situational judgement tests in health professions education: A systematic review and meta‐analysis. Medical Education, 60(12), 1319-1331. https://doi.org/10.1111/medu.70245
Vancouver
MacIntosh A, Henning C, Altef S, Erickson I, Hussaini SS, Morgan E, et al. Validity of constructed‐response situational judgement tests in health professions education: A systematic review and meta‐analysis. Medical Education. 2026;60(12):1319-1331. doi:10.1111/medu.70245
BibTeX
@article{MacIntosh2026,
author = {MacIntosh, Alexander and Henning, Colin and Altef, Safea and Erickson, Ivy and Hussaini, Sobia Shariff and Morgan, Elizabeth and Park, Yoon Soo and Sam, Amir H},
title = {Validity of constructed‐response situational judgement tests in health professions education: A systematic review and meta‐analysis},
journal = {Medical Education},
year = {2026},
volume = {60},
number = {12},
pages = {1319--1331},
doi = {10.1111/medu.70245}
}
EUVIMED