Developing and evaluating a rubric to assess medical students’ analytic thinking and predict academic risk
Academic Medicine · 14.10.2026
DOI: 10.1093/acamed/wvag324 ↗Kurz gefasst
Fachartikel in der Zeitschrift Academic Medicine, online erschienen am 14.10.2026. Verfasst von Krisztina Fischer und 6 weiteren Autor:innen. Beteiligte Einrichtung: Master of Medical Sciences in Medical Education Program, Harvard Medical School, Boston, MA, United States. Die Arbeit nennt 10 Literaturangaben.
Abstract (Originalsprache)
Problem Analytic reasoning requires learners to integrate knowledge, demonstrate analytical skills, and employ metacognition to avoid cognitive biases. Mechanism Maps (MMs), student-generated concept maps, support this process by requiring students to link clinical findings with pathophysiologic mechanisms. However, broader adoption of MMs has been limited by challenges in assessment: existing scoring approaches emphasize structural features rather than accuracy of reasoning, show variable interrater reliability, and are time-intensive. Importantly, it remains unclear whether MM-based assessments can identify students at risk of difficulty in analytic reasoning before examination performance. Approach The authors developed a five-domain rubric (comprehensiveness, analytical detail, organization, accuracy, overall quality) to assess MMs. First-year medical students (n = 134) in a systems-based physiology/pathophysiology course at Harvard Medical School in February–March 2019 created three MMs for clinical vignettes as ungraded exercises. Each MM was assigned one week before a course assessment. Outcomes Rater training used an initial set of MMs, after which four trained raters, working in pairs, independently scored the remaining maps. Scoring showed good-to-excellent agreement between raters across most domains, with lowest inter-rater agreement for the accuracy domain. MM scores were associated with later examination performance. For Vignette 2, each 1-point increase in score roughly doubled the odds of scoring in the top three examination quartiles rather than the bottom quartile; this association held even after accounting for quiz performance. Vignette 3 scores showed a similar pattern, though the association was attenuated after adjustment for quiz performance. Next Steps The multidimensional rubric enabled reliable MM scoring, associated with examination performance, and suggests potential early identification of lower-performing students. Future work should consider prospective validation in additional institutional contexts.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Master of Medical Sciences in Medical Education Program, Harvard Medical School, Boston, MA, United States.
- Department of Radiology, Brigham and Women's Hospital, Boston, MA, United States.
- Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, United States.
- Harvard Medical School, Boston, MA, United States.
- Department of Medical Education, Western Atlantic University School of Medicine, Freeport, Bahamas.
- Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States.
ORCID-Profile: Krisztina Fischer, Alexander A Iyer, Jeremy B Richards, Johannah Mitchell, Amy M Sullivan, Richard M Schwartzstein
Zitieren
APA
Fischer, K., Furfaro, D., Iyer, A. A., Richards, J. B., Mitchell, J., Sullivan, A. M., & Schwartzstein, R. M. (2026). Developing and evaluating a rubric to assess medical students’ analytic thinking and predict academic risk. Academic Medicine. https://doi.org/10.1093/acamed/wvag324
Vancouver
Fischer K, Furfaro D, Iyer AA, Richards JB, Mitchell J, Sullivan AM, et al. Developing and evaluating a rubric to assess medical students’ analytic thinking and predict academic risk. Academic Medicine. 2026. doi:10.1093/acamed/wvag324
BibTeX
@article{Fischer2026,
author = {Fischer, Krisztina and Furfaro, David and Iyer, Alexander A and Richards, Jeremy B and Mitchell, Johannah and Sullivan, Amy M and Schwartzstein, Richard M},
title = {Developing and evaluating a rubric to assess medical students’ analytic thinking and predict academic risk},
journal = {Academic Medicine},
year = {2026},
doi = {10.1093/acamed/wvag324}
}
EUVIMED