Feasibility of Privacy-preserving Locally Hosted Language Models for Assessing Medical Student Reflective Writing
Medical Science Educator · 01.09.2026 · 36 (5) · 2157-2161
DOI: 10.1007/s40670-026-02865-7 ↗Kurz gefasst
Fachartikel in der Zeitschrift Medical Science Educator, online erschienen am 01.09.2026. Verfasst von Rick Rejeleene und 4 weiteren Autor:innen. Beteiligte Einrichtung: Cleveland Clinic Lerner College of Medicine at Case Western Reserve University, Cleveland, OH USA. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 18 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Reflective writing assessment in medical education is labor-intensive and raises data privacy concerns that preclude cloud-based AI solutions. We conducted a feasibility study using a locally-hosted, open-weight large language model (LLM) deployed on institutional servers to assess medical students’ reflective writing. The model evaluated 10 de-identified student narratives using two published rubrics across three independent passes, generating rubric-aligned scores and formative feedback. Scores were consistent across passes and rubrics with no statistically significant differences. Expert faculty rated LLM-generated feedback as meeting most quality criteria (mean 5.48/7). These findings support feasibility of a privacy-preserving, locally-hosted LLM for scalable reflective writing assessment.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Cleveland Clinic Lerner College of Medicine at Case Western Reserve University, Cleveland, OH USA.
- Department of Medical Education, Tufts University School of Medicine, 136 Harrison Ave., Boston, MA USA.
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN USA.
ORCID-Profile: Rick Rejeleene, Jessica N Byram, Richard M Frankel, J Harry Isaacson, Neil B Mehta
Zitieren
APA
Rejeleene, R., Byram, J. N., Frankel, R. M., Isaacson, J. H., & Mehta, N. B. (2026). Feasibility of Privacy-preserving Locally Hosted Language Models for Assessing Medical Student Reflective Writing. Medical Science Educator, 36(5), 2157-2161. https://doi.org/10.1007/s40670-026-02865-7
Vancouver
Rejeleene R, Byram JN, Frankel RM, Isaacson JH, Mehta NB. Feasibility of Privacy-preserving Locally Hosted Language Models for Assessing Medical Student Reflective Writing. Medical Science Educator. 2026;36(5):2157-2161. doi:10.1007/s40670-026-02865-7
BibTeX
@article{Rejeleene2026,
author = {Rejeleene, Rick and Byram, Jessica N and Frankel, Richard M and Isaacson, J Harry and Mehta, Neil B},
title = {Feasibility of Privacy-preserving Locally Hosted Language Models for Assessing Medical Student Reflective Writing},
journal = {Medical Science Educator},
year = {2026},
volume = {36},
number = {5},
pages = {2157--2161},
doi = {10.1007/s40670-026-02865-7}
}
EUVIMED