EUVIMEDEUVIMED entdecken
Open Access

Leveraging a Large Language Model to Generate Quality Improvement Feedback for Clinical Notes

Christopher J Kim, Joseph Gelfinbein, Nihan Gencerliler, Nusrat Jahan, Jahnavi Udaikumar, Lauren M Heery, Adam Goodman, Sarah Ng, Joel Attard, Sharmin Asha, Jesse Burk-Rafel, Benedict V Guzman und 3 weitere

Applied Clinical Informatics · 01.03.2026 · 17 (02) · 252-258

DOI: 10.1055/a-2851-0739 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Clinical Informatics, erschienen am 01.03.2026. Verfasst von Christopher J Kim und 14 weiteren Autor:innen. Beteiligte Einrichtung: Division of Hospital Medicine, Department of Medicine, NYU Langone Health, New York, New York, United States. Hauptthemen laut MEDLINE: Large Language Models, Quality Improvement, Feedback, Documentation. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Poor documentation quality can significantly affect health care operations, but the feedback process for clinicians to improve clinical notes is time-consuming and often insufficient. Large language models (LLMs) such as Generative Pre-trained Transformer 4 (GPT-4) have the potential to streamline this process.

Objectives

This study aimed to determine whether an LLM can generate feedback to improve the medical contingency and discharge planning (MCDP) component of clinical documentation that is non-inferior to feedback by physicians.

Methods

A cross-sectional study of GPT-4 feedback and physician feedback on inpatient progress notes was conducted. A random sample of 64 inpatient progress notes identified by the validated artificial intelligence (AI) Audit Tool as having a low likelihood of containing MCDP was included from adult general medicine patients hospitalized at New York University Langone Health (NYULH) in December 2023. Both the GPT-4 model and attending physicians generated feedback on these inpatient progress notes. A/B testing was then conducted on the measures of understandability, usefulness, acceptability, and impartiality. Evaluations employed 5-point Likert scales that were converted to 10-point bidirectional interval scales for interpretability, ranging from −10 (human suggestions significantly better) to +10 (GPT-4 suggestions significantly better), with a non-inferiority threshold set to −1 for the primary endpoint.

Results

Sixty-four inpatient progress notes were included, representing 55% female patients with a median age of 73 years. GPT-4 feedback was non-inferior to physician feedback in all measures: Understandability (mean, 1.27, 95% CI: 0.73–1.8, p < 0.001), usefulness (mean, 2.09, 95% CI: 1.27–2.91, p < 0.001), acceptability (mean, 2.07, 95% CI: 1.33–2.81, p < 0.001), and impartiality (mean, −0.20, 95% CI: −0.52 to 0.12, p < 0.001).

Conclusion

This study shows that an LLM can be leveraged to generate note quality feedback that is non-inferior to expert clinician feedback.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsQuality ImprovementFeedbackDocumentationElectronic Health RecordsHumansCross-Sectional Studies

Autor:innen und Institutionen

  • Division of Hospital Medicine, Department of Medicine, NYU Langone Health, New York, New York, United States.
  • Department of Medicine, NYU Grossman Long Island School of Medicine, Mineola, New York, United States.
  • Department of Foundations of Medicine, NYU Grossman Long Island School of Medicine, Mineola, New York, United States.
  • Department of Medicine, NYU Grossman School of Medicine, New York, New York, United States.
  • Department of Health Informatics, Medical Center Information Technology, NYU Langone Health, New York, New York, United States.
  • Division of Gastroenterology and Hepatology, Department of Medicine, NYU Langone Health, New York, New York, United States.

Zitieren

APA

Kim, C. J., Gelfinbein, J., Gencerliler, N., Jahan, N., Udaikumar, J., Heery, L. M., Goodman, A., Ng, S., Attard, J., Asha, S., Burk-Rafel, J., Guzman, B. V., Hochman, K. A., Testa, P. A., & Feldman, J. (2026). Leveraging a Large Language Model to Generate Quality Improvement Feedback for Clinical Notes. Applied Clinical Informatics, 17(02), 252-258. https://doi.org/10.1055/a-2851-0739

Vancouver

Kim CJ, Gelfinbein J, Gencerliler N, Jahan N, Udaikumar J, Heery LM, et al. Leveraging a Large Language Model to Generate Quality Improvement Feedback for Clinical Notes. Applied Clinical Informatics. 2026;17(02):252-258. doi:10.1055/a-2851-0739

BibTeX
@article{Kim2026,
  author = {Kim, Christopher J and Gelfinbein, Joseph and Gencerliler, Nihan and Jahan, Nusrat and Udaikumar, Jahnavi and Heery, Lauren M and Goodman, Adam and Ng, Sarah and Attard, Joel and Asha, Sharmin and Burk-Rafel, Jesse and Guzman, Benedict V and Hochman, Katherine A and Testa, Paul A and Feldman, Jonah},
  title = {Leveraging a Large Language Model to Generate Quality Improvement Feedback for Clinical Notes},
  journal = {Applied Clinical Informatics},
  year = {2026},
  volume = {17},
  number = {02},
  pages = {252--258},
  doi = {10.1055/a-2851-0739}
}
Kostenlos starten →