EUVIMEDEUVIMED entdecken
Scoping ReviewOpen Access

Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review

Lydie Bednarczyk iD, Daniel Reichenpfader iD, Christophe Gaudet-Blavignac iD, Amon Kenna Ette iD, Jamil Zaghir iD, Yuanyuan Zheng iD, Adel Bensahla iD, Mina Bjelogrlic iD, Christian Lovis iD

Journal of Medical Internet Research · 15.05.2025 · 27 · e68998

DOI: 10.2196/68998 ↗

Kurz gefasst

Scoping Review in der Zeitschrift Journal of Medical Internet Research, online erschienen am 15.05.2025. Verfasst von Lydie Bednarczyk und 8 weiteren Autor:innen. Beteiligte Einrichtung: Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland. Hauptthemen laut MEDLINE: Electronic Health Records, Natural Language Processing. Die Arbeit wurde bisher 99-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Information overload in electronic health records requires effective solutions to alleviate clinicians’ administrative tasks. Automatically summarizing clinical text has gained significant attention with the rise of large language models. While individual studies show optimism, a structured overview of the research landscape is lacking.

Objective

This study aims to present the current state of the art on clinical text summarization using large language models, evaluate the level of evidence in existing research and assess the applicability of performance findings in clinical settings.

Methods

This scoping review complied with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Literature published between January 1, 2019, and June 18, 2024, was identified from 5 databases: PubMed, Embase, Web of Science, IEEE Xplore, and ACM Digital Library. Studies were excluded if they did not describe transformer-based models, did not focus on clinical text summarization, did not engage with free-text data, were not original research, were nonretrievable, were not peer-reviewed, or were not in English, French, Spanish, or German. Data related to study context and characteristics, scope of research, and evaluation methodologies were systematically collected and analyzed by 3 authors independently.

Results

A total of 30 original studies were included in the analysis. All used observational retrospective designs, mainly using real patient data (n=28, 93%). The research landscape demonstrated a narrow research focus, often centered on summarizing radiology reports (n=17, 57%), primarily involving data from the intensive care unit (n=15, 50%) of US-based institutions (n=19, 73%), in English (n=26, 87%). This focus aligned with the frequent reliance on the open-source Medical Information Mart for Intensive Care dataset (n=15, 50%). Summarization methodologies predominantly involved abstractive approaches (n=17, 57%) on single-document inputs (n=4, 13%) with unstructured data (n=13, 43%), yet reporting on methodological details remained inconsistent across studies. Model selection involved both open-source models (n=26, 87%) and proprietary models (n=7, 23%). Evaluation frameworks were highly heterogeneous. All studies conducted internal validation, but external validation (n=2, 7%), failure analysis (n=6, 20%), and patient safety risks analysis (n=1, 3%) were infrequent, and none reported bias assessment. Most studies used both automated metrics and human evaluation (n=16, 53%), while 10 (33%) used only automated metrics, and 4 (13%) only human evaluation.

Conclusions

Key barriers hinder the translation of current research into trustworthy, clinically valid applications. Current research remains exploratory and limited in scope, with many applications yet to be explored. Performance assessments often lack reliability, and clinical impact evaluations are insufficient raising concerns about model utility, safety, fairness, and data privacy. Advancing the field requires more robust evaluation frameworks, a broader research scope, and a stronger focus on real-world applicability.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsNatural Language ProcessingHumansLarge Language Models

Stichwörter der Autor:innen

artificial intelligenceelectronic health recordshealth carelarge language modelsnatural language processingscoping reviewsummarizationtranslational research

Autor:innen und Institutionen

  • Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.
  • Institute for Patient-centered Digital Health, Bern University of Applied Sciences, Biel, Switzerland.
  • Faculty of Medicine, University of Geneva, Geneva, Switzerland.

ORCID-Profile: Lydie Bednarczyk, Daniel Reichenpfader, Christophe Gaudet-Blavignac, Amon Kenna Ette, Jamil Zaghir, Yuanyuan Zheng, Adel Bensahla, Mina Bjelogrlic, Christian Lovis

Zitieren

APA

Bednarczyk, L., Reichenpfader, D., Gaudet-Blavignac, C., Ette, A. K., Zaghir, J., Zheng, Y., Bensahla, A., Bjelogrlic, M., & Lovis, C. (2025). Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review. Journal of Medical Internet Research, 27, e68998. https://doi.org/10.2196/68998

Vancouver

Bednarczyk L, Reichenpfader D, Gaudet-Blavignac C, Ette AK, Zaghir J, Zheng Y, et al. Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review. Journal of Medical Internet Research. 2025;27:e68998. doi:10.2196/68998

BibTeX
@article{Bednarczyk2025,
  author = {Bednarczyk, Lydie and Reichenpfader, Daniel and Gaudet-Blavignac, Christophe and Ette, Amon Kenna and Zaghir, Jamil and Zheng, Yuanyuan and Bensahla, Adel and Bjelogrlic, Mina and Lovis, Christian},
  title = {Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e68998},
  doi = {10.2196/68998}
}
Kostenlos starten →