EUVIMEDAvasta EUVIMED
Systematische ÜbersichtOpen access

Effectiveness, Safety, and Workflow Burden of Large Language Model–Based Medical Report Generation: Systematic Review

Wirksamkeit, Sicherheit und Workflow-Belastung der medizinischen Berichtsgenerierung auf Basis großer Sprachmodelle: systematische Übersichtsarbeit

Jie-Lin Huang, Ji-Qing Zhu, Xiao-Guang Ni

Journal of Medical Internet Research · 2026-09-09 · 28 · e97007-e97007

DOI: 10.2196/97007 ↗

Auf einen Blick

101 Studien untersuchten LLMs, multimodale LLMs und Vision-Language-Modelle zur medizinischen Berichtsgenerierung. Die Evidenz zu Akzeptanz, Sicherheit und Workflow war heterogen und meist nicht poolbar.

Fragestellung

Wie wirksam, sicher und arbeitsintensiv ist die LLM-basierte medizinische Berichtsgenerierung in Bildgebungs- und verwandten klinischen Workflows?

Methode

Systematische Übersichtsarbeit; Suche in PubMed/MEDLINE, Embase, Web of Science Core Collection, Scopus und Cochrane Library von 1. Januar 2016 bis 15. Mai 2026; narrative Bewertung mit GRADE-Orientierung.

Ergebnisse

101 Studien. Keine Studie hatte geringes Verzerrungsrisiko; 15 moderat, 72 hoch, 14 schwer. In einer Chest-x-ray-Studie war die Akzeptanz ähnlich, die falsch-negativen Befunde leicht höher; Workfloweffekte waren gemischt.

Einschränkungen

Keine Metaanalyse; keine vergleichbaren Outcomes mit mindestens 2 Studien und analysierbaren Daten; Evidenz heterogen, verzerrt und spärlich auf Fallniveau.

Schlussfolgerung der Autor:innen

Die Evidenz spricht für einen möglichen Nutzen in ausgewählten, überwachten Settings, reicht aber nicht für klinische Autonomie. Die Autor:innen fordern lokale Validierung und standardisierte Berichterstattung.

Kernaussagen

  • Akzeptanz und Präferenz deuteten auf Nutzen in ausgewählten überwachten Settings hin
  • Sicherheits- und Workflowdaten waren uneinheitlich und meist nicht poolbar
  • Keine Metaanalyse wegen fehlender vergleichbarer, analysierbarer Outcomes

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Systematische Übersicht in Journal of Medical Internet Research, published online on 2026-09-09. Written by Jie-Lin Huang and 2 co-authors. Institution: Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and…. Major MEDLINE topics: Large Language Models, Workflow. 131 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstraktSubjectsBibliograafilised andmedCiteRelated studies

Abstract (original language)

Background

Systems based on large language models (LLMs), multimodal LLMs, and vision-language foundation models are increasingly being evaluated for medical report generation in imaging and related clinical workflows. Existing reviews have summarized technical architectures, radiology applications, readability, and benchmark performance, but clinical readiness remains uncertain because safety, human oversight, and workflow outcomes are sparsely and inconsistently reported.

Objective

The aim of this study is to assess the effectiveness (expert acceptance and blinded preference), safety (clinically significant, omission, and commission errors), and workflow burden (reporting time, corrections, edit distance, and editing burden) of LLM-based medical report generation.

Methods

We searched PubMed/MEDLINE, Embase, Web of Science Core Collection, Scopus, and the Cochrane Library for studies published from January 1, 2016, through May 15, 2026. Eligible studies evaluated LLMs, multimodal LLMs, or vision-language foundation models for image-to-report generation, impression generation from findings, report drafting, or structured reporting in imaging workflows. Two reviewers performed screening, extraction, risk-of-bias assessment, and Grading of Recommendations Assessment, Development, and Evaluation–informed narrative certainty assessment. Outcomes were clinically significant error rate, omission error rate, commission error rate, reporting time, edit burden, expert acceptance, and blinded expert preference. Meta-analysis was not performed because no comparable outcome had at least 2 studies with compatible task structure and analyzable data.

Results

A total of 101 studies were included. Chest x-ray was the largest modality group (36 studies), followed by computed tomography, magnetic resonance imaging (MRI), ultrasound, endoscopy, pathology, ophthalmic, electrocardiographic, dental, and mixed-modality contexts. No study was judged at low risk of bias; 15 were moderate, 72 high, and 14 serious. Safety and workflow evidence remained heterogeneous and largely nonpoolable. In a chest x-ray study, AI report acceptance was similar to that of radiologist reports (6047/8580, 70.5% vs 6288/8580, 73.3%), but false-negative findings were slightly higher (1584/8580, 18.5% vs 1527/8580, 17.8%). In a clinician-collaboration chest x-ray study, AI reports were equivalent or preferred in 233 of 300 (77.7%) and 170 of 303 (56.1%) cases across 2 datasets; yet, clinically significant errors persisted. In a brain MRI study, AI assistance reduced reading time from 61 to 53 seconds, whereas impression drafting increased editing time and edit distance.

Conclusions

This review shifts the synthesis from plausible report generation to clinically interpretable effectiveness, safety, and workflow effects. Expert acceptance and preference suggested assistive value in selected supervised settings, but these signals were limited by inconsistent reporting of clinically significant errors, omissions, commissions, and failed generations. Workflow effects were mixed, with some studies reporting shorter reading time or drafting support, and others reporting greater editing time or edit distance. The evidence remains too heterogeneous, biased, and sparse on case-level end points to support a pooled meta-analysis or autonomous clinical-readiness claims. Adoption should remain locally validated, clinician-supervised, and accompanied by standardized reporting of acceptance, preference, omissions, commissions, failed generations, reporting time, corrections, and editing burden.

Subjects

MeSH terms (MEDLINE)

Large Language ModelsWorkflowHumans

Author keywords

endoscopygenerative AIlarge language modelsmedical report generationpathologyradiologysystematic reviewworkflow

Authors and institutions

  • Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan South Lane, Chaoyang…

Cite

APA

Huang, J. L., Zhu, J. Q., & Ni, X. G. (2026). Effectiveness, Safety, and Workflow Burden of Large Language Model–Based Medical Report Generation: Systematic Review. Journal of Medical Internet Research, 28, e97007-e97007. https://doi.org/10.2196/97007

Vancouver

Huang JL, Zhu JQ, Ni XG. Effectiveness, Safety, and Workflow Burden of Large Language Model–Based Medical Report Generation: Systematic Review. Journal of Medical Internet Research. 2026;28:e97007-e97007. doi:10.2196/97007

BibTeX
@article{Huang2026,
  author = {Huang, Jie-Lin and Zhu, Ji-Qing and Ni, Xiao-Guang},
  title = {Effectiveness, Safety, and Workflow Burden of Large Language Model–Based Medical Report Generation: Systematic Review},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e97007--e97007},
  doi = {10.2196/97007}
}
Start for free →