EUVIMEDEUVIMED entdecken
Scoping ReviewOpen Access · CC BY 4.0

Clinical Plausibility in Large Language Model Robustness Testing for Medicine: A Scoping Review

Clinical Plausibility in Large Language Model Robustness Testing for Medicine: Eine Scoping Review

Yu Chang, Ming-Hong Hsieh, Po-Chung Ju, Yi-Chun Liu, Cheng-Chen Chang

Journal of Medical Systems · 11.05.2026 · 50 (1)

DOI: 10.1007/s10916-026-02405-1 ↗

Auf einen Blick

Die Scoping Review kartierte Robustheitstests von LLMs in der Medizin. Häufig wurden irreführende und adversariale Prompts genutzt; klinisch plausible Szenarien waren seltener.

Fragestellung

Welche Methoden werden für Robustheitstests von LLMs im medizinischen Kontext genutzt, und wie klinisch plausibel sind sie?

Methode

Scoping Review nach PRISMA-ScR; Suche in PubMed, Embase, Web of Science, IEEE Xplore, ACM Digital Library, arXiv und MedRxiv von Januar 2023 bis September 2025.

Ergebnisse

33 Studien erfüllten die Kriterien, überwiegend aus 2025 (82%). Häufig waren misleading prompts (49 %) und adversarial prompts (39 %); 33 % bildeten plausible klinische Szenarien klar nach.

Schlussfolgerung der Autor:innen

Die Literatur zu Robustheitstests von LLMs in der Medizin fokussiert laut Autor:innen eher technische Schwachstellen als klinisch plausible Routinetests.

Kernaussagen

  • 33 % der Studien mit klinisch plausiblen Szenarien
  • Misleading prompts: 49 %; adversarial prompts: 39 %
  • 58 % berichteten Expert:innenbeteiligung

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Scoping Review in der Zeitschrift Journal of Medical Systems, online erschienen am 11.05.2026. Verfasst von Yu Chang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychiatry, Chung Shan Medical University Hospital, Taichung, 40106, Taiwan. Hauptthemen laut MEDLINE: Large Language Models. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Chung Shan Medical University.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) show promise in medical applications, yet their translation into clinical practice requires rigorous validation. Current robustness testing often employs adversarial approaches borrowed from AI safety, raising questions about their alignment with authentic clinical scenarios. To systematically map methodologies used for robustness testing of LLMs in medical contexts and assess their clinical plausibility. A scoping review was conducted following PRISMA-ScR guidelines, searching PubMed, Embase, Web of Science, IEEE Xplore, ACM Digital Library, arXiv, and MedRxiv from January 2023 to September 2025. Two independent physician reviewers screened 5,331 articles, extracting data on testing methodologies, medical domains, expert involvement, and clinical plausibility. Thirty-three studies met inclusion criteria, predominantly from 2025 (82%). The most common robustness testing approaches were misleading prompts (49%) and adversarial prompts (39%). Only 33% of studies designed tests clearly mimicking plausible clinical scenarios. While 58% reported expert involvement, the depth of integration varied considerably. Studies predominantly addressed mixed medical domains (73%) rather than specialized fields. The emerging literature suggests that LLM robustness testing in medicine often emphasizes technical vulnerability detection, with fewer studies examining clinically plausible scenarios of routine use. Future frameworks should complement adversarial testing with clinically grounded, longitudinal, and specialty focused evaluations to support deployment-relevant inference.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsHumans

Stichwörter der Autor:innen

Clinical decision support systemClinical plausibilityLarge language modelRed teamingRobustness testing

Autor:innen und Institutionen

  • Department of Psychiatry, Chung Shan Medical University Hospital, Taichung, 40106, Taiwan.
  • School of Medicine, Chung Shan Medical University, No. 110, Jianguo N. Rd., Sec. 1, Taichung City, 40106, Taiwan.
  • Post Baccalaureate Medicine, National Chung Hsing University, Taichung, Taiwan.
  • Department of Psychiatry, Changhua Christian Hospital, No.135, Nanhsiao Street, Changhua, 500, Taiwan.
  • Department of Psychiatry, Changhua Christian Children's Hospital, Changhua, Taiwan.
  • Department of Health Business Administration, Hungkuang University, Taichung City, Taiwan.

Förderung

  • Chung Shan Medical University

Zitieren

APA

Chang, Y., Hsieh, M. H., Ju, P. C., Liu, Y. C., & Chang, C. C. (2026). Clinical Plausibility in Large Language Model Robustness Testing for Medicine: A Scoping Review. Journal of Medical Systems, 50(1). https://doi.org/10.1007/s10916-026-02405-1

Vancouver

Chang Y, Hsieh MH, Ju PC, Liu YC, Chang CC. Clinical Plausibility in Large Language Model Robustness Testing for Medicine: A Scoping Review. Journal of Medical Systems. 2026;50(1). doi:10.1007/s10916-026-02405-1

BibTeX
@article{Chang2026,
  author = {Chang, Yu and Hsieh, Ming-Hong and Ju, Po-Chung and Liu, Yi-Chun and Chang, Cheng-Chen},
  title = {Clinical Plausibility in Large Language Model Robustness Testing for Medicine: A Scoping Review},
  journal = {Journal of Medical Systems},
  year = {2026},
  volume = {50},
  number = {1},
  doi = {10.1007/s10916-026-02405-1}
}
Kostenlos starten →