EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparative Evaluation of Large Language Models in Clinical Diagnostics for Real-World Medical Cases

Michał Młodawski, Stanisław Deniziak, Mirosław Płaza, Aleksandra Kwiatkowska, Dawid Jaszczyk, Arkadiusz Gajda, Karol Twardowski

Applied Sciences · 03.04.2026 · 16 (7) · 3499

DOI: 10.3390/app16073499 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 03.04.2026. Verfasst von Michał Młodawski und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Centre for Research and Development.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Aim: Although large language models (LLMs) achieve >90% accuracy on medical multiple-choice questions, their actual diagnostic utility remains unproven. This study compared four approaches to general medical diagnostics using real clinical data and LLMs: standalone LLM, LLM with Retrieval-Augmented Generation (RAG), LLM with expert system, and full integration of LLM, RAG, and expert system.

Methods

Twenty LLMs were tested on 1655 unpublished clinical cases (paediatrics and internal medicine) in Polish and English, yielding 264,800 diagnostic evaluations. TOP-1, TOP-3, and grouped International Classification of Diseases, 10th Revision (ICD-10) accuracy were measured.

Results

Standalone LLMs achieved only 16–20% TOP-1 accuracy. RAG added minimal benefit (+2–5 percentage points). The expert system improved performance 2.5-fold (55% paediatrics, 39% internal medicine). Unexpectedly, combining all components reduced results compared to the expert system alone, revealing an “integration paradox”.

Conclusions

LLMs alone are insufficient for clinical diagnostics in practical applications with incomplete data. Developers of clinical decision support systems should not treat LLMs as standalone diagnostic engines. Expert systems based on machine learning algorithms provide better support and should serve as the primary component in hybrid architectures. Combining LLM, RAG, and expert systems without deliberate output weighting paradoxically reduces performance. Hybrid systems should implement dynamic source selection or prediction weighting mechanisms rather than simple integration.

Förderung

  • National Centre for Research and Development

Zitieren

APA

Młodawski, M., Deniziak, S., Płaza, M., Kwiatkowska, A., Jaszczyk, D., Gajda, A., & Twardowski, K. (2026). Comparative Evaluation of Large Language Models in Clinical Diagnostics for Real-World Medical Cases. Applied Sciences, 16(7), 3499. https://doi.org/10.3390/app16073499

Vancouver

Młodawski M, Deniziak S, Płaza M, Kwiatkowska A, Jaszczyk D, Gajda A, et al. Comparative Evaluation of Large Language Models in Clinical Diagnostics for Real-World Medical Cases. Applied Sciences. 2026;16(7):3499. doi:10.3390/app16073499

BibTeX
@article{Modawski2026,
  author = {Młodawski, Michał and Deniziak, Stanisław and Płaza, Mirosław and Kwiatkowska, Aleksandra and Jaszczyk, Dawid and Gajda, Arkadiusz and Twardowski, Karol},
  title = {Comparative Evaluation of Large Language Models in Clinical Diagnostics for Real-World Medical Cases},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {7},
  pages = {3499},
  doi = {10.3390/app16073499}
}
Kostenlos starten →