Large language models vs thrombosis experts: a comparative study on patient education and clinical decision-making in venous thromboembolism
Journal of Thrombosis and Haemostasis · 2026
DOI: 10.1016/j.jtha.2025.09.004 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Thrombosis and Haemostasis, 2026. Verfasst von Nikola Vladic und 7 weiteren Autor:innen. Beteiligte Einrichtung: Division of Hematology and Haemostaseology, Department of Medicine I, Medical University of Vienna, Vienna, Austria. Electronic address: https://x.co…. Hauptthemen laut MEDLINE: Clinical Decision-Making, Patient Education as Topic, Venous Thromboembolism, Decision Support Techniques. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Themen: Thromboseprophylaxe, Selbstmanagement bei chronischen Erkrankungen
Abstract (Originalsprache)
BACKGROUND: Large language models (LLMs) have demonstrated remarkable capabilities in various medical fields, yet their performance in thrombosis and hemostasis, particularly in patient education and complex clinical decision making, is unexplored. OBJECTIVES: We aimed to compare the quality of responses from LLMs vs thrombosis experts and assess clinician's ability to distinguish between them. METHODS: Three experts on thrombosis and hemostasis and 3 LLMs (Le Chat Pixtral Large, DeepSeek-R1, and ChatGPT-4.5) answered 3 patient education and 3 clinical decision-making queries. Thirty-seven physicians rated responses for adequacy (1 = very poor; 10 = excellent) and estimated their origin (1 = certainly LLM; 10 = certainly human). Mean differences were assessed via t-tests, medians via Wilcoxon tests, and correlations via Spearman test. All P values were adjusted with Bonferroni correction. RESULTS: LLMs provided significantly better patient education responses than experts. Mean adequacy score differences were as follows: Le Chat Pixtral Large +1.6 (95% CI, 1.3-2.0; P < .01), DeepSeek-R1 +1.7 (95% CI, 1.3-2.1; P < .001), and ChatGPT-4.5 +1.9 (95% CI, 1.6-2.3; P < .001). In clinical decision-making, DeepSeek-R1 outperformed experts (+1.4; 95% CI, 1.1-1.8; P < .001), whereas Le Chat Pixtral Large (-0.3; 95% CI, -0.8-0.1; P = .96) and ChatGPT-4.5 (+0.5; 95% CI, 0.0-0.9; P = .18), performed comparably with experts. Evaluators could not distinguish between expert (median, 6.0; IQR, 3.0-8.0) and LLM-generated responses (median, 6.0; IQR, 4.0-8.0). CONCLUSION: LLMs outperform experts in venous thromboembolism-related patient education and match or exceed them in clinical decision making, providing responses indistinguishable from experts. Although major barriers need to be addressed, LLMs have strong potential to support clinical management and patient education in the field of thrombosis and hemostasis.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Hematology and Haemostaseology, Department of Medicine I, Medical University of Vienna, Vienna, Austria. Electronic address: https://x.com/VladicNikola.
- Division of Hematology and Haemostaseology, Department of Medicine I, Medical University of Vienna, Vienna, Austria.
- Division of Hematology and Haemostaseology, Department of Medicine I, Medical University of Vienna, Vienna, Austria. Electronic address: https://x.com/IngridPabinger.
- Department of Medicine, University of Padua, Padua, Italy.
- Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA. Electronic address: https://linkedin.com/5590527b.
- Division of Hematology and Haemostaseology, Department of Medicine I, Medical University of Vienna, Vienna, Austria. Electronic address: https://x.com/Sa_Eichinger_MD.
Zitieren
APA
Vladic, N., Nopp, S., Pabinger, I., Ageno, W., Connors, J. M., Eichinger, S., Ay, C., & ClotGPT study group (2026). Large language models vs thrombosis experts: a comparative study on patient education and clinical decision-making in venous thromboembolism. Journal of Thrombosis and Haemostasis. https://doi.org/10.1016/j.jtha.2025.09.004
Vancouver
Vladic N, Nopp S, Pabinger I, Ageno W, Connors JM, Eichinger S, et al. Large language models vs thrombosis experts: a comparative study on patient education and clinical decision-making in venous thromboembolism. Journal of Thrombosis and Haemostasis. 2026. doi:10.1016/j.jtha.2025.09.004
BibTeX
@article{Vladic2026,
author = {Vladic, Nikola and Nopp, Stephan and Pabinger, Ingrid and Ageno, Walter and Connors, Jean M and Eichinger, Sabine and Ay, Cihan and ClotGPT study group, ClotGPT study group},
title = {Large language models vs thrombosis experts: a comparative study on patient education and clinical decision-making in venous thromboembolism},
journal = {Journal of Thrombosis and Haemostasis},
year = {2026},
doi = {10.1016/j.jtha.2025.09.004}
}
EUVIMED