GastroGPT: Development and controlled testing of a proof-of-concept customized clinical language model
Endoscopy International Open · 06.08.2025 · 13 (CP)
DOI: 10.1055/a-2637-2163 ↗Kurz gefasst
Fachartikel in der Zeitschrift Endoscopy International Open, erschienen am 06.08.2025. Verfasst von Cem Simsek und 13 weiteren Autor:innen. Beteiligte Einrichtung: Gastroenterology & Hepatology, Johns Hopkins Medical Institutions Campus, Baltimore, United States. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Current general-purpose artificial intelligence (AI) large language models (LLMs) demonstrate limited efficacy in clinical medicine, often constrained to question-answering, documentation, and literature summarization roles. We developed GastroGPT, a proof-of-concept specialty-specific, multi-task, clinical LLM, and evaluated its performance against leading general-purpose LLMs across key gastroenterology tasks and diverse case scenarios. In this structured analysis, GastroGPT was compared with three state-of-the-art general-purpose LLMs (LLM-A: GPT-4, LLM-B: Bard, LLM-C: Claude). Models were assessed on seven clinical tasks and overall performance across 10 simulated gastroenterology cases varying in complexity, frequency, and patient demographics. Standardized prompts facilitated structured comparisons. A blinded expert panel rated model outputs per task on a 10-point Likert scale, judging clinical utility. Comprehensive statistical analyses were conducted. A total of 2,240 expert ratings were obtained. GastroGPT achieved significantly higher mean overall scores (8.1 ± 1.8) compared with GPT-4 (5.2 ± 3.0), Bard (5.7 ± 3.3), and Claude (7.0 ± 2.7) (all P < 0.001). It outperformed comparators in six of seven tasks (P < 0.05), except follow-up planning. GastroGPT demonstrated superior score consistency (variance 34.95) versus general models (97.4–260.35) (P < 0.001). Its performance remained consistent across case complexities and frequencies, unlike the comparators (P < 0.001). Multivariate analysis revealed that model type significantly predicted performance (P < 0.001). This study pioneered development and comparison of a specialty-specific, clinically-oriented AI model to general-purpose LLMs. GastroGPT demonstrated superior utility overall and on key gastroenterology tasks, highlighting the potential for tailored, task-focused AI models in medicine.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Gastroenterology & Hepatology, Johns Hopkins Medical Institutions Campus, Baltimore, United States.
- internal medicine, Hacettepe University Faculty of Medicine, Ankara, Turkey.
- Dr Balmis General University Hospital, Alicante, Spain.
- Division of Gastroenterology, Universitätsklinikum Augsburg, Augsburg, Germany.
- Palacky University Olomouc, Olomouc, Czech Republic.
- Liverpool University Hospitals NHS Foundation Trust, Liverpool, United Kingdom of Great Britain and Northern Ireland.
ORCID-Profile: Giulio Antonelli
Zitieren
APA
Simsek, C., Ucdal, M., de-Madaria, E., Ebigbo, A., Vanek, P., Elshaarawy, O., Voiosu, T. A., Antonelli, G., Turró, R., Gisbert, J. P., Nyssen, O. P., Hassan, C., Messmann, H., & Jalan, R. (2025). GastroGPT: Development and controlled testing of a proof-of-concept customized clinical language model. Endoscopy International Open, 13(CP). https://doi.org/10.1055/a-2637-2163
Vancouver
Simsek C, Ucdal M, de-Madaria E, Ebigbo A, Vanek P, Elshaarawy O, et al. GastroGPT: Development and controlled testing of a proof-of-concept customized clinical language model. Endoscopy International Open. 2025;13(CP). doi:10.1055/a-2637-2163
BibTeX
@article{Simsek2025,
author = {Simsek, Cem and Ucdal, Mete and de-Madaria, Enrique and Ebigbo, Alanna and Vanek, Petr and Elshaarawy, Omar and Voiosu, Theodor Alexandru and Antonelli, Giulio and Turró, Román and Gisbert, Javier P and Nyssen, Olga P and Hassan, Cesare and Messmann, Helmut and Jalan, Rajiv},
title = {GastroGPT: Development and controlled testing of a proof-of-concept customized clinical language model},
journal = {Endoscopy International Open},
year = {2025},
volume = {13},
number = {CP},
doi = {10.1055/a-2637-2163}
}
EUVIMED