EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Evaluating Multiple Input Strategies of Large Language Models for Gallbladder Polyps on Ultrasound: Comparative Study

Lin Jiang iD, Jiaqian Yao iD, Zebang Yang iD, Fuqiu Tang iD, Xin Zheng iD, Xiaoer Zhang iD, Xiaoyan Xie iD, Ming Xu iD, Tongyi Huang iD

JMIR Medical Informatics · 2025

DOI: 10.2196/71178 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Informatics, 2025. Verfasst von Lin Jiang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, The First Affiliated Hospital, Sun Yat-sen University, Guan…. Hauptthemen laut MEDLINE: Polyps, Gallbladder, Gallbladder Neoplasms, Gallbladder Diseases. Die Arbeit nennt 56 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Gallbladder polyps have a high prevalence and are predominantly benign lesions, often detected via ultrasound. They impose diagnostic burdens on radiologists while generating substantial patient demand for report interpretation. Benign polyps include nonneoplastic polyps without malignant potential and premalignant adenomas that require cholecystectomy. Current guidelines recommending surgery for polyps ≥1.0 cm may lead to unnecessary interventions. Advanced multimodal large language models (LLMs) such as ChatGPT-4o (OpenAI) and Claude 3.5 Sonnet (Anthropic PBC) demonstrate emerging capabilities in medical image analysis. Implementing LLMs in gallbladder polyp ultrasound evaluation can potentially alleviate radiologists’ workload, provide patient-accessible consultation platforms, and even reduce overtreatment.

Objective

We aimed to analyze the feasibility and conduct an early-stage evaluation of using LLMs for differentiating between adenomatous and nonneoplastic gallbladder polyps (≥1.0 cm) based on ChatGPT-4o and Claude 3.5 Sonnet, compared to assessments by radiologists and the guideline.

Methods

Ultrasound images and reports of gallbladder polyps ≥1.0 cm with pathology were retrospectively collected from a hospital between January 2011 and January 2022. LLM performance was evaluated using three input strategies: (1) direct image analysis (LLMs-image), (2) feature-based text analysis (LLMs-text), and (3) scoring model-based text analysis (LLMs-model). Both intra- and interreader agreement and diagnostic performance of LLMs were evaluated for all three strategies. The diagnostic performance metrics—including sensitivity, specificity, accuracy, area under the receiver operating characteristic curve, and unnecessary resection rate of nonneoplastic polyps of LLMs in the three strategies were compared with the guideline. Additionally, the strategy LLMs-model was specifically compared with radiologists using the same scoring system (strategy readers-model).

Results

This study included 223 patients (aged 18-72 years; 132/223, 59.2% female) as the initial cohort, with 48 adenomatous polyps and 175 nonneoplastic polyps. The external test set comprised 100 patients. The intrareader agreement coefficients for strategy LLMs-model were significantly higher than those for strategy LLMs-image and LLMs-text (all PLLMs-text>LLMs-image. The sensitivity of strategies LLMs-image and LLMs-text was significantly lower than that of the guideline (all P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PolypsGallbladderGallbladder NeoplasmsGallbladder DiseasesNatural Language ProcessingHumansUltrasonographyFemaleMaleMiddle AgedRetrospective StudiesAgedAdultLarge Language Models

Stichwörter der Autor:innen

artificial intelligencediagnostic accuracygallbladder polypslarge language modelsmedical image analysisultrasound

Autor:innen und Institutionen

  • Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China.

ORCID-Profile: Lin Jiang, Jiaqian Yao, Zebang Yang, Fuqiu Tang, Xin Zheng, Xiaoer Zhang, Xiaoyan Xie, Ming Xu, Tongyi Huang

Zitieren

APA

Jiang, L., Yao, J., Yang, Z., Tang, F., Zheng, X., Zhang, X., Xie, X., Xu, M., & Huang, T. (2025). Evaluating Multiple Input Strategies of Large Language Models for Gallbladder Polyps on Ultrasound: Comparative Study. JMIR Medical Informatics. https://doi.org/10.2196/71178

Vancouver

Jiang L, Yao J, Yang Z, Tang F, Zheng X, Zhang X, et al. Evaluating Multiple Input Strategies of Large Language Models for Gallbladder Polyps on Ultrasound: Comparative Study. JMIR Medical Informatics. 2025. doi:10.2196/71178

BibTeX
@article{Jiang2025,
  author = {Jiang, Lin and Yao, Jiaqian and Yang, Zebang and Tang, Fuqiu and Zheng, Xin and Zhang, Xiaoer and Xie, Xiaoyan and Xu, Ming and Huang, Tongyi},
  title = {Evaluating Multiple Input Strategies of Large Language Models for Gallbladder Polyps on Ultrasound: Comparative Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  doi = {10.2196/71178}
}
Kostenlos starten →