PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent
PancreaGuide zur Interpretation von CT-Berichten und perioperativer Beratung bei Pankreaskarzinom: Entwicklung und retrospektive Evaluation eines wissensgestützten LLM-Agenten
Journal of Medical Internet Research · 2026-11-09 · 28 · e101607-e101607
DOI: 10.2196/101607 ↗Auf einen Blick
Die Studie entwickelte PancreaGuide zur CT-gestützten Interpretation von Resektabilitätsinformationen und perioperativer Beratung bei Pankreaskarzinom. Nach Integration verbesserten sich die Kennwerte und Expertenbewertungen der 3 LLMs.
Fragestellung
Kann ein wissensaugmentierter LLM-Agent die Interpretation von CT-Berichten und die perioperative Beratung bei Pankreaskarzinom unterstützen?
Methode
Entwicklung und retrospektive Evaluation eines agentenbasierten Frameworks mit strukturiertem Prompting, Retrieval-augmented Generation und Wissensbasis; 444 Patient:innen; Vergleich von ChatGPT, DeepSeek und Gemini in Basis- und PancreaGuide-Version.
Ergebnisse
Bei 444 Patient:innen zeigten alle 3 Modelle nach Integration höhere Kennwerte. Gemini+ erreichte die höchste Accuracy von 87,4 % (95 %-KI 0,840–0,902) und Sensitivität von 95,9 % (95 %-KI 0,927–0,978).
Einschränkungen
Die CT-basierten Ergebnisse wurden als Übereinstimmung mit dem beobachteten chirurgischen Management bewertet, nicht als unabhängige anatomische Resektabilität. PancreaGuide ersetzt keine radiologische Beurteilung.
Schlussfolgerung der Autor:innen
PancreaGuide bietet ein einheitliches, wissensaugmentiertes Rahmenwerk für CT-basierte Interpretation und perioperative Informationsunterstützung; es soll formale Radiologie, multidisziplinäre Bewertung und klinisches Urteil nicht ersetzen.
Kernaussagen
- 444 Patient:innen mit Kontrast-CT wurden retrospektiv analysiert
- Gemini+ hatte die höchste Accuracy und Sensitivität
- Expertenratings stiegen in allen 5 Bewertungsdimensionen
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
At a glance
Journal article in Journal of Medical Internet Research, published online on 2026-11-09. Written by Tianrui Kuang and 6 co-authors. Institution: Division of Pancreatic Surgery, Department of General Surgery and Laboratory of Pancreatic Cancer, West China Hospital of Sichuan University, No. 37,…. Major MEDLINE topics: Large Language Models, Pancreatic Neoplasms, Tomography, X-Ray Computed, Referral and Consultation. 35 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Accurate interpretation of pancreatic cancer status and resectability-related information, together with effective perioperative communication, remains a major challenge in pancreatic cancer care. In routine practice, unstructured computed tomography (CT) reports and complex clinical information may limit the accessibility and consistency of clinical information delivery and increase the communication burden on clinicians. Large language models (LLMs) offer new opportunities for clinical information support; however, existing applications are often task-specific and rarely combine CT report interpretation with perioperative consultation within a unified patient-oriented framework.
Objective
This study aimed to develop and retrospectively evaluate PancreaGuide, a knowledge-augmented LLM agent designed to support clinician-supervised CT report-based interpretation of pancreatic cancer resectability-related information and standardized perioperative consultation.
Methods
We developed PancreaGuide, an agent-based framework integrating structured prompting, retrieval-augmented generation, and a domain-specific pancreatic surgery knowledge base to support clinical information interpretation and perioperative consultation. A retrospective cohort of 444 patients who underwent contrast-enhanced CT at West China Hospital between 2023 and 2025 was analyzed. Three LLMs, ChatGPT, DeepSeek, and Gemini, were evaluated in both base and PancreaGuide-integrated configurations. CT report-based interpretation of resectability-related information was evaluated using accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F 1 -score, receiver operating characteristic curves (ROCs), and decision curve analysis (DCA) using a surgery-based proxy reference. Perioperative consultation quality was evaluated across 5 dimensions: accuracy, completeness, consistency, safety, and robustness.
Results
Higher classification metric point estimates were observed after integration of the PancreaGuide framework across all 3 backbone models. In CT report-based interpretation of resectability-related information, Gemini+ achieved the highest accuracy of 87.4% (95% CI 0.840‐0.902) and a sensitivity of 95.9% (95% CI 0.927‐0.978). All models showed improved sensitivity and specificity estimates after PancreaGuide integration, with improved agreement between model-generated interpretations and observed surgical management status. ROC and DCA suggested improved discrimination and estimated net benefit using the surgery-based proxy reference. In perioperative consultation, PancreaGuide-integrated models received higher expert-rated scores than their corresponding base models across all 5 evaluation dimensions. Gemini+ achieved the highest overall score (mean 3.88, SD 0.38), followed by ChatGPT+ (mean 3.8, SD 10.36) and DeepSeek+ (mean 3.75, SD 0.35). The greatest improvements were observed in completeness and robustness, with consistent gains across clinical consultation domains.
Conclusions
PancreaGuide provides a unified, knowledge-augmented framework for clinician-supervised CT report-based interpretation of resectability-related information and perioperative information support across multiple LLM backbones. The CT-based findings should be interpreted as agreement with observed surgical management status rather than validation against an independently adjudicated anatomical resectability. PancreaGuide may serve as an auxiliary information interpretation framework to improve the accessibility and consistency of clinical information delivery, while it should not replace formal radiological assessment, multidisciplinary evaluation, or clinician judgment.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Division of Pancreatic Surgery, Department of General Surgery and Laboratory of Pancreatic Cancer, West China Hospital of Sichuan University, No. 37, Guoxue Lane, Wuhou District, CHENGDU, China, 86 17786132270.
- Center of Gerontology and Geriatrics, National Clinical Research Center for Geriatrics, West China Hospital of Sichuan University, Chengdu, China.
Cite
APA
Kuang, T., Wang, R., Zhang, S., Sun, Y., Zhou, J., Wu, Z., & Peng, B. (2026). PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent. Journal of Medical Internet Research, 28, e101607-e101607. https://doi.org/10.2196/101607
Vancouver
Kuang T, Wang R, Zhang S, Sun Y, Zhou J, Wu Z, et al. PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent. Journal of Medical Internet Research. 2026;28:e101607-e101607. doi:10.2196/101607
BibTeX
@article{Kuang2026,
author = {Kuang, Tianrui and Wang, Rui and Zhang, Shaomin and Sun, Yunwei and Zhou, Jin and Wu, Zhong and Peng, Bing},
title = {PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e101607--e101607},
doi = {10.2196/101607}
}
EUVIMED