EUVIMEDEUVIMED entdecken
Leitlinie

Locally deployed context-aware chatbot outperforms generic large language models for guideline-concordant pediatric imaging recommendations

Amit Gupta, Krithika Rangarajan, R G Krishna Kumar, Somil Anshal

Pediatric Radiology · 2025

DOI: 10.1007/s00247-025-06453-6 ↗

Kurz gefasst

Leitlinie in der Zeitschrift Pediatric Radiology, 2025. Verfasst von Amit Gupta und 3 weiteren Autor:innen. Beteiligte Einrichtung: Room No 48, Department of Diagnostic and Interventional Onco-radiology, Dr BRAIRCH, All India Institute of Medical Sciences, Ansari Nagar, New Delhi,…. Hauptthemen laut MEDLINE: Pediatrics, Guideline Adherence, Practice Guidelines as Topic, Diagnostic Imaging. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 22 Literaturangaben. Gefördert von: Ministry of Education, India.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Accurate modality selection in pediatric imaging is critical, yet adherence to the American College of Radiology (ACR) Appropriateness Criteria remains limited. Large language models (LLMs) offer potential as decision support tools but often lack domain-specific accuracy. OBJECTIVE: To evaluate the performance of a locally run, context-aware chatbot (ped-Llama) based on an open-source LLM for providing personalized pediatric imaging recommendations grounded in ACR guidelines. MATERIALS AND METHODS: A simulation-based study was conducted using 50 pediatric clinical scenarios derived from ACR guideline variants. The ped-Llama chatbot, built using a retrieval-augmented generation (RAG) approach with a locally deployed Llama-3.1-8B model, was compared against three generic LLMs (Llama-3.1-8B without RAG, Generative pre-trained transformer-4o [GPT-4o], Claude Opus) and three radiologists (junior resident, senior resident, specialist pediatric radiologist). Each provided imaging recommendations, which were classified as "Usually appropriate," "May be appropriate," or "Usually not appropriate" per ACR guidelines. Modal outputs were analyzed, and consistency across three independent LLM runs was assessed. RESULTS: ped-Llama achieved 80% (40/50) "Usually appropriate" recommendations, outperforming Llama-3.1-8B (46%), GPT-4o (54%), and Claude Opus (46%), and matching the specialist radiologist (76%). When both "Usually appropriate" and "May be appropriate" were accepted, ped-Llama achieved 90% accuracy. Consistency across runs was 72% for ped-Llama versus 44-50% for generic LLMs. CONCLUSION: This study shows that a locally run, RAG-enabled chatbot based on an open-source LLM can provide guideline-concordant pediatric imaging recommendations with expert-level accuracy. Such systems offer a practical and scalable approach to artificial intelligence-assisted decision support in radiology.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PediatricsGuideline AdherencePractice Guidelines as TopicDiagnostic ImagingDecision Support Systems, ClinicalLarge Language ModelsHumansChildComputer Simulation

Stichwörter der Autor:innen

Artificial intelligenceClinical decision support systemsGuideline adherenceImaging appropriatenessLarge language models

Autor:innen und Institutionen

  • Room No 48, Department of Diagnostic and Interventional Onco-radiology, Dr BRAIRCH, All India Institute of Medical Sciences, Ansari Nagar, New Delhi, 110029, India. amit.aiims2014@gmail.com.
  • Room No 48, Department of Diagnostic and Interventional Onco-radiology, Dr BRAIRCH, All India Institute of Medical Sciences, Ansari Nagar, New Delhi, 110029, India.
  • Department of Radiodiagnosis and Interventional Radiology, All India Institute of Medical Sciences, New Delhi, India.

Förderung

  • Ministry of Education, India

Zitieren

APA

Gupta, A., Rangarajan, K., Krishna Kumar, R. G., & Anshal, S. (2025). Locally deployed context-aware chatbot outperforms generic large language models for guideline-concordant pediatric imaging recommendations. Pediatric Radiology. https://doi.org/10.1007/s00247-025-06453-6

Vancouver

Gupta A, Rangarajan K, Krishna Kumar RG, Anshal S. Locally deployed context-aware chatbot outperforms generic large language models for guideline-concordant pediatric imaging recommendations. Pediatric Radiology. 2025. doi:10.1007/s00247-025-06453-6

BibTeX
@article{Gupta2025,
  author = {Gupta, Amit and Rangarajan, Krithika and Krishna Kumar, R G and Anshal, Somil},
  title = {Locally deployed context-aware chatbot outperforms generic large language models for guideline-concordant pediatric imaging recommendations},
  journal = {Pediatric Radiology},
  year = {2025},
  doi = {10.1007/s00247-025-06453-6}
}
Kostenlos starten →