Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study
Journal of Medical Internet Research · 05.03.2025 · 27 · e64364
DOI: 10.2196/64364 ↗Kurz gefasst
Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 05.03.2025. Verfasst von Eliza Berman und 4 weiteren Autor:innen. Beteiligte Einrichtung: Center for Digital Health, University Hospital Tuebingen, Tuebingen, Germany. Hauptthemen laut MEDLINE: Neoplasms, Algorithms. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Molecular tumor boards (MTBs) require intensive manual investigation to generate optimal treatment recommendations for patients. Large language models (LLMs) can catalyze MTB recommendations, decrease human error, improve accessibility to care, and enhance the efficiency of precision oncology.
Objective
In this study, we aimed to investigate the efficacy of LLM-generated treatments for MTB patients. We specifically investigate the LLMs’ ability to generate evidence-based treatment recommendations using PubMed references.
Methods
We built a retrieval augmented generation pipeline using PubMed data. We prompted the resulting LLM to generate treatment recommendations with PubMed references using a test set of patients from an MTB conference at a large comprehensive cancer center at a tertiary care institution. Members of the MTB manually assessed the relevancy and correctness of the generated responses.
Results
A total of 75% of the referenced articles were properly cited from PubMed, while 17% of the referenced articles were hallucinations, and the remaining were not properly cited from PubMed. Clinician-generated LLM queries achieved higher accuracy through clinician evaluation than automated queries, with clinicians labeling 25% of LLM responses as equal to their recommendations and 37.5% as alternative plausible treatments.
Conclusions
This study demonstrates how retrieval augmented generation–enhanced LLMs can be a powerful tool in accelerating MTB conferences, as LLMs are sometimes capable of achieving clinician-equal treatment recommendations. However, further investigation is required to achieve stable results with zero hallucinations. LLMs signify a scalable solution to the time-intensive process of MTB investigations. However, LLM performance demonstrates that they must be used with heavy clinician supervision, and cannot yet fully automate the MTB pipeline.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Center for Digital Health, University Hospital Tuebingen, Tuebingen, Germany.
- Center for Personalized Medicine, University Hospital Tuebingen, Tuebingen, Germany.
- Department of Internal Medicine I, University Hospital Tuebingen, Tuebingen, Germany.
ORCID-Profile: Eliza Berman, Holly Sundberg Malek, Michael Bitzer, Nisar Malek, Carsten Eickhoff
Zitieren
APA
Berman, E., Sundberg Malek, H., Bitzer, M., Malek, N., & Eickhoff, C. (2025). Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study. Journal of Medical Internet Research, 27, e64364. https://doi.org/10.2196/64364
Vancouver
Berman E, Sundberg Malek H, Bitzer M, Malek N, Eickhoff C. Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study. Journal of Medical Internet Research. 2025;27:e64364. doi:10.2196/64364
BibTeX
@article{Berman2025,
author = {Berman, Eliza and Sundberg Malek, Holly and Bitzer, Michael and Malek, Nisar and Eickhoff, Carsten},
title = {Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e64364},
doi = {10.2196/64364}
}
EUVIMED