EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study

Eliza Berman iD, Holly Sundberg Malek iD, Michael Bitzer iD, Nisar Malek iD, Carsten Eickhoff iD

Journal of Medical Internet Research · 05.03.2025 · 27 · e64364

DOI: 10.2196/64364 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 05.03.2025. Verfasst von Eliza Berman und 4 weiteren Autor:innen. Beteiligte Einrichtung: Center for Digital Health, University Hospital Tuebingen, Tuebingen, Germany. Hauptthemen laut MEDLINE: Neoplasms, Algorithms. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Molecular tumor boards (MTBs) require intensive manual investigation to generate optimal treatment recommendations for patients. Large language models (LLMs) can catalyze MTB recommendations, decrease human error, improve accessibility to care, and enhance the efficiency of precision oncology.

Objective

In this study, we aimed to investigate the efficacy of LLM-generated treatments for MTB patients. We specifically investigate the LLMs’ ability to generate evidence-based treatment recommendations using PubMed references.

Methods

We built a retrieval augmented generation pipeline using PubMed data. We prompted the resulting LLM to generate treatment recommendations with PubMed references using a test set of patients from an MTB conference at a large comprehensive cancer center at a tertiary care institution. Members of the MTB manually assessed the relevancy and correctness of the generated responses.

Results

A total of 75% of the referenced articles were properly cited from PubMed, while 17% of the referenced articles were hallucinations, and the remaining were not properly cited from PubMed. Clinician-generated LLM queries achieved higher accuracy through clinician evaluation than automated queries, with clinicians labeling 25% of LLM responses as equal to their recommendations and 37.5% as alternative plausible treatments.

Conclusions

This study demonstrates how retrieval augmented generation–enhanced LLMs can be a powerful tool in accelerating MTB conferences, as LLMs are sometimes capable of achieving clinician-equal treatment recommendations. However, further investigation is required to achieve stable results with zero hallucinations. LLMs signify a scalable solution to the time-intensive process of MTB investigations. However, LLM performance demonstrates that they must be used with heavy clinician supervision, and cannot yet fully automate the MTB pipeline.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

NeoplasmsAlgorithmsHumansPubMed

Stichwörter der Autor:innen

LLMsLLaMAMTBaccessibility to careaugmented therapyclinical trialsevidence-basedlarge language modelsmolecular tumormolecular tumor boardoncologypatient careprecision oncologyretrieval augmented generationtreatment

Autor:innen und Institutionen

  • Center for Digital Health, University Hospital Tuebingen, Tuebingen, Germany.
  • Center for Personalized Medicine, University Hospital Tuebingen, Tuebingen, Germany.
  • Department of Internal Medicine I, University Hospital Tuebingen, Tuebingen, Germany.

ORCID-Profile: Eliza Berman, Holly Sundberg Malek, Michael Bitzer, Nisar Malek, Carsten Eickhoff

Zitieren

APA

Berman, E., Sundberg Malek, H., Bitzer, M., Malek, N., & Eickhoff, C. (2025). Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study. Journal of Medical Internet Research, 27, e64364. https://doi.org/10.2196/64364

Vancouver

Berman E, Sundberg Malek H, Bitzer M, Malek N, Eickhoff C. Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study. Journal of Medical Internet Research. 2025;27:e64364. doi:10.2196/64364

BibTeX
@article{Berman2025,
  author = {Berman, Eliza and Sundberg Malek, Holly and Bitzer, Michael and Malek, Nisar and Eickhoff, Carsten},
  title = {Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e64364},
  doi = {10.2196/64364}
}
Kostenlos starten →