EUVIMEDOpdag EUVIMED
ValidierungsstudieOpen access

Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study

Eliza Berman iD, Holly Sundberg Malek iD, Michael Bitzer iD, Nisar Malek iD, Carsten Eickhoff iD

Journal of Medical Internet Research · 2025-03-05 · 27 · e64364

DOI: 10.2196/64364 ↗

At a glance

Validierungsstudie in Journal of Medical Internet Research, published online on 2025-03-05. Written by Eliza Berman and 4 co-authors. Institution: Center for Digital Health, University Hospital Tuebingen, Tuebingen, Germany. Major MEDLINE topics: Neoplasms, Algorithms. Cited 18 times so far (Crossref); 39 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Molecular tumor boards (MTBs) require intensive manual investigation to generate optimal treatment recommendations for patients. Large language models (LLMs) can catalyze MTB recommendations, decrease human error, improve accessibility to care, and enhance the efficiency of precision oncology.

Objective

In this study, we aimed to investigate the efficacy of LLM-generated treatments for MTB patients. We specifically investigate the LLMs’ ability to generate evidence-based treatment recommendations using PubMed references.

Methods

We built a retrieval augmented generation pipeline using PubMed data. We prompted the resulting LLM to generate treatment recommendations with PubMed references using a test set of patients from an MTB conference at a large comprehensive cancer center at a tertiary care institution. Members of the MTB manually assessed the relevancy and correctness of the generated responses.

Results

A total of 75% of the referenced articles were properly cited from PubMed, while 17% of the referenced articles were hallucinations, and the remaining were not properly cited from PubMed. Clinician-generated LLM queries achieved higher accuracy through clinician evaluation than automated queries, with clinicians labeling 25% of LLM responses as equal to their recommendations and 37.5% as alternative plausible treatments.

Conclusions

This study demonstrates how retrieval augmented generation–enhanced LLMs can be a powerful tool in accelerating MTB conferences, as LLMs are sometimes capable of achieving clinician-equal treatment recommendations. However, further investigation is required to achieve stable results with zero hallucinations. LLMs signify a scalable solution to the time-intensive process of MTB investigations. However, LLM performance demonstrates that they must be used with heavy clinician supervision, and cannot yet fully automate the MTB pipeline.

Subjects

MeSH terms (MEDLINE)

NeoplasmsAlgorithmsHumansPubMed

Author keywords

LLMsLLaMAMTBaccessibility to careaugmented therapyclinical trialsevidence-basedlarge language modelsmolecular tumormolecular tumor boardoncologypatient careprecision oncologyretrieval augmented generationtreatment

Authors and institutions

  • Center for Digital Health, University Hospital Tuebingen, Tuebingen, Germany.
  • Center for Personalized Medicine, University Hospital Tuebingen, Tuebingen, Germany.
  • Department of Internal Medicine I, University Hospital Tuebingen, Tuebingen, Germany.

ORCID profiles: Eliza Berman, Holly Sundberg Malek, Michael Bitzer, Nisar Malek, Carsten Eickhoff

Cite

APA

Berman, E., Sundberg Malek, H., Bitzer, M., Malek, N., & Eickhoff, C. (2025). Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study. Journal of Medical Internet Research, 27, e64364. https://doi.org/10.2196/64364

Vancouver

Berman E, Sundberg Malek H, Bitzer M, Malek N, Eickhoff C. Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study. Journal of Medical Internet Research. 2025;27:e64364. doi:10.2196/64364

BibTeX
@article{Berman2025,
  author = {Berman, Eliza and Sundberg Malek, Holly and Bitzer, Michael and Malek, Nisar and Eickhoff, Carsten},
  title = {Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e64364},
  doi = {10.2196/64364}
}
Start for free →