EUVIMEDEUVIMED entdecken
Open Access

Leveraging Large Language Models for Decision Support in Personalized Oncology

Manuela Benary, Xing David Wang, Max Schmidt, Dominik Soll, Georg Hilfenhaus, Mani Nassir, Christian Sigler, Maren Knödler, Ulrich Keller, Dieter Beule, Ulrich Keilholz, Ulf Leser und 1 weitere

JAMA Network Open · 17.11.2023 · 6 (11) · e2343689

DOI: 10.1001/jamanetworkopen.2023.43689 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMA Network Open, online erschienen am 17.11.2023. Verfasst von Manuela Benary und 12 weiteren Autor:innen. Beteiligte Einrichtung: Charité Comprehensive Cancer Center, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Ber…. Hauptthemen laut MEDLINE: Precision Medicine, Lung Neoplasms. Die Arbeit wurde bisher 327-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Importance Clinical interpretation of complex biomarkers for precision oncology currently requires manual investigations of previous studies and databases. Conversational large language models (LLMs) might be beneficial as automated tools for assisting clinical decision-making.

Objective

To assess performance and define their role using 4 recent LLMs as support tools for precision oncology. Design, Setting, and Participants This diagnostic study examined 10 fictional cases of patients with advanced cancer with genetic alterations. Each case was submitted to 4 different LLMs (ChatGPT, Galactica, Perplexity, and BioMedLM) and 1 expert physician to identify personalized treatment options in 2023. Treatment options were masked and presented to a molecular tumor board (MTB), whose members rated the likelihood of a treatment option coming from an LLM on a scale from 0 to 10 (0, extremely unlikely; 10, extremely likely) and decided whether the treatment option was clinically useful. Main Outcomes and Measures Number of treatment options, precision, recall, F1 score of LLMs compared with human experts, recognizability, and usefulness of recommendations.

Results

For 10 fictional cancer patients (4 with lung cancer, 6 with other; median [IQR] 3.5 [3.0-4.8] molecular alterations per patient), a median (IQR) number of 4.0 (4.0-4.0) compared with 3.0 (3.0-5.0), 7.5 (4.3-9.8), 11.5 (7.8-13.0), and 13.0 (11.3-21.5) treatment options each was identified by the human expert and 4 LLMs, respectively. When considering the expert as a criterion standard, LLM-proposed treatment options reached F1 scores of 0.04, 0.17, 0.14, and 0.19 across all patients combined. Combining treatment options from different LLMs allowed a precision of 0.29 and a recall of 0.29 for an F1 score of 0.29. LLM-generated treatment options were recognized as AI-generated with a median (IQR) 7.5 (5.3-9.0) points in contrast to 2.0 (1.0-3.0) points for manually annotated cases. A crucial reason for identifying AI-generated treatment options was insufficient accompanying evidence. For each patient, at least 1 LLM generated a treatment option that was considered helpful by MTB members. Two unique useful treatment options (including 1 unique treatment strategy) were identified only by LLM. Conclusions and Relevance In this diagnostic study, treatment options of LLMs in precision oncology did not reach the quality and credibility of human experts; however, they generated helpful ideas that might have complemented established procedures. Considering technological progress, LLMs could play an increasingly important role in assisting with screening and selecting relevant biomedical literature to support evidence-based, personalized treatment decisions.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Precision MedicineLung NeoplasmsHumansMedical OncologyLanguageCommunication

Autor:innen und Institutionen

  • Charité Comprehensive Cancer Center, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany.
  • Core Unit Bioinformatics, Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Charitéplatz 1, Berlin, Germany.
  • Knowledge Management in Bioinformatics, Humboldt-Universität zu Berlin, Berlin, Germany.
  • Department of Hematology, Oncology and Cancer Immunology, Campus Benjamin Franklin, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany.
  • Department of Endocrinology and Metabolic Diseases, Charité Universitätsmedizin Berlin, Department of Endocrinology and Metabolic Diseases, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin…
  • Department of Hematology, Oncology and Cancer Immunology, Campus Charité Mitte, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany.

Zitieren

APA

Benary, M., Wang, X. D., Schmidt, M., Soll, D., Hilfenhaus, G., Nassir, M., Sigler, C., Knödler, M., Keller, U., Beule, D., Keilholz, U., Leser, U., & Rieke, D. T. (2023). Leveraging Large Language Models for Decision Support in Personalized Oncology. JAMA Network Open, 6(11), e2343689. https://doi.org/10.1001/jamanetworkopen.2023.43689

Vancouver

Benary M, Wang XD, Schmidt M, Soll D, Hilfenhaus G, Nassir M, et al. Leveraging Large Language Models for Decision Support in Personalized Oncology. JAMA Network Open. 2023;6(11):e2343689. doi:10.1001/jamanetworkopen.2023.43689

BibTeX
@article{Benary2023,
  author = {Benary, Manuela and Wang, Xing David and Schmidt, Max and Soll, Dominik and Hilfenhaus, Georg and Nassir, Mani and Sigler, Christian and Knödler, Maren and Keller, Ulrich and Beule, Dieter and Keilholz, Ulrich and Leser, Ulf and Rieke, Damian T},
  title = {Leveraging Large Language Models for Decision Support in Personalized Oncology},
  journal = {JAMA Network Open},
  year = {2023},
  volume = {6},
  number = {11},
  pages = {e2343689},
  doi = {10.1001/jamanetworkopen.2023.43689}
}
Kostenlos starten →