EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evolution of publicly available large language models for complex decision-making in breast cancer care

Sebastian Griewing iD, Johannes Knitza iD, Jelena Boekhoff, Christoph Hillen, Fabian Lechner, Uwe Wagner, Markus Wallwiener iD, Sebastian Kuhn iD

Archives of Gynecology and Obstetrics · 29.05.2024 · 310 (1) · 537-550

DOI: 10.1007/s00404-024-07565-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Archives of Gynecology and Obstetrics, online erschienen am 29.05.2024. Verfasst von Sebastian Griewing und 7 weiteren Autor:innen. Beteiligte Einrichtung: Institute for Digital Medicine, Philipps-University Marburg, Marburg, Germany. s.griewing@uni-marburg.de. Hauptthemen laut MEDLINE: Breast Neoplasms. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Philipps-Universität Marburg.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study investigated the concordance of five different publicly available Large Language Models (LLM) with the recommendations of a multidisciplinary tumor board regarding treatment recommendations for complex breast cancer patient profiles.

Methods

Five LLM, including three versions of ChatGPT (version 4 and 3.5, with data access until September 3021 and January 2022), Llama2, and Bard were prompted to produce treatment recommendations for 20 complex breast cancer patient profiles. LLM recommendations were compared to the recommendations of a multidisciplinary tumor board (gold standard), including surgical, endocrine and systemic treatment, radiotherapy, and genetic testing therapy options.

Results

GPT4 demonstrated the highest concordance (70.6%) for invasive breast cancer patient profiles, followed by GPT3.5 September 2021 (58.8%), GPT3.5 January 2022 (41.2%), Llama2 (35.3%) and Bard (23.5%). Including precancerous lesions of ductal carcinoma in situ, the identical ranking was reached with lower overall concordance for each LLM (GPT4 60.0%, GPT3.5 September 2021 50.0%, GPT3.5 January 2022 35.0%, Llama2 30.0%, Bard 20.0%). GPT4 achieved full concordance (100%) for radiotherapy. Lowest alignment was reached in recommending genetic testing, demonstrating a varying concordance (55.0% for GPT3.5 January 2022, Llama2 and Bard up to 85.0% for GPT4).

Conclusion

This early feasibility study is the first to compare different LLM in breast cancer care with regard to changes in accuracy over time, i.e., with access to more data or through technological upgrades. Methodological advancement, i.e., the optimization of prompting techniques, and technological development, i.e., enabling data input control and secure data processing, are necessary in the preparation of large-scale and multicenter studies to provide evidence on their safe and reliable clinical application. At present, safe and evidenced use of LLM in clinical breast cancer care is not yet feasible.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Breast NeoplasmsHumansFemaleClinical Decision-MakingDecision Support Techniques

Stichwörter der Autor:innen

Artificial intelligenceBreast cancerChatGPTLarge language modelsTumor board

Autor:innen und Institutionen

  • Institute for Digital Medicine, Philipps-University Marburg, Marburg, Germany. s.griewing@uni-marburg.de.
  • Department of Gynecology and Obstetrics, Philipps-University Marburg, Marburg, Germany. s.griewing@uni-marburg.de.
  • Kommission Digitale Medizin, Deutsche Gesellschaft für Gynäkologie und Geburtshilfe, Berlin, Germany. s.griewing@uni-marburg.de.
  • Institute for Digital Medicine, Philipps-University Marburg, Marburg, Germany.
  • Department of Gynecology and Obstetrics, Philipps-University Marburg, Marburg, Germany.
  • Department of Gynecology and Gynecologic Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.

ORCID-Profile: Sebastian Griewing, Johannes Knitza, Markus Wallwiener, Sebastian Kuhn

Förderung

  • Philipps-Universität Marburg

Zitieren

APA

Griewing, S., Knitza, J., Boekhoff, J., Hillen, C., Lechner, F., Wagner, U., Wallwiener, M., & Kuhn, S. (2024). Evolution of publicly available large language models for complex decision-making in breast cancer care. Archives of Gynecology and Obstetrics, 310(1), 537-550. https://doi.org/10.1007/s00404-024-07565-4

Vancouver

Griewing S, Knitza J, Boekhoff J, Hillen C, Lechner F, Wagner U, et al. Evolution of publicly available large language models for complex decision-making in breast cancer care. Archives of Gynecology and Obstetrics. 2024;310(1):537-550. doi:10.1007/s00404-024-07565-4

BibTeX
@article{Griewing2024,
  author = {Griewing, Sebastian and Knitza, Johannes and Boekhoff, Jelena and Hillen, Christoph and Lechner, Fabian and Wagner, Uwe and Wallwiener, Markus and Kuhn, Sebastian},
  title = {Evolution of publicly available large language models for complex decision-making in breast cancer care},
  journal = {Archives of Gynecology and Obstetrics},
  year = {2024},
  volume = {310},
  number = {1},
  pages = {537--550},
  doi = {10.1007/s00404-024-07565-4}
}
Kostenlos starten →