EUVIMEDEUVIMED entdecken
Crossref

Current Strengths and Weaknesses of ChatGPT as a Resource for Radiation Oncology Patients and Providers

Warren Floyd, Troy Kleber, David J Carpenter, Melisa Pasli, Jamiluddin Qazi, Christina Huang, Jim Leng, Bradley G Ackerson, Matthew Pierpoint, Joseph K Salama, Matthew J Boyer

International Journal of Radiation Oncology*Biology*Physics · 2024-03 · 118 (4) · 905-915

DOI: 10.1016/j.ijrobp.2023.10.020 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Radiation Oncology*Biology*Physics, 2024-03. Verfasst von Warren Floyd und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, Texas. Hauptthemen laut MEDLINE: Radiation Oncology, Natural Language Processing. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 28 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

PURPOSE: Chat Generative Pre-Trained Transformer (ChatGPT), an artificial intelligence program that uses natural language processing to generate conversational-style responses to questions or inputs, is increasingly being used by both patients and health care professionals. This study aims to evaluate the accuracy and comprehensiveness of ChatGPT in radiation oncology-related domains, including answering common patient questions, summarizing landmark clinical research studies, and providing literature reviews with specific references supporting current standard-of-care clinical practice in radiation oncology. METHODS AND MATERIALS: We assessed the performance of ChatGPT version 3.5 (ChatGPT3.5) in 3 areas. We evaluated ChatGPT3.5's ability to answer 28 templated patient-centered questions applied across 9 cancer types. We then tested ChatGPT3.5's ability to summarize specific portions of 10 landmark studies in radiation oncology. Next, we used ChatGPT3.5 to identify scientific studies supporting current standard-of-care practice in clinical radiation oncology for 5 different cancer types. Each response was graded independently by 2 reviewers, with discordant grades resolved by a third reviewer. RESULTS: ChatGPT3.5 frequently generated inaccurate or incomplete responses. Only 39.7% of responses to patient-centered questions were considered correct and comprehensive. When summarizing landmark studies in radiation oncology, 35.0% of ChatGPT3.5's responses were accurate and comprehensive, improving to 43.3% when provided the full text of the study. ChatGPT3.5's ability to present a list of studies related to standard-of-care clinical practices was also unsatisfactory, with 50.6% of the provided studies fabricated. CONCLUSIONS: ChatGPT should not be considered a reliable radiation oncology resource for patients or providers at this time, as it frequently generates inaccurate or incomplete responses. However, natural language programming-based artificial intelligence programs are rapidly evolving, and future versions of ChatGPT or similar programs may demonstrate improved performance in this domain.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Radiation OncologyNatural Language ProcessingHumansArtificial IntelligenceNeoplasmsPatient-Centered Care

Autor:innen und Institutionen

  • Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, Texas.
  • Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina; Radiation Oncology Clinical Service, Durham VA Health Care System, Durham, North Carolina.
  • Brody School of Medicine, East Carolina University, Greenville, North Carolina.
  • Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina.
  • Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina; Radiation Oncology Clinical Service, Durham VA Health Care System, Durham, North Carolina. Electronic address: Matthew.Boyer@…

Zitieren

APA

Floyd, W., Kleber, T., Carpenter, D. J., Pasli, M., Qazi, J., Huang, C., Leng, J., Ackerson, B. G., Pierpoint, M., Salama, J. K., & Boyer, M. J. (2024). Current Strengths and Weaknesses of ChatGPT as a Resource for Radiation Oncology Patients and Providers. International Journal of Radiation Oncology*Biology*Physics, 118(4), 905-915. https://doi.org/10.1016/j.ijrobp.2023.10.020

Vancouver

Floyd W, Kleber T, Carpenter DJ, Pasli M, Qazi J, Huang C, et al. Current Strengths and Weaknesses of ChatGPT as a Resource for Radiation Oncology Patients and Providers. International Journal of Radiation Oncology*Biology*Physics. 2024;118(4):905-915. doi:10.1016/j.ijrobp.2023.10.020

BibTeX
@article{Floyd2024,
  author = {Floyd, Warren and Kleber, Troy and Carpenter, David J and Pasli, Melisa and Qazi, Jamiluddin and Huang, Christina and Leng, Jim and Ackerson, Bradley G and Pierpoint, Matthew and Salama, Joseph K and Boyer, Matthew J},
  title = {Current Strengths and Weaknesses of ChatGPT as a Resource for Radiation Oncology Patients and Providers},
  journal = {International Journal of Radiation Oncology*Biology*Physics},
  year = {2024},
  volume = {118},
  number = {4},
  pages = {905--915},
  doi = {10.1016/j.ijrobp.2023.10.020}
}
Kostenlos starten →