EUVIMEDEUVIMED entdecken
Crossref

Generative artificial intelligence as a source of breast cancer information for patients: Proceed with caution

Ko Un Park iD, Stuart Lipsitz, Laura S Dominici, Filipa Lynce iD, Christina A Minami, Faina Nakhlis, Adrienne G Waks, Laura E Warren, Nadine Eidman, Jeannie Frazier, Lourdes Hernandez, Carla Leslie und 5 weitere

Cancer · 2025

DOI: 10.1002/cncr.35521 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cancer, 2025. Verfasst von Ko Un Park und 16 weiteren Autor:innen. Beteiligte Einrichtung: Division of Breast Surgery, Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts, USA. Hauptthemen laut MEDLINE: Breast Neoplasms, Artificial Intelligence. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 27 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

This study evaluated the accuracy, clinical concordance, and readability of the chatbot interface generative pretrained transformer (ChatGPT) 3.5 as a source of breast cancer information for patients.

Methods

Twenty questions that patients are likely to ask ChatGPT were identified by breast cancer advocates. These were posed to ChatGPT 3.5 in July 2023 and were repeated three times. Responses were graded in two domains: accuracy (4‐point Likert scale, 4 = worst) and clinical concordance (information is clinically similar to physician response; 5‐point Likert scale, 5 = not similar at all). The concordance of responses with repetition was estimated using intraclass correlation coefficient (ICC) of word counts. Response readability was calculated using the Flesch Kincaid readability scale. References were requested and verified.

Results

The overall average accuracy was 1.88 (range 1.0–3.0; 95% confidence interval [CI], 1.42–1.94), and clinical concordance was 2.79 (range 1.0–5.0; 95% CI, 1.94–3.64). The average word count was 310 words per response (range, 146–441 words per response) with high concordance (ICC, 0.75; 95% CI, 0.59–0.91; p < .001). The average readability was poor at 37.9 (range, 18.0–60.5) with high concordance (ICC, 0.73; 95% CI, 0.57–0.90; p < .001). There was a weak correlation between ease of readability and better clinical concordance (−0.15; p = .025). Accuracy did not correlate with readability (0.05; p = .079). The average number of references was 1.97 (range, 1–4; total, 119). ChatGPT cited peer‐reviewed articles only once and often referenced nonexistent websites (41%).

Conclusions

Because ChatGPT 3.5 responses were incorrect 24% of the time and did not provide real references 41% of the time, patients should be cautioned about using ChatGPT for medical information.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Breast NeoplasmsArtificial IntelligenceHumansFemaleComprehensionPatient Education as TopicHealth Literacy

Stichwörter der Autor:innen

artificial intelligencebreast cancerchatbot interface generative pretrained transformer (ChatGPT)health care information

Autor:innen und Institutionen

  • Division of Breast Surgery, Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts, USA.
  • Breast Oncology Program, Dana-Farber Brigham Cancer Center, Boston, Massachusetts, USA.
  • Harvard Medical School, Boston, Massachusetts, USA.
  • Center for Surgery and Public Health, Brigham and Women's Hospital, Boston, Massachusetts, USA.
  • Medical Oncology, Dana-Farber Cancer Institute, Boston, Massachusetts, USA.
  • Radiation Oncology, Dana-Farber Brigham Cancer Center, Boston, Massachusetts, USA.

ORCID-Profile: Ko Un Park, Filipa Lynce, Tari A King

Zitieren

APA

Park, K. U., Lipsitz, S., Dominici, L. S., Lynce, F., Minami, C. A., Nakhlis, F., Waks, A. G., Warren, L. E., Eidman, N., Frazier, J., Hernandez, L., Leslie, C., Rafte, S., Stroud, D., Weissman, J. S., King, T. A., & Mittendorf, E. A. (2025). Generative artificial intelligence as a source of breast cancer information for patients: Proceed with caution. Cancer. https://doi.org/10.1002/cncr.35521

Vancouver

Park KU, Lipsitz S, Dominici LS, Lynce F, Minami CA, Nakhlis F, et al. Generative artificial intelligence as a source of breast cancer information for patients: Proceed with caution. Cancer. 2025. doi:10.1002/cncr.35521

BibTeX
@article{Park2025,
  author = {Park, Ko Un and Lipsitz, Stuart and Dominici, Laura S and Lynce, Filipa and Minami, Christina A and Nakhlis, Faina and Waks, Adrienne G and Warren, Laura E and Eidman, Nadine and Frazier, Jeannie and Hernandez, Lourdes and Leslie, Carla and Rafte, Susan and Stroud, Delia and Weissman, Joel S and King, Tari A and Mittendorf, Elizabeth A},
  title = {Generative artificial intelligence as a source of breast cancer information for patients: Proceed with caution},
  journal = {Cancer},
  year = {2025},
  doi = {10.1002/cncr.35521}
}
Kostenlos starten →