EUVIMEDEUVIMED entdecken
Systematische ÜbersichtOpen Access

Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review

Anwendungen und Bedenken von ChatGPT und anderen konversationellen Large Language Models im Gesundheitswesen: systematische Übersichtsarbeit

Leyao Wang iD, Zhiyu Wan iD, Congning Ni iD, Qingyuan Song iD, Yang Li iD, Ellen Clayton iD, Bradley Malin iD, Zhijun Yin iD

Journal of Medical Internet Research · 07.11.2024 · 26 · e22769

DOI: 10.2196/22769 ↗

Auf einen Blick

Die Übersichtsarbeit fasst Anwendungen und Bedenken konversationeller LLMs im Gesundheitswesen zusammen. ChatGPT dominierte; besonders häufig wurden Zusammenfassung und medizinische Wissensabfrage beschrieben.

Fragestellung

Welche Anwendungen und welche Bedenken zu konversationellen Large Language Models im Gesundheitswesen werden in der Literatur beschrieben?

Methode

Systematische Übersichtsarbeit nach PRISMA; Suche in PubMed, ACM und IEEE-Datenbanken nach peer-reviewten Arbeiten vor dem 1. September 2023.

Ergebnisse

65 von 820 Papern erfüllten die Kriterien. 49 Arbeiten betrafen Zusammenfassung oder medizinische Wissensabfrage; 58 äußerten Bedenken zu Reliabilität oder Bias. Für komplexe Aufgaben wie Diagnose waren die Antworten nicht immer zuverlässig.

Schlussfolgerung der Autor:innen

Künftige Studien sollten die Zuverlässigkeit von LLM-Anwendungen bei komplexen gesundheitsbezogenen Aufgaben verbessern und die Mechanismen von Bias- und Privacy-Problemen untersuchen.

Kernaussagen

  • 65 von 820 Papern wurden eingeschlossen
  • ChatGPT war das häufigste Modell (60/65, 92 %)
  • Häufige Bedenken: Reliabilität, Bias, Privacy, Akzeptanz

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Systematische Übersicht in der Zeitschrift Journal of Medical Internet Research, online erschienen am 07.11.2024. Verfasst von Leyao Wang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Vanderbilt University, Nashville, TN, United States. Hauptthemen laut MEDLINE: Delivery of Health Care. Die Arbeit wurde bisher 197-mal zitiert (Crossref) und nennt 100 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCI NIH HHS, NHGRI NIH HHS.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The launch of ChatGPT (OpenAI) in November 2022 attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including health care. Numerous studies have since been conducted regarding how to use state-of-the-art LLMs in health-related scenarios.

Objective

This review aims to summarize applications of and concerns regarding conversational LLMs in health care and provide an agenda for future research in this field.

Methods

We used PubMed, ACM, and the IEEE digital libraries as primary sources for this review. We followed the guidance of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) to screen and select peer-reviewed research articles that (1) were related to health care applications and conversational LLMs and (2) were published before September 1, 2023, the date when we started paper collection. We investigated these papers and classified them according to their applications and concerns.

Results

Our search initially identified 820 papers according to targeted keywords, out of which 65 (7.9%) papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT (60/65, 92% of papers), followed by Bard (Google LLC; 1/65, 2% of papers), LLaMA (Meta; 1/65, 2% of papers), and other LLMs (6/65, 9% papers). These papers were classified into four categories of applications: (1) summarization, (2) medical knowledge inquiry, (3) prediction (eg, diagnosis, treatment recommendation, and drug synergy), and (4) administration (eg, documentation and information collection), and four categories of concerns: (1) reliability (eg, training data quality, accuracy, interpretability, and consistency in responses), (2) bias, (3) privacy, and (4) public acceptability. There were 49 (75%) papers using LLMs for either summarization or medical knowledge inquiry, or both, and there are 58 (89%) papers expressing concerns about either reliability or bias, or both. We found that conversational LLMs exhibited promising results in summarization and providing general medical knowledge to patients with a relatively high accuracy. However, conversational LLMs such as ChatGPT are not always able to provide reliable answers to complex health-related tasks (eg, diagnosis) that require specialized domain expertise. While bias or privacy issues are often noted as concerns, no experiments in our reviewed papers thoughtfully examined how conversational LLMs lead to these issues in health care research.

Conclusions

Future studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications bring bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in health care.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Delivery of Health CareHumansLanguageCommunication

Stichwörter der Autor:innen

ChatGPTartificial intelligencebiashealth carelarge language modelmedical knowledge inquirynatural language processingprivacyreliabilitysummarization

Autor:innen und Institutionen

  • Department of Computer Science, Vanderbilt University, Nashville, TN, United States.
  • Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States.
  • School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.
  • Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, United States.
  • School of Law, Vanderbilt University Medical Center, Nashville, TN, United States.
  • Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States.

ORCID-Profile: Leyao Wang, Zhiyu Wan, Congning Ni, Qingyuan Song, Yang Li, Ellen Clayton, Bradley Malin, Zhijun Yin

Förderung

  • NCI NIH HHS
  • NHGRI NIH HHS

Zitieren

APA

Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E., Malin, B., & Yin, Z. (2024). Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review. Journal of Medical Internet Research, 26, e22769. https://doi.org/10.2196/22769

Vancouver

Wang L, Wan Z, Ni C, Song Q, Li Y, Clayton E, et al. Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review. Journal of Medical Internet Research. 2024;26:e22769. doi:10.2196/22769

BibTeX
@article{Wang2024,
  author = {Wang, Leyao and Wan, Zhiyu and Ni, Congning and Song, Qingyuan and Li, Yang and Clayton, Ellen and Malin, Bradley and Yin, Zhijun},
  title = {Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e22769},
  doi = {10.2196/22769}
}
Kostenlos starten →