EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance Assessment of ChatGPT versus Bard in Detecting Alzheimer’s Dementia

Balamurali B T iD, Jer-Ming Chen iD

Diagnostics · 15.04.2024 · 14 (8) · 817

DOI: 10.3390/diagnostics14080817 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 15.04.2024. Verfasst von Balamurali B T und 1 weiteren Autor:innen. Beteiligte Einrichtung: Science, Mathematics & Technology (SMT), Singapore University of Technology & Design, 8 Somapah Rd, Singapore 487372, Singapore. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) find increasing applications in many fields. Here, three LLM chatbots (ChatGPT-3.5, ChatGPT-4, and Bard) are assessed in their current form, as publicly available, for their ability to recognize Alzheimer’s dementia (AD) and Cognitively Normal (CN) individuals using textual input derived from spontaneous speech recordings. A zero-shot learning approach is used at two levels of independent queries, with the second query (chain-of-thought prompting) eliciting more detailed information than the first. Each LLM chatbot’s performance is evaluated on the prediction generated in terms of accuracy, sensitivity, specificity, precision, and F1 score. LLM chatbots generated a three-class outcome (“AD”, “CN”, or “Unsure”). When positively identifying AD, Bard produced the highest true-positives (89% recall) and highest F1 score (71%), but tended to misidentify CN as AD, with high confidence (low “Unsure” rates); for positively identifying CN, GPT-4 resulted in the highest true-negatives at 56% and highest F1 score (62%), adopting a diplomatic stance (moderate “Unsure” rates). Overall, the three LLM chatbots can identify AD vs. CN, surpassing chance-levels, but do not currently satisfy the requirements for clinical application.

Schlagwörter

Stichwörter der Autor:innen

Alzheimer’s dementiaBardChatGPTGPT-3.5GPT-4Large Language Modelschain-of-thoughtchatbotsecological diagnostic screeningspontaneous speechzero-shot learning

Autor:innen und Institutionen

  • Science, Mathematics & Technology (SMT), Singapore University of Technology & Design, 8 Somapah Rd, Singapore 487372, Singapore.

ORCID-Profile: Balamurali B T, Jer-Ming Chen

Zitieren

APA

B T, B., & Chen, J. M. (2024). Performance Assessment of ChatGPT versus Bard in Detecting Alzheimer’s Dementia. Diagnostics, 14(8), 817. https://doi.org/10.3390/diagnostics14080817

Vancouver

B T B, Chen JM. Performance Assessment of ChatGPT versus Bard in Detecting Alzheimer’s Dementia. Diagnostics. 2024;14(8):817. doi:10.3390/diagnostics14080817

BibTeX
@article{BT2024,
  author = {B T, Balamurali and Chen, Jer-Ming},
  title = {Performance Assessment of ChatGPT versus Bard in Detecting Alzheimer’s Dementia},
  journal = {Diagnostics},
  year = {2024},
  volume = {14},
  number = {8},
  pages = {817},
  doi = {10.3390/diagnostics14080817}
}
Kostenlos starten →