Assessing the Role of the Generative Pretrained Transformer (GPT) in Alzheimer’s Disease Management: Comparative Study of Neurologist- and Artificial Intelligence–Generated Responses
Journal of Medical Internet Research · 2024
DOI: 10.2196/51095 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Medical Internet Research, 2024. Verfasst von Jiaqi Zeng und 11 weiteren Autor:innen. Beteiligte Einrichtung: West China Medical School, Sichuan University, Chengdu, China. Hauptthemen laut MEDLINE: Alzheimer Disease, Neurologists, Artificial Intelligence. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Alzheimer’s disease (AD) is a progressive neurodegenerative disorder posing challenges to patients, caregivers, and society. Accessible and accurate information is crucial for effective AD management.
Objective
This study aimed to evaluate the accuracy, comprehensibility, clarity, and usefulness of the Generative Pretrained Transformer’s (GPT) answers concerning the management and caregiving of patients with AD.
Methods
In total, 14 questions related to the prevention, treatment, and care of AD were identified and posed to GPT-3.5 and GPT-4 in Chinese and English, respectively, and 4 respondent neurologists were asked to answer them. We generated 8 sets of responses (total 112) and randomly coded them in answer sheets. Next, 5 evaluator neurologists and 5 family members of patients were asked to rate the 112 responses using separate 5-point Likert scales. We evaluated the quality of the responses using a set of 8 questions rated on a 5-point Likert scale. To gauge comprehensibility and participant satisfaction, we included 3 questions dedicated to each aspect within the same set of 8 questions.
Results
As of April 10, 2023, the 5 evaluator neurologists and 5 family members of patients with AD rated the 112 responses: GPT-3.5: n=28, 25%, responses; GPT-4: n=28, 25%, responses; respondent neurologists: 56 (50%) responses. The top 5 (4.5%) responses rated by evaluator neurologists had 4 (80%) GPT (GPT-3.5+GPT-4) responses and 1 (20%) respondent neurologist’s response. For the top 5 (4.5%) responses rated by patients’ family members, all but the third response were GPT responses. Based on the evaluation by neurologists, the neurologist-generated responses achieved a mean score of 3.9 (SD 0.7), while the GPT-generated responses scored significantly higher (mean 4.4, SD 0.6; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- West China Medical School, Sichuan University, Chengdu, China.
- Department of Neurology, West China Hospital, Sichuan University, Chengdu, China.
- Department of Neurology, Chengdu Shangjin Nanfu Hospital, Chengdu, China.
- Department of Neurology, Guizhou Provincial People's Hospital, Guiyang, China.
- Mental Health Center, West China Hospital, Sichuan University, Chengdu, China.
- West China College of Stomatology, Sichuan University, Chengdu, China.
ORCID-Profile: Jiaqi Zeng, Xiaoyi Zou, Shirong Li, Yao Tang, Sisi Teng, Huanhuan Li, Changyu Wang, Yuxuan Wu, Luyao Zhang, Yunheng Zhong, Jialin Liu, Siru Liu
Zitieren
APA
Zeng, J., Zou, X., Li, S., Tang, Y., Teng, S., Li, H., Wang, C., Wu, Y., Zhang, L., Zhong, Y., Liu, J., & Liu, S. (2024). Assessing the Role of the Generative Pretrained Transformer (GPT) in Alzheimer’s Disease Management: Comparative Study of Neurologist- and Artificial Intelligence–Generated Responses. Journal of Medical Internet Research. https://doi.org/10.2196/51095
Vancouver
Zeng J, Zou X, Li S, Tang Y, Teng S, Li H, et al. Assessing the Role of the Generative Pretrained Transformer (GPT) in Alzheimer’s Disease Management: Comparative Study of Neurologist- and Artificial Intelligence–Generated Responses. Journal of Medical Internet Research. 2024. doi:10.2196/51095
BibTeX
@article{Zeng2024,
author = {Zeng, Jiaqi and Zou, Xiaoyi and Li, Shirong and Tang, Yao and Teng, Sisi and Li, Huanhuan and Wang, Changyu and Wu, Yuxuan and Zhang, Luyao and Zhong, Yunheng and Liu, Jialin and Liu, Siru},
title = {Assessing the Role of the Generative Pretrained Transformer (GPT) in Alzheimer’s Disease Management: Comparative Study of Neurologist- and Artificial Intelligence–Generated Responses},
journal = {Journal of Medical Internet Research},
year = {2024},
doi = {10.2196/51095}
}
EUVIMED