FROM TEXT TO DIAGNOSE: CHATGPT’S EFFICACY IN MEDICAL DECISION-MAKING
Wiadomości Lekarskie · 2023-11 · 76 (11) · 2345-2350
DOI: 10.36740/wlek202311101 ↗Kurz gefasst
Fachartikel in der Zeitschrift Wiadomości Lekarskie, 2023. Verfasst von Yaroslav Mykhalko und 4 weiteren Autor:innen. Beteiligte Einrichtung: UZHHOROD NATIONAL UNIVERSITY, UZHHOROD, UKRAINE. Hauptthemen laut MEDLINE: Artificial Intelligence, Physicians. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 23 Literaturangaben.
Abstract (Originalsprache)
The aim: Evaluate the diagnostic capabilities of the ChatGPT in the field of medical diagnosis. Materialrrs and methods: We utilized 50 clinical cases, employing Large Language Model ChatGPT-3.5. The experiment had three phases, each with a new chat setup. In the initial phase, ChatGPT received detailed clinical case descriptions, guided by a “Persona Pattern” prompt. In the second phase, cases with diagnostic errors were addressed by providing potential diagnoses for ChatGPT to choose from. The final phase assessed artificial intelligence’s ability to mimic a medical practitioner’s diagnostic process, with prompts limiting initial information to symptoms and history.
Results
In the initial phase, ChatGPT showed a 66.00% diagnostic accuracy, surpassing physicians by nearly 50%. Notably, in 11 cases requiring image interpretation, ChatGPT struggled initially but achieved a correct diagnosis for four without added interpretations. In the second phase, ChatGPT demonstrated a remarkable 70.59% diagnostic accuracy, while physicians averaged 41.47%. Furthermore, the overall accuracy of Large Language Model in first and second phases together was 90.00%. In the third phase emulating real doctor decision-making, ChatGPT achieved a 46.00% success rate.
Conclusions
Our research underscores ChatGPT’s strong potential in clinical medicine as a diagnostic tool, especially in structured scenarios. It emphasizes the need for supplementary data and the complexity of medical diagnosis. This contributes valuable insights to AI-driven clinical diagnostics, with a nod to the importance of prompt engineering techniques in ChatGPT’s interaction with doctors.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- UZHHOROD NATIONAL UNIVERSITY, UZHHOROD, UKRAINE.
Zitieren
APA
Mykhalko, Y., Kish, P., Rubtsova, Y., Kutsyn, O., & Koval, V. (2023). FROM TEXT TO DIAGNOSE: CHATGPT’S EFFICACY IN MEDICAL DECISION-MAKING. Wiadomości Lekarskie, 76(11), 2345-2350. https://doi.org/10.36740/wlek202311101
Vancouver
Mykhalko Y, Kish P, Rubtsova Y, Kutsyn O, Koval V. FROM TEXT TO DIAGNOSE: CHATGPT’S EFFICACY IN MEDICAL DECISION-MAKING. Wiadomości Lekarskie. 2023;76(11):2345-2350. doi:10.36740/wlek202311101
BibTeX
@article{Mykhalko2023,
author = {Mykhalko, Yaroslav and Kish, Pavlo and Rubtsova, Yelyzaveta and Kutsyn, Oleksandr and Koval, Valentyna},
title = {FROM TEXT TO DIAGNOSE: CHATGPT’S EFFICACY IN MEDICAL DECISION-MAKING},
journal = {Wiadomości Lekarskie},
year = {2023},
volume = {76},
number = {11},
pages = {2345--2350},
doi = {10.36740/wlek202311101}
}
EUVIMED