EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers

Ana Suárez iD, Víctor Díaz-Flores García iD, Juan Algar, Margarita Gómez Sánchez iD, María Llorente de Pedro, Yolanda Freire iD

International Endodontic Journal · 2024-01 · 57 (1) · 108-113

DOI: 10.1111/iej.13985 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Endodontic Journal, online erschienen am 09.10.2023. Verfasst von Ana Suárez und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pre-Clinic Dentistry, School of Biomedical Sciences, Universidad Europea de Madrid, Madrid, Spain. Hauptthemen laut MEDLINE: Artificial Intelligence, Software. Die Arbeit wurde bisher 203-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aim

Chatbot Generative Pre‐trained Transformer (ChatGPT) is a generative artificial intelligence (AI) software based on large language models (LLMs), designed to simulate human conversations and generate novel content based on the training data it has been exposed to. The aim of this study was to evaluate the consistency and accuracy of ChatGPT‐generated answers to clinical questions in endodontics, compared to answers provided by human experts. Methodology Ninety‐one dichotomous (yes/no) questions were designed and categorized into three levels of difficulty. Twenty questions were randomly selected from each difficulty level. Sixty answers were generated by ChatGPT for each question. Two endodontic experts independently answered the 60 questions. Statistical analysis was performed using the SPSS program to calculate the consistency and accuracy of the answers generated by ChatGPT compared to the experts. Confidence intervals (95%) and standard deviations were used to estimate variability.

Results

The answers generated by ChatGPT showed high consistency (85.44%). No significant differences in consistency were found based on question difficulty. In terms of answer accuracy, ChatGPT achieved an average accuracy of 57.33%. However, significant differences in accuracy were observed based on question difficulty, with lower accuracy for easier questions.

Conclusions

Currently, ChatGPT is not capable of replacing dentists in clinical decision‐making. As ChatGPT's performance improves through deep learning, it is expected to become more useful and effective in the field of endodontics. However, careful attention and ongoing evaluation are needed to ensure its accuracy, reliability and safety in endodontics.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceSoftwareHumansReproducibility of ResultsClinical Decision-MakingDental Care

Stichwörter der Autor:innen

ChatGPTartificial intelligencechatbotdentistryendodonticslarge language models

Autor:innen und Institutionen

  • Department of Pre-Clinic Dentistry, School of Biomedical Sciences, Universidad Europea de Madrid, Madrid, Spain.
  • Department of Clinical Dentistry, School of Biomedical Sciences, Universidad Europea de Madrid, Madrid, Spain.

ORCID-Profile: Ana Suárez, Víctor Díaz-Flores García, Margarita Gómez Sánchez, Yolanda Freire

Zitieren

APA

Suárez, A., Díaz-Flores García, V., Algar, J., Gómez Sánchez, M., Llorente de Pedro, M., & Freire, Y. (2024). Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers. International Endodontic Journal, 57(1), 108-113. https://doi.org/10.1111/iej.13985

Vancouver

Suárez A, Díaz-Flores García V, Algar J, Gómez Sánchez M, Llorente de Pedro M, Freire Y. Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers. International Endodontic Journal. 2024;57(1):108-113. doi:10.1111/iej.13985

BibTeX
@article{Surez2024,
  author = {Suárez, Ana and Díaz-Flores García, Víctor and Algar, Juan and Gómez Sánchez, Margarita and Llorente de Pedro, María and Freire, Yolanda},
  title = {Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers},
  journal = {International Endodontic Journal},
  year = {2024},
  volume = {57},
  number = {1},
  pages = {108--113},
  doi = {10.1111/iej.13985}
}
Kostenlos starten →