EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments

Paul Künzle, Sebastian Paris

Clinical Oral Investigations · 07.10.2024 · 28 (11)

DOI: 10.1007/s00784-024-05968-w ↗

Kurz gefasst

Fachartikel in der Zeitschrift Clinical Oral Investigations, online erschienen am 07.10.2024. Verfasst von Paul Künzle und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Aßmannshauser Str. 4-6, Berlin, 14197, Germany. pa…. Hauptthemen laut MEDLINE: Artificial Intelligence, Endodontics. Die Arbeit wurde bisher 67-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Charité - Universitätsmedizin Berlin.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

The advent of artificial intelligence (AI) and large language model (LLM)-based AI applications (LLMAs) has tremendous implications for our society. This study analyzed the performance of LLMAs on solving restorative dentistry and endodontics (RDE) student assessment questions. Materials and methods 151 questions from a RDE question pool were prepared for prompting using LLMAs from OpenAI (ChatGPT-3.5,-4.0 and -4.0o) and Google (Gemini 1.0). Multiple-choice questions were sorted into four question subcategories, entered into LLMAs and answers recorded for analysis. P-value and chi-square statistical analyses were performed using Python 3.9.16.

Results

The total answer accuracy of ChatGPT-4.0o was the highest, followed by ChatGPT-4.0, Gemini 1.0 and ChatGPT-3.5 (72%, 62%, 44% and 25%, respectively) with significant differences between all LLMAs except GPT-4.0 models. The performance on subcategories direct restorations and caries was the highest, followed by indirect restorations and endodontics.

Conclusions

Overall, there are large performance differences among LLMAs. Only the ChatGPT-4 models achieved a success ratio that could be used with caution to support the dental academic curriculum. Clinical relevance While LLMAs could support clinicians to answer dental field-related questions, this capacity depends strongly on the employed model. The most performant model ChatGPT-4.0o achieved acceptable accuracy rates in some subject sub-categories analyzed.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceEndodonticsHumansEducation, DentalEducational MeasurementStudents, DentalDentistry, OperativeClinical CompetenceSurveys and Questionnaires

Stichwörter der Autor:innen

Artificial intelligenceChatGPTGeminiGenAINatural language processing

Autor:innen und Institutionen

  • Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Aßmannshauser Str. 4-6, Berlin, 14197, Germany. paul.kuenzle@charite.de.
  • Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Aßmannshauser Str. 4-6, Berlin, 14197, Germany.

Förderung

  • Charité - Universitätsmedizin Berlin

Zitieren

APA

Künzle, P., & Paris, S. (2024). Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments. Clinical Oral Investigations, 28(11). https://doi.org/10.1007/s00784-024-05968-w

Vancouver

Künzle P, Paris S. Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments. Clinical Oral Investigations. 2024;28(11). doi:10.1007/s00784-024-05968-w

BibTeX
@article{Knzle2024,
  author = {Künzle, Paul and Paris, Sebastian},
  title = {Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments},
  journal = {Clinical Oral Investigations},
  year = {2024},
  volume = {28},
  number = {11},
  doi = {10.1007/s00784-024-05968-w}
}
Kostenlos starten →