EUVIMEDEUVIMED entdecken
Crossref

Leveraging multimodal large language model chatbots in oral radiology: a comprehensive evaluation using questions from a Korean dental university

Hui Jeong, Kug Jin Jeon iD, Chena Lee iD, Yoon Joo Choi, Gyu-Dong Jo, Sang-Sun Han iD

Dentomaxillofacial Radiology · 2026

DOI: 10.1093/dmfr/twaf083 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Dentomaxillofacial Radiology, 2026. Verfasst von Hui Jeong und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Oral and Maxillofacial Radiology, Yonsei University College of Dentistry, Seoul 03722, Republic of Korea. Hauptthemen laut MEDLINE: Radiology, Education, Dental, Radiography, Dental, Language. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Gefördert von: National Research Foundation of Korea, Korean government.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

This study aimed to conduct a comprehensive evaluation of general-purpose multimodal large language model (LLM) chatbots in oral radiology.

Methods

Ninety text- and image-based oral radiology questions from a Korean dental university were extracted and categorized into six educational contents and two question types. ChatGPT-4o and Gemini 2.0 Flash were evaluated with following items: accuracy with group differences across six contents (using Fisher’s exact test with Bonferroni correction, P < .0167), answer consistency across ten repeated outputs (evaluated as the mean agreement and Fleiss’ kappa coefficient), and hallucination (evaluated as the mean of the 5-point Global Quality Score assigned by two oral radiologists).

Results

Multimodal LLM chatbots (ChatGPT-4o and Gemini 2.0 Flash) achieved excellent performance on text-based questions with over 80% accuracy but showed limited performance on image-based tasks, with accuracy under 30%. Additionally, image-based tasks exhibited high response variability, and hallucinations were frequently observed, providing incorrect information. These findings suggest that AI chatbots are not yet suitable for reliable use in oral radiology.

Conclusions

This study provided timely insights into the capabilities and limitations of general-purpose multimodal LLM chatbots in the oral radiology, and will serve as a foundation for more safe and effective applications of AI chatbots in the oral radiology field in the future. Advances in knowledge This is the first study to comprehensively assess multimodal LLM chatbots in oral radiology. It provides key insights into the performance benchmarks for AI chatbots in oral radiology, promoting the responsible and transparent integration of AI into dental education.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RadiologyEducation, DentalRadiography, DentalLanguageHumansRepublic of KoreaSchools, DentalEducational MeasurementLarge Language ModelsGenerative Artificial Intelligence

Stichwörter der Autor:innen

accuracyanswer consistencyhallucinationmultimodal large language modeloral radiology

Autor:innen und Institutionen

  • Department of Oral and Maxillofacial Radiology, Yonsei University College of Dentistry, Seoul 03722, Republic of Korea.
  • Institute for Innovative in Digital Healthcare, Yonsei University, Seoul 03722, Republic of Korea.
  • Oral Science Research Center, Yonsei University College of Dentistry, Seoul 03722, Republic of Korea.

ORCID-Profile: Kug Jin Jeon, Chena Lee, Sang-Sun Han

Förderung

  • National Research Foundation of Korea
  • Korean government

Zitieren

APA

Jeong, H., Jeon, K. J., Lee, C., Choi, Y. J., Jo, G. D., & Han, S. S. (2026). Leveraging multimodal large language model chatbots in oral radiology: a comprehensive evaluation using questions from a Korean dental university. Dentomaxillofacial Radiology. https://doi.org/10.1093/dmfr/twaf083

Vancouver

Jeong H, Jeon KJ, Lee C, Choi YJ, Jo GD, Han SS. Leveraging multimodal large language model chatbots in oral radiology: a comprehensive evaluation using questions from a Korean dental university. Dentomaxillofacial Radiology. 2026. doi:10.1093/dmfr/twaf083

BibTeX
@article{Jeong2026,
  author = {Jeong, Hui and Jeon, Kug Jin and Lee, Chena and Choi, Yoon Joo and Jo, Gyu-Dong and Han, Sang-Sun},
  title = {Leveraging multimodal large language model chatbots in oral radiology: a comprehensive evaluation using questions from a Korean dental university},
  journal = {Dentomaxillofacial Radiology},
  year = {2026},
  doi = {10.1093/dmfr/twaf083}
}
Kostenlos starten →