Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study
PLOS One · 28.10.2025 · 20 (10) · e0335775
DOI: 10.1371/journal.pone.0335775 ↗Kurz gefasst
Querschnittstudie in der Zeitschrift PLOS One, online erschienen am 28.10.2025. Verfasst von Viet Anh Nguyen und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Dentistry, Phenikaa University, Hanoi, Vietnam. Hauptthemen laut MEDLINE: Benchmarking, Language, Models, Anatomic, Mouth. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Multimodal large-language models (LLMs) have recently gained the ability to interpret images. However, their accuracy on anatomy tasks remains unclear.
Methods
A cross-sectional, atlas-based benchmark study was conducted in which six publicly accessible chat endpoints, including paired “deep-reasoning” and “low-latency” modes from OpenAI, Microsoft Copilot, and Google Gemini, identified 260 numbered landmarks on 26 high-resolution plates from a classical anatomic atlas. Each image was processed twice per model. Two blinded anatomy lecturers scored responses, including accuracy, run-to-run consistency, and per-label latency, which were compared with χ² and Kruskal–Wallis tests.
Results
Overall accuracy differed significantly among models (χ² = 73.2, P < 0.001). OpenAI o3 achieved the highest correctness (53.1%), outperforming its sibling GPT-4o and both Copilot variants, but required the longest inference time. Musculoskeletal structures were recognised more accurately than neurovascular targets, reflecting the greater visual complexity of fine vessels and nerves. Consistency ranged from 43.5% (Gemini Flash) to 65.0% (GPT-4o); deeper modes improved stability for Copilot and Gemini but not accuracy. Median per-label latency spanned three orders of magnitude, from 0.5 s for Gemini Flash to 33 s for o3.
Conclusions
Currently, publicly available multimodal LLMs can only moderately identify oral and maxillofacial landmarks, and no endpoint is sufficiently reliable to serve as a stand-alone answer key. Higher accuracy was achievable with a trade-off in latency, highlighting the need for domain-specific tuning and human oversight. This atlas benchmark study introduced here provides a reproducible yardstick for future model refinement and educational integration.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Dentistry, Phenikaa University, Hanoi, Vietnam.
- Private Practice, Viet Anh Orthodontic Clinic, Hanoi, Vietnam.
- Faculty of Medicine, Vinh Medical University, Nghe An, Vietnam.
ORCID-Profile: Viet Anh Nguyen
Zitieren
APA
Nguyen, V. A., Vuong, T. Q. T., & Nguyen, V. H. (2025). Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLOS One, 20(10), e0335775. https://doi.org/10.1371/journal.pone.0335775
Vancouver
Nguyen VA, Vuong TQT, Nguyen VH. Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLOS One. 2025;20(10):e0335775. doi:10.1371/journal.pone.0335775
BibTeX
@article{Nguyen2025,
author = {Nguyen, Viet Anh and Vuong, Thi Quynh Trang and Nguyen, Van Hung},
title = {Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study},
journal = {PLOS One},
year = {2025},
volume = {20},
number = {10},
pages = {e0335775},
doi = {10.1371/journal.pone.0335775}
}
EUVIMED