EUVIMEDEUVIMED entdecken
QuerschnittstudieOpen Access · CC BY 4.0

Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study

Viet Anh Nguyen iD, Thi Quynh Trang Vuong, Van Hung Nguyen

PLOS One · 28.10.2025 · 20 (10) · e0335775

DOI: 10.1371/journal.pone.0335775 ↗

Kurz gefasst

Querschnittstudie in der Zeitschrift PLOS One, online erschienen am 28.10.2025. Verfasst von Viet Anh Nguyen und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Dentistry, Phenikaa University, Hanoi, Vietnam. Hauptthemen laut MEDLINE: Benchmarking, Language, Models, Anatomic, Mouth. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Multimodal large-language models (LLMs) have recently gained the ability to interpret images. However, their accuracy on anatomy tasks remains unclear.

Methods

A cross-sectional, atlas-based benchmark study was conducted in which six publicly accessible chat endpoints, including paired “deep-reasoning” and “low-latency” modes from OpenAI, Microsoft Copilot, and Google Gemini, identified 260 numbered landmarks on 26 high-resolution plates from a classical anatomic atlas. Each image was processed twice per model. Two blinded anatomy lecturers scored responses, including accuracy, run-to-run consistency, and per-label latency, which were compared with χ² and Kruskal–Wallis tests.

Results

Overall accuracy differed significantly among models (χ² = 73.2, P < 0.001). OpenAI o3 achieved the highest correctness (53.1%), outperforming its sibling GPT-4o and both Copilot variants, but required the longest inference time. Musculoskeletal structures were recognised more accurately than neurovascular targets, reflecting the greater visual complexity of fine vessels and nerves. Consistency ranged from 43.5% (Gemini Flash) to 65.0% (GPT-4o); deeper modes improved stability for Copilot and Gemini but not accuracy. Median per-label latency spanned three orders of magnitude, from 0.5 s for Gemini Flash to 33 s for o3.

Conclusions

Currently, publicly available multimodal LLMs can only moderately identify oral and maxillofacial landmarks, and no endpoint is sufficiently reliable to serve as a stand-alone answer key. Higher accuracy was achievable with a trade-off in latency, highlighting the need for domain-specific tuning and human oversight. This atlas benchmark study introduced here provides a reproducible yardstick for future model refinement and educational integration.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BenchmarkingLanguageModels, AnatomicMouthFaceHumansCross-Sectional Studies

Autor:innen und Institutionen

  • Faculty of Dentistry, Phenikaa University, Hanoi, Vietnam.
  • Private Practice, Viet Anh Orthodontic Clinic, Hanoi, Vietnam.
  • Faculty of Medicine, Vinh Medical University, Nghe An, Vietnam.

ORCID-Profile: Viet Anh Nguyen

Zitieren

APA

Nguyen, V. A., Vuong, T. Q. T., & Nguyen, V. H. (2025). Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLOS One, 20(10), e0335775. https://doi.org/10.1371/journal.pone.0335775

Vancouver

Nguyen VA, Vuong TQT, Nguyen VH. Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLOS One. 2025;20(10):e0335775. doi:10.1371/journal.pone.0335775

BibTeX
@article{Nguyen2025,
  author = {Nguyen, Viet Anh and Vuong, Thi Quynh Trang and Nguyen, Van Hung},
  title = {Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study},
  journal = {PLOS One},
  year = {2025},
  volume = {20},
  number = {10},
  pages = {e0335775},
  doi = {10.1371/journal.pone.0335775}
}
Kostenlos starten →