EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Visual recognition limitations in multimodal large language models: A comparative analysis of histological image interpretation

Volodymyr Mavrych iD, Einas M Yousef iD, Ahmed Yaqinuddin, Aftab Ahmed Shaikh, Olena Bolgova iD

PLOS Digital Health · 19.03.2026 · 5 (3) · e0001306

DOI: 10.1371/journal.pdig.0001306 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 19.03.2026. Verfasst von Volodymyr Mavrych und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal large language models (LLMs) with image recognition capabilities have emerged as potential tools for medical image analysis, yet their performance in specialized domains like histology remains largely unexplored. The objective of this study was to systematically evaluate the performance of leading multimodal LLMs in histological image interpretation and assess their visual recognition capabilities. Four multimodal LLMs (GPT-4o, Claude Sonnet 4, Gemini 2.5 Flash, and Copilot) were evaluated using 144 histological images representing four tissue types (epithelial, connective, muscle, and nervous) at three magnification levels. Each image was assessed using three standardized questions: tissue identification, morphological features, and functional analysis. Three expert faculty members independently graded responses using a 4-point scale (1 = Poor to 4 = Excellent). Friedman tests, ICC, and post-hoc power analyses were performed with statistical significance set at p < .05. A clear performance hierarchy emerged with Gemini demonstrating superior performance (mean score: 3.35/4.00), significantly outperforming all other models. Copilot and GPT-4o tied for second place (both 2.76/4.00), while Claude showed the lowest performance (2.55/4.00). Performance varied across tissue types, with epithelial tissue showing the greatest inter-model variation. Inter-rater reliability was good across all models (ICC > 0.85), confirming assessment consistency. Post-hoc power analysis validated statistical significance for primary comparisons but indicated insufficient power to distinguish between the three lower-performing models. Current multimodal LLMs exhibit significant limitations in visual recognition relative to text processing performance. The substantial cross-modal performance gaps reveal some constraints in visual processing architectures, though the underlying mechanisms require further investigation. These findings establish technical benchmarks for multimodal LLM development and highlight the need for specialized visual processing innovations in their imaging processes.

Autor:innen und Institutionen

  • College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia.

ORCID-Profile: Volodymyr Mavrych, Einas M Yousef, Olena Bolgova

Zitieren

APA

Mavrych, V., Yousef, E. M., Yaqinuddin, A., Shaikh, A. A., & Bolgova, O. (2026). Visual recognition limitations in multimodal large language models: A comparative analysis of histological image interpretation. PLOS Digital Health, 5(3), e0001306. https://doi.org/10.1371/journal.pdig.0001306

Vancouver

Mavrych V, Yousef EM, Yaqinuddin A, Shaikh AA, Bolgova O. Visual recognition limitations in multimodal large language models: A comparative analysis of histological image interpretation. PLOS Digital Health. 2026;5(3):e0001306. doi:10.1371/journal.pdig.0001306

BibTeX
@article{Mavrych2026,
  author = {Mavrych, Volodymyr and Yousef, Einas M and Yaqinuddin, Ahmed and Shaikh, Aftab Ahmed and Bolgova, Olena},
  title = {Visual recognition limitations in multimodal large language models: A comparative analysis of histological image interpretation},
  journal = {PLOS Digital Health},
  year = {2026},
  volume = {5},
  number = {3},
  pages = {e0001306},
  doi = {10.1371/journal.pdig.0001306}
}
Kostenlos starten →