EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models

Yating Pan, Janna Hastings

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti250910 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Yating Pan und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computational Linguistics, University of Zurich. Hauptthemen laut MEDLINE: Eye Diseases, Natural Language Processing, Diagnosis, Computer-Assisted. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual foundation models have advanced ocular disease diagnosis, yet providing interpretable explanations remains challenging. We evaluate multimodal LLMs for generating explanations of ocular diagnoses, combining Vision Transformer-derived saliency maps with clinical metadata. After finetuning RETFound for improved performance on the BRSET dataset (AUC-ROC 0.9664/0.8611 for diabetic retinopathy/glaucoma), we compared five LLMs through technical and clinical evaluations. GPT-o1 demonstrated superior performance across technical dimensions and clinical metrics (79.32% precision, 77.18% recall, 78.25% F1, 20.68% hallucination rate). Our findings highlight the importance of underlying diagnostic accuracy and advanced model architecture for generating reliable clinical explanations, suggesting opportunities for integrated verification mechanisms in future developments. The code and details can be found at: https://github.com/YatingPan/ocular-llm-explainability.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Eye DiseasesNatural Language ProcessingDiagnosis, Computer-AssistedHumansLarge Language Models

Stichwörter der Autor:innen

Large Language ModelsXAIexplainabilityophthalmology

Autor:innen und Institutionen

  • Department of Computational Linguistics, University of Zurich.
  • Institute of Implementation Science, Faculty of Medicine, University of Zurich.
  • School of Medicine, University of St. Gallen.
  • Swiss Institute of Bioinformatics.

Zitieren

APA

Pan, Y., & Hastings, J. (2025). Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250910

Vancouver

Pan Y, Hastings J. Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250910

BibTeX
@article{Pan2025,
  author = {Pan, Yating and Hastings, Janna},
  title = {Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti250910}
}
Kostenlos starten →