Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models
Studies in Health Technology and Informatics · 2025
DOI: 10.3233/shti250910 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Yating Pan und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computational Linguistics, University of Zurich. Hauptthemen laut MEDLINE: Eye Diseases, Natural Language Processing, Diagnosis, Computer-Assisted. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Visual foundation models have advanced ocular disease diagnosis, yet providing interpretable explanations remains challenging. We evaluate multimodal LLMs for generating explanations of ocular diagnoses, combining Vision Transformer-derived saliency maps with clinical metadata. After finetuning RETFound for improved performance on the BRSET dataset (AUC-ROC 0.9664/0.8611 for diabetic retinopathy/glaucoma), we compared five LLMs through technical and clinical evaluations. GPT-o1 demonstrated superior performance across technical dimensions and clinical metrics (79.32% precision, 77.18% recall, 78.25% F1, 20.68% hallucination rate). Our findings highlight the importance of underlying diagnostic accuracy and advanced model architecture for generating reliable clinical explanations, suggesting opportunities for integrated verification mechanisms in future developments. The code and details can be found at: https://github.com/YatingPan/ocular-llm-explainability.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computational Linguistics, University of Zurich.
- Institute of Implementation Science, Faculty of Medicine, University of Zurich.
- School of Medicine, University of St. Gallen.
- Swiss Institute of Bioinformatics.
Zitieren
APA
Pan, Y., & Hastings, J. (2025). Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250910
Vancouver
Pan Y, Hastings J. Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250910
BibTeX
@article{Pan2025,
author = {Pan, Yating and Hastings, Janna},
title = {Enhancing Interpretability of Ocular Disease Diagnosis: A Zero-Shot Study of Multimodal Large Language Models},
journal = {Studies in Health Technology and Informatics},
year = {2025},
doi = {10.3233/shti250910}
}
EUVIMED