Large language model-based multimodal system for detecting and grading ocular surface diseases from smartphone images
Frontiers in Cell and Developmental Biology · 23.05.2025 · 13
DOI: 10.3389/fcell.2025.1600202 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Cell and Developmental Biology, online erschienen am 23.05.2025. Verfasst von Zhongwen Li und 8 weiteren Autor:innen. Beteiligte Einrichtung: Ningbo Key Laboratory of Medical Research on Blinding Eye Diseases, Ningbo Eye Institute, Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, Ch…. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Zhejiang Province, Natural Science Foundation of Ningbo Municipality.
Abstract (Originalsprache)
Background
The development of medical artificial intelligence (AI) models is primarily driven by the need to address healthcare resource scarcity, particularly in underserved regions. Proposing an affordable, accessible, interpretable, and automated AI system for non-clinical settings is crucial to expanding access to quality healthcare.
Methods
This cross-sectional study developed the Multimodal Ocular Surface Assessment and Interpretation Copilot (MOSAIC) using three multimodal large language models: gpt-4-turbo, claude-3-opus, and gemini-1.5-pro-latest, for detecting three ocular surface diseases (OSDs) and grading keratitis and pterygium. A total of 375 smartphone-captured ocular surface images collected from 290 eyes were utilized to validate MOSAIC. The performance of MOSAIC was evaluated in both zero-shot and few-shot settings, with tasks including image quality control, OSD detection, analysis of the severity of keratitis, and pterygium grading. The interpretability of the system was also evaluated.
Results
MOSAIC achieved 95.00% accuracy in image quality control, 86.96% in OSD detection, 88.33% in distinguishing mild from severe keratitis, and 66.67% in determining pterygium grades with five-shot settings. The performance significantly improved with the increasing learning shots (p < 0.01). The system attained high ROUGE-L F1 scores of 0.70–0.78, depicting its interpretable image comprehension capability.
Conclusion
MOSAIC exhibited exceptional few-shot learning capabilities, achieving high accuracy in OSD management with minimal training examples. This system has significant potential for smartphone integration to enhance the accessibility and effectiveness of OSD detection and grading in resource-limited settings.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Ningbo Key Laboratory of Medical Research on Blinding Eye Diseases, Ningbo Eye Institute, Ningbo Eye Hospital, Wenzhou Medical University, Ningbo, China.
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China.
- Department of Ophthalmology, West China Second University Hospital, Sichuan University, Chengdu, China.
- First People's Hospital of Aksu, Aksu, China.
- Shenzhen Eye Hospital, Shenzhen Eye Medical Center, Southern Medical University, Shenzhen, China.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Zhejiang Province
- Natural Science Foundation of Ningbo Municipality
Zitieren
APA
Li, Z., Wang, Z., Xiu, L., Zhang, P., Wang, W., Wang, Y., Chen, G., Yang, W., & Chen, W. (2025). Large language model-based multimodal system for detecting and grading ocular surface diseases from smartphone images. Frontiers in Cell and Developmental Biology, 13. https://doi.org/10.3389/fcell.2025.1600202
Vancouver
Li Z, Wang Z, Xiu L, Zhang P, Wang W, Wang Y, et al. Large language model-based multimodal system for detecting and grading ocular surface diseases from smartphone images. Frontiers in Cell and Developmental Biology. 2025;13. doi:10.3389/fcell.2025.1600202
BibTeX
@article{Li2025,
author = {Li, Zhongwen and Wang, Zhouqian and Xiu, Liheng and Zhang, Pengyao and Wang, Wenfang and Wang, Yangyang and Chen, Gang and Yang, Weihua and Chen, Wei},
title = {Large language model-based multimodal system for detecting and grading ocular surface diseases from smartphone images},
journal = {Frontiers in Cell and Developmental Biology},
year = {2025},
volume = {13},
doi = {10.3389/fcell.2025.1600202}
}
EUVIMED