EUVIMEDEUVIMED entdecken
Crossref

VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge

Zihan Li, Diping Song, Zefeng Yang, Deming Wang, Fei Li, Xiulan Zhang, Paul E Kinahan, Yu Qiao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-12 · 47 (12) · 11848-11862

DOI: 10.1109/tpami.2025.3598734 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-12. Verfasst von Zihan Li und 7 weiteren Autor:innen. Hauptthemen laut MEDLINE: Ophthalmology, Natural Language Processing, Diagnostic Techniques, Ophthalmological. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: Shanghai Artificial Intelligence Laboratory, National Key R&D Program of China, National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The need for improved diagnostic methods in ophthalmology is acute, especially in the underdeveloped regions with limited access to specialists and advanced equipment. Therefore, we introduce VisionUnite, a novel vision-language foundation model for ophthalmology enhanced with clinical knowledge. VisionUnite has been pretrained on an extensive dataset comprising 1.24 million image-text pairs, and further refined using our proposed MMFundus dataset, which includes 296,379 high-quality fundus image-text pairs and 889,137 simulated doctor-patient dialogue instances. Our experiments indicate that VisionUnite outperforms existing generative foundation models such as GPT-4V and Gemini Pro. It also demonstrates diagnostic capabilities comparable to junior ophthalmologists. VisionUnite performs well in various clinical scenarios including open-ended multi-disease diagnosis, clinical explanation, and patient interaction, making it a highly versatile tool for initial ophthalmic disease screening. VisionUnite can also serve as an educational aid for junior ophthalmologists, accelerating their acquisition of knowledge regarding both common and underrepresented ophthalmic conditions. VisionUnite represents a significant advancement in ophthalmology, with broad implications for diagnostics, medical education, and understanding of disease mechanisms.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

OphthalmologyNatural Language ProcessingDiagnostic Techniques, OphthalmologicalHumansDatabases, Factual

Förderung

  • Shanghai Artificial Intelligence Laboratory
  • National Key R&D Program of China
  • National Natural Science Foundation of China
  • China Postdoctoral Science Foundation

Zitieren

APA

Li, Z., Song, D., Yang, Z., Wang, D., Li, F., Zhang, X., Kinahan, P. E., & Qiao, Y. (2025). VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(12), 11848-11862. https://doi.org/10.1109/tpami.2025.3598734

Vancouver

Li Z, Song D, Yang Z, Wang D, Li F, Zhang X, et al. VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(12):11848-11862. doi:10.1109/tpami.2025.3598734

BibTeX
@article{Li2025,
  author = {Li, Zihan and Song, Diping and Yang, Zefeng and Wang, Deming and Li, Fei and Zhang, Xiulan and Kinahan, Paul E and Qiao, Yu},
  title = {VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced With Clinical Knowledge},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {12},
  pages = {11848--11862},
  doi = {10.1109/tpami.2025.3598734}
}
Kostenlos starten →