EUVIMEDEUVIMED entdecken
Open Access

Benchmarking Vision Encoders for Image Classification in Ophthalmology

Jay Zoellin, Colin Merk, Bence György iD

Computational and Structural Biotechnology Journal · 01.01.2026 · 35 (2)

DOI: 10.34133/csbj.0178 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computational and Structural Biotechnology Journal, online erschienen am 01.09.2026. Verfasst von Jay Zoellin und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Molecular and Clinical Ophthalmology Basel (IOB), Basel, Switzerland. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 85 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Foundation vision encoders are rapidly emerging as the standard for retinal artificial intelligence. Yet, ophthalmology still lacks a comprehensive benchmark, leaving model selection for basic science and clinical translation as guesswork. Here, we present a large-scale comparison of 34 pretrained encoders on 39 classification tasks covering color fundus photography, optical coherence tomography, scanning laser ophthalmoscopy, and ultrawidefield imaging. Using a unified pipeline, we compare frozen-feature evaluation, linear probing, and end-to-end fine-tuning to determine which models translate into strong downstream performance. We show that ophthalmic transfer is highly task dependent: no single encoder dominates, and model rankings vary across datasets. Contrary to common expectations, retina-specific pretraining does not confer an advantage. Instead, several natural-image and cross-domain medical encoders match or surpass ophthalmology-specialized models, with the histopathology-pretrained Virchow achieving the strongest overall performance. In addition, pathology-pretrained encoders consistently place near the top, revealing the value of cross-domain pretraining for ophthalmic applications. We further show that inexpensive proxy evaluations are unreliable substitutes for full fine-tuning. Across fairness analyses, all encoders exhibit similar age- and sex-associated performance gaps, and larger models appear more sensitive to suboptimal learning rates, whereas smaller encoders are robust. Together, these findings provide an objective reference for encoder selection in ophthalmology and show that reliable retinal artificial intelligence depends not only on model scale or domain-specific pretraining but also on careful, protocol-aware evaluation. By releasing our code, splits, and benchmarking pipeline, we aim to establish a transparent foundation for future ophthalmic foundation-model research.

Autor:innen und Institutionen

  • Institute of Molecular and Clinical Ophthalmology Basel (IOB), Basel, Switzerland.
  • Department of Ophthalmology, University of Basel, Basel, Switzerland.

ORCID-Profile: Bence György

Förderung

  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung

Zitieren

APA

Zoellin, J., Merk, C., & György, B. (2026). Benchmarking Vision Encoders for Image Classification in Ophthalmology. Computational and Structural Biotechnology Journal, 35(2). https://doi.org/10.34133/csbj.0178

Vancouver

Zoellin J, Merk C, György B. Benchmarking Vision Encoders for Image Classification in Ophthalmology. Computational and Structural Biotechnology Journal. 2026;35(2). doi:10.34133/csbj.0178

BibTeX
@article{Zoellin2026,
  author = {Zoellin, Jay and Merk, Colin and György, Bence},
  title = {Benchmarking Vision Encoders for Image Classification in Ophthalmology},
  journal = {Computational and Structural Biotechnology Journal},
  year = {2026},
  volume = {35},
  number = {2},
  doi = {10.34133/csbj.0178}
}
Kostenlos starten →