EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis

Georgios Feretzakis, Christina Karakosta, Aris Gkoulalas-Divanis, Anastasios Bisoukis, Iris Zoe Boufeas, Effrosyni Bazakidou, Aikaterini Sakagianni, Dimitris Kalles, Vassilios S Verykios

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti250567 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Georgios Feretzakis und 8 weiteren Autor:innen. Beteiligte Einrichtung: School of Science and Technology, Hellenic Open University, Patras, Greece. Hauptthemen laut MEDLINE: Tomography, Optical Coherence, Deep Learning, Retina, Retinal Diseases. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Optical Coherence Tomography (OCT) has become an indispensable imaging modality in ophthalmology, providing high-resolution cross-sectional images of the retina. Accurate classification of OCT images is crucial for diagnosing retinal diseases such as Age-related Macular Degeneration (AMD) and Diabetic Macular Edema (DME). This study explores the efficacy of various deep learning models, including convolutional neural networks (CNNs) and Vision Transformers (ViTs), in classifying OCT images. We also investigate the impact of integrating metadata (patient age, sex, eye laterality, and year) into the classification process, even when a significant portion of metadata is missing. Our results demonstrate that multimodal models leveraging both image and metadata inputs, such as the Multimodal ResNet18, can achieve competitive performance compared to image-only models, such as DenseNet121. Notably, DenseNet121 and Multimodal ResNet18 achieved the highest accuracy of 95.16%, with DenseNet121 showing a slightly higher F1-score of 0.9313. The multimodal ViT-based model also demonstrated promising results, achieving an accuracy of 93.22%, indicating the potential of Vision Transformers (ViTs) in medical image analysis, especially for handling complex multimodal data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Tomography, Optical CoherenceDeep LearningRetinaRetinal DiseasesImage Interpretation, Computer-AssistedHumansNeural Networks, ComputerMacular Degeneration

Stichwörter der Autor:innen

Machine LearningMultimodal Deep LearningOptical Coherence TomographyRetinal Disease ClassificationVision Transformers

Autor:innen und Institutionen

  • School of Science and Technology, Hellenic Open University, Patras, Greece.
  • School of Medicine, National and Kapodistrian University of Athens, Athens, Greece.
  • Merative, Healthcare, Dublin Docklands, Dublin 2, Ireland.
  • Medical Retina Department, Bristol Eye Hospital, Bristol, UK.
  • Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK.
  • Medical School, Humanitas University, Milan, Italy.

Zitieren

APA

Feretzakis, G., Karakosta, C., Gkoulalas-Divanis, A., Bisoukis, A., Boufeas, I. Z., Bazakidou, E., Sakagianni, A., Kalles, D., & Verykios, V. S. (2025). Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250567

Vancouver

Feretzakis G, Karakosta C, Gkoulalas-Divanis A, Bisoukis A, Boufeas IZ, Bazakidou E, et al. Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250567

BibTeX
@article{Feretzakis2025,
  author = {Feretzakis, Georgios and Karakosta, Christina and Gkoulalas-Divanis, Aris and Bisoukis, Anastasios and Boufeas, Iris Zoe and Bazakidou, Effrosyni and Sakagianni, Aikaterini and Kalles, Dimitris and Verykios, Vassilios S},
  title = {Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti250567}
}
Kostenlos starten →