Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis
Studies in Health Technology and Informatics · 2025
DOI: 10.3233/shti250567 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Georgios Feretzakis und 8 weiteren Autor:innen. Beteiligte Einrichtung: School of Science and Technology, Hellenic Open University, Patras, Greece. Hauptthemen laut MEDLINE: Tomography, Optical Coherence, Deep Learning, Retina, Retinal Diseases. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Optical Coherence Tomography (OCT) has become an indispensable imaging modality in ophthalmology, providing high-resolution cross-sectional images of the retina. Accurate classification of OCT images is crucial for diagnosing retinal diseases such as Age-related Macular Degeneration (AMD) and Diabetic Macular Edema (DME). This study explores the efficacy of various deep learning models, including convolutional neural networks (CNNs) and Vision Transformers (ViTs), in classifying OCT images. We also investigate the impact of integrating metadata (patient age, sex, eye laterality, and year) into the classification process, even when a significant portion of metadata is missing. Our results demonstrate that multimodal models leveraging both image and metadata inputs, such as the Multimodal ResNet18, can achieve competitive performance compared to image-only models, such as DenseNet121. Notably, DenseNet121 and Multimodal ResNet18 achieved the highest accuracy of 95.16%, with DenseNet121 showing a slightly higher F1-score of 0.9313. The multimodal ViT-based model also demonstrated promising results, achieving an accuracy of 93.22%, indicating the potential of Vision Transformers (ViTs) in medical image analysis, especially for handling complex multimodal data.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Science and Technology, Hellenic Open University, Patras, Greece.
- School of Medicine, National and Kapodistrian University of Athens, Athens, Greece.
- Merative, Healthcare, Dublin Docklands, Dublin 2, Ireland.
- Medical Retina Department, Bristol Eye Hospital, Bristol, UK.
- Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK.
- Medical School, Humanitas University, Milan, Italy.
Zitieren
APA
Feretzakis, G., Karakosta, C., Gkoulalas-Divanis, A., Bisoukis, A., Boufeas, I. Z., Bazakidou, E., Sakagianni, A., Kalles, D., & Verykios, V. S. (2025). Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250567
Vancouver
Feretzakis G, Karakosta C, Gkoulalas-Divanis A, Bisoukis A, Boufeas IZ, Bazakidou E, et al. Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250567
BibTeX
@article{Feretzakis2025,
author = {Feretzakis, Georgios and Karakosta, Christina and Gkoulalas-Divanis, Aris and Bisoukis, Anastasios and Boufeas, Iris Zoe and Bazakidou, Effrosyni and Sakagianni, Aikaterini and Kalles, Dimitris and Verykios, Vassilios S},
title = {Leveraging Vision Transformers in Multimodal Models for Retinal OCT Analysis},
journal = {Studies in Health Technology and Informatics},
year = {2025},
doi = {10.3233/shti250567}
}
EUVIMED