VG-RETFound: a hybrid retinal AI framework for early vision impairment screening through vessel-graph learning and cross-modal fusion

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Introduction Vision impairment caused by retinal diseases stands out as one of the significant sources of irreversible blindness, highlighting the importance of developing robust automated screening tools to detect eye diseases at an early stage. Despite the promising results of recent deep learning approaches to the automatic classification of retinal images, most existing solutions focus on image-based representations and do not consider the importance of vascular topology. To overcome the limitations, this study proposes VG-RETFound – a hybrid framework based on a retinal foundation model and graph-based vascular-topology modeling that can be used to classify multiple retinal categories and detect vision impairment. Methods The authors use RETFound—a retinal foundation model—pretrained on a vast number of retinal images, as well as the Graph Attention Network (GAT), to learn vascular topology features by representing retinal vessels as graphs. The presented approach employs a cross-modal multi-head attention fusion method to integrate features from both modalities in real time. To evaluate the efficiency of VG-RETFound, experiments were conducted using the RFMiD and ODIR-5 K datasets for model training and testing across six retinal categories: Normal, Diabetic Retinopathy, Pathological Myopia, Age-related Macular Degeneration, Hypertensive Retinopathy, and Other Retinal Diseases. Results The accuracy of 96.39%, F1-score of 95.84%, MCC of 94.87%, and Kappa of 94.71% were attained on an internal test dataset, while cross-dataset external validation on the BRSET dataset revealed accuracy of 94.81%, F1-score of 94.25%, MCC of 93.09%, and Kappa of 92.94%. Discussion These findings demonstrate that integrating retinal foundation models with anatomically informed vessel-topology learning significantly improves classification accuracy, interpretability, and generalization, highlighting the potential of VG-RETFound as a tool for large-scale vision impairment. screening and retinal disease diagnosis.