Multi-Modal AI for Multi-Label Retinal Disease Prediction Using OCT and Fundus Images: A Hybrid Approach
Sensors · 19.07.2025 · 25 (14) · 4492
DOI: 10.3390/s25144492 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 19.07.2025. Verfasst von Amina Zedadra und 3 weiteren Autor:innen. Beteiligte Einrichtung: LabSTIC Laboratory, University 8 May 1945 Guelma, Algeria, BP 401, Guelma 24000, Algeria. Hauptthemen laut MEDLINE: Tomography, Optical Coherence, Retinal Diseases. Die Arbeit wurde bisher 36-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union - NextGenerationEU427 - National Recovery and Resilience Plan (Piano Nazionale di Ripresa e Resilienza, PNRR) - Project:428 "SoBigData.it - Stre…, European Union NextGenerationEU National Recovery and Resilience Plan (Piano Nazionale di Ripresa e Resilienza, PNRR)—Project: “SoBigData.it—Strengthening the….
Abstract (Originalsprache)
Ocular diseases can significantly affect vision and overall quality of life, with diagnosis often being time-consuming and dependent on expert interpretation. While previous computer-aided diagnostic systems have focused primarily on medical imaging, this paper proposes VisionTrack, a multi-modal AI system for predicting multiple retinal diseases, including Diabetic Retinopathy (DR), Age-related Macular Degeneration (AMD), Diabetic Macular Edema (DME), drusen, Central Serous Retinopathy (CSR), and Macular Hole (MH), as well as normal cases. The proposed framework integrates a Convolutional Neural Network (CNN) for image-based feature extraction, a Graph Neural Network (GNN) to model complex relationships among clinical risk factors, and a Large Language Model (LLM) to process patient medical reports. By leveraging diverse data sources, VisionTrack improves prediction accuracy and offers a more comprehensive assessment of retinal health. Experimental results demonstrate the effectiveness of this hybrid system, highlighting its potential for early detection, risk assessment, and personalized ophthalmic care. Experiments were conducted using two publicly available datasets, RetinalOCT and RFMID, which provide diverse retinal imaging modalities: OCT images and fundus images, respectively. The proposed multi-modal AI system demonstrated strong performance in multi-label disease prediction. On the RetinalOCT dataset, the model achieved an accuracy of 0.980, F1-score of 0.979, recall of 0.978, and precision of 0.979. Similarly, on the RFMID dataset, it reached an accuracy of 0.989, F1-score of 0.881, recall of 0.866, and precision of 0.897. These results confirm the robustness, reliability, and generalization capability of the proposed approach across different imaging modalities.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- LabSTIC Laboratory, University 8 May 1945 Guelma, Algeria, BP 401, Guelma 24000, Algeria.
- Medical-Surgical Ophthalmology Office, 284 CNEP Apartments, University Road, Block 37, No. 1, Guelma 24006, Algeria.
- ICAR-CNR-Institute for High Performance Computing and Networking, National Research Council of Italy, Via P. Bucci 8/9C, 87036 Rende, Italy.
ORCID-Profile: Amina Zedadra, Ouarda Zedadra, Antonio Guerrieri
Förderung
- European Union - NextGenerationEU427 - National Recovery and Resilience Plan (Piano Nazionale di Ripresa e Resilienza, PNRR) - Project:428 "SoBigData.it - Stre…
- European Union NextGenerationEU National Recovery and Resilience Plan (Piano Nazionale di Ripresa e Resilienza, PNRR)—Project: “SoBigData.it—Strengthening the…
Zitieren
APA
Zedadra, A., Salah-Salah, M. Y., Zedadra, O., & Guerrieri, A. (2025). Multi-Modal AI for Multi-Label Retinal Disease Prediction Using OCT and Fundus Images: A Hybrid Approach. Sensors, 25(14), 4492. https://doi.org/10.3390/s25144492
Vancouver
Zedadra A, Salah-Salah MY, Zedadra O, Guerrieri A. Multi-Modal AI for Multi-Label Retinal Disease Prediction Using OCT and Fundus Images: A Hybrid Approach. Sensors. 2025;25(14):4492. doi:10.3390/s25144492
BibTeX
@article{Zedadra2025,
author = {Zedadra, Amina and Salah-Salah, Mahmoud Yassine and Zedadra, Ouarda and Guerrieri, Antonio},
title = {Multi-Modal AI for Multi-Label Retinal Disease Prediction Using OCT and Fundus Images: A Hybrid Approach},
journal = {Sensors},
year = {2025},
volume = {25},
number = {14},
pages = {4492},
doi = {10.3390/s25144492}
}
EUVIMED