Refined Graph-Guided Fusion Network for Explainable Multimodal Lung Cancer Classification Using CT Imaging and Semantic Features

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Classifying benign and malignant lung nodules from computed tomography (CT) images remains difficult because lung nodules can be hard to classify, and unimodal models cannot capture complementary diagnostic information. Despite the success of deep learning, existing methods rely only on image information and miss semantic information that can be obtained from an expert radiologist’s knowledge. Hence, the authors propose a new multimodal lung nodule classification model in this study, named the Graph-Guided Fusion Network (R-GGFN), that combines three-dimensional (3D) CT image features and structured radiologist annotations. The proposed architecture consists of three models. A 3D ResNet-18 network for image feature extraction, an MLP network for encoding semantic information, and a Graph Attention Network (GAT) for capturing inter-nodule relationships and fusing multimodal information with the graph. We add a tabular skip connection to preserve discriminative semantic features and use focal loss to address imbalance during training. To prevent data leakage, we partitioned the publicly available LIDC-IDRI dataset at the patient level. Experimental results on a held-out patient-level test set, accessed only once after model selection was finalized, show that the proposed R-GGFN achieves an accuracy of 85.21%, an AUROC of 0.9147, a PR-AUC of 0.9213, and an F1-score of 0.8609. Among all unimodal and multimodal baselines internally evaluated, R-GGFN achieved the best value on every reported metric, including accuracy, AUROC, PR-AUC, F1-score, Precision, sensitivity, and specificity. Furthermore, the proposed approach enhances model transparency by combining explainable AI techniques (e.g., 3D Grad-CAM, SHAP, and graph visualization) to explain the model at the image, feature, and graph levels. The results show that the graph-guided multimodal fusion method can fully leverage complementary image and semantic information, improving diagnostic accuracy and interpretability. The framework proposed here is a good and understandable computer-aided diagnosis decision-support system for lung cancer and a step towards future external dataset validation.