Automated Chest Diseases Classification Using Multimodal Deep Learning
Biomimetics · 19.11.2026 · 11 (11) · 842
DOI: 10.3390/biomimetics11110842 ↗Kurz gefasst
Fachartikel in der Zeitschrift Biomimetics, online erschienen am 19.11.2026. Verfasst von Fatimah Almatrafi und 2 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University.
Abstract (Originalsprache)
Background
Computer-Aided Diagnosis systems can support physicians by reducing workload, medical errors, and time required for patient care. However, many existing systems primarily rely on medical images while overlooking valuable information available in associated medical reports. This study proposes a multimodal deep learning approach for chest disease classification by integrating chest X-ray images and radiology reports from the MIMIC-CXR dataset. (2)
Methods
Different Deep Learning models are evaluated for each modality, followed by a comprehensive comparison of intermediate and late fusion strategies to investigate the effectiveness of combining visual and textual information. The proposed framework also addresses practical challenges associated with multimodal medical data, including generating disease labels for unannotated data, handling class imbalance, aggregating multiple images at the study level, selecting an appropriate radiology report section for textual input, and managing missing textual information. Furthermore, unlike existing fusion strategies that rely on traditional fusion strategies or combine features into a single representation, the proposed label-aware intermediate fusion employs disease-specific queries to guide multimodal feature fusion, generating disease-specific representations for predicting the 14 disease labels. (3)
Results
The best-performing approach combines visual features extracted from chest X-ray images using the Swin Transformer with textual features extracted from the Findings section of radiology reports using Bio-Clinical BERT through a label-aware intermediate fusion mechanism. This approach achieved a macro-AUC of 90.7% and a micro-AUC of 94.4%, demonstrating its effectiveness in chest disease classification. (4)
Conclusions
The results demonstrate the value of combining complementary visual and textual information for chest disease classification and highlight the importance of selecting an appropriate fusion strategy for multimodal medical data. Beyond its diagnostic contribution, the study illustrates how a biological principle—hierarchical integration of complementary sensory channels, with compensation under modality loss—can be operationalized and evaluated within a clinical learning architecture. Ultimately, these efforts aim to improve, develop, and deliver healthcare services in an integrated manner.
Förderung
- Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University
Zitieren
APA
Almatrafi, F., Bennaceur, H., & Saeed, F. (2026). Automated Chest Diseases Classification Using Multimodal Deep Learning. Biomimetics, 11(11), 842. https://doi.org/10.3390/biomimetics11110842
Vancouver
Almatrafi F, Bennaceur H, Saeed F. Automated Chest Diseases Classification Using Multimodal Deep Learning. Biomimetics. 2026;11(11):842. doi:10.3390/biomimetics11110842
BibTeX
@article{Almatrafi2026,
author = {Almatrafi, Fatimah and Bennaceur, Hachemi and Saeed, Fahman},
title = {Automated Chest Diseases Classification Using Multimodal Deep Learning},
journal = {Biomimetics},
year = {2026},
volume = {11},
number = {11},
pages = {842},
doi = {10.3390/biomimetics11110842}
}
EUVIMED