EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automated Chest Diseases Classification Using Multimodal Deep Learning

Fatimah Almatrafi, Hachemi Bennaceur, Fahman Saeed

Biomimetics · 19.11.2026 · 11 (11) · 842

DOI: 10.3390/biomimetics11110842 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biomimetics, online erschienen am 19.11.2026. Verfasst von Fatimah Almatrafi und 2 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Computer-Aided Diagnosis systems can support physicians by reducing workload, medical errors, and time required for patient care. However, many existing systems primarily rely on medical images while overlooking valuable information available in associated medical reports. This study proposes a multimodal deep learning approach for chest disease classification by integrating chest X-ray images and radiology reports from the MIMIC-CXR dataset. (2)

Methods

Different Deep Learning models are evaluated for each modality, followed by a comprehensive comparison of intermediate and late fusion strategies to investigate the effectiveness of combining visual and textual information. The proposed framework also addresses practical challenges associated with multimodal medical data, including generating disease labels for unannotated data, handling class imbalance, aggregating multiple images at the study level, selecting an appropriate radiology report section for textual input, and managing missing textual information. Furthermore, unlike existing fusion strategies that rely on traditional fusion strategies or combine features into a single representation, the proposed label-aware intermediate fusion employs disease-specific queries to guide multimodal feature fusion, generating disease-specific representations for predicting the 14 disease labels. (3)

Results

The best-performing approach combines visual features extracted from chest X-ray images using the Swin Transformer with textual features extracted from the Findings section of radiology reports using Bio-Clinical BERT through a label-aware intermediate fusion mechanism. This approach achieved a macro-AUC of 90.7% and a micro-AUC of 94.4%, demonstrating its effectiveness in chest disease classification. (4)

Conclusions

The results demonstrate the value of combining complementary visual and textual information for chest disease classification and highlight the importance of selecting an appropriate fusion strategy for multimodal medical data. Beyond its diagnostic contribution, the study illustrates how a biological principle—hierarchical integration of complementary sensory channels, with compensation under modality loss—can be operationalized and evaluated within a clinical learning architecture. Ultimately, these efforts aim to improve, develop, and deliver healthcare services in an integrated manner.

Förderung

  • Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University

Zitieren

APA

Almatrafi, F., Bennaceur, H., & Saeed, F. (2026). Automated Chest Diseases Classification Using Multimodal Deep Learning. Biomimetics, 11(11), 842. https://doi.org/10.3390/biomimetics11110842

Vancouver

Almatrafi F, Bennaceur H, Saeed F. Automated Chest Diseases Classification Using Multimodal Deep Learning. Biomimetics. 2026;11(11):842. doi:10.3390/biomimetics11110842

BibTeX
@article{Almatrafi2026,
  author = {Almatrafi, Fatimah and Bennaceur, Hachemi and Saeed, Fahman},
  title = {Automated Chest Diseases Classification Using Multimodal Deep Learning},
  journal = {Biomimetics},
  year = {2026},
  volume = {11},
  number = {11},
  pages = {842},
  doi = {10.3390/biomimetics11110842}
}
Kostenlos starten →