Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification
Diagnostics · 28.10.2026 · 16 (21) · 3551
DOI: 10.3390/diagnostics16213551 ↗Kurz gefasst
Fachartikel in der Zeitschrift Diagnostics, online erschienen am 28.10.2026. Verfasst von Aqeeb Hussain und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Software Engineering, Capital University of Science and Technology, Islamabad 44000, Pakistan. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Saud University, Ongoing Research Funding Program, King Saud University.
Abstract (Originalsprache)
Background/Objectives: Medical vision–language learning integrates radiographic images and textual information for chest X-ray (CXR) classification. However, in long-tailed multi-label CXR classification, modality imbalance can cause models to rely disproportionately on radiology reports while underutilizing visual information. This study investigates text dominance and proposes a modality-balanced framework to strengthen visual representation while preserving multimodal performance.
Methods
Experiments used MIMIC-CXR with patient-level training, validation, and test splits of 32,942, 4024, and 3955 paired samples, respectively. Swin-Tiny and PubMedBERT served as visual and textual encoders. The proposed framework combines adaptive gated fusion, Asymmetric Loss (ASL), and training-time modality dropout. It was compared with Binary Cross-Entropy (BCE) concatenation, ASL concatenation, gated fusion, and cross-attention fusion.
Results
The BCE baseline achieved a Macro AUROC of 0.9957 under Image + Text evaluation and 0.5804 under Image-Only evaluation, while Text-Only evaluation yielded 0.9955, indicating substantial modality disparity. Other alternatives achieved Image-Only Macro AUROC values of 0.5610–0.5720. The proposed framework improved Image-Only Macro AUROC to 0.7697 and Macro PR-AUC from 0.1814 to 0.3428 while maintaining an Image + Text Macro AUROC of 0.9958. The modality-dominance gap decreased from 0.4151 to 0.2260, a 45.6% reduction in performance disparity. Grad-CAM and Integrated Gradients provided insights into visual evidence and textual attribution, revealing reliance on explicit disease-related terminology in reports.
Conclusions
These findings suggest that combining adaptive gated fusion, ASL, and modality dropout can improve Image-Only performance while preserving multimodal discrimination in long-tailed multi-label CXR classification.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Software Engineering, Capital University of Science and Technology, Islamabad 44000, Pakistan.
- Department of Computer Science, COMSATS University, Islamabad 44000, Pakistan.
- Department of Computer and Software Engineering, College of Electrical and Mechanical Engineering, National University of Sciences and Technology (NUST), Islamabad 44800, Pakistan.
- James Watt School of Engnireeing, University of Glasgow, Glasgow G12 8QQ, UK.
- Department of Electrical and Communication Engineering, College of Engineering, UAE University, Al Ain P.O. Box 15551, United Arab Emirates.
- LyRICA Laboratory, Meridian Team, School of Information Sciences, Rabat 10100, Morocco.
ORCID-Profile: Aqeeb Hussain, Sabeen Masood, Samera Batool, Ali Hassan, Sanaa Ghouzali, Wadood Abdul
Förderung
- King Saud University
- Ongoing Research Funding Program, King Saud University
Zitieren
APA
Hussain, A., Masood, S., Batool, S., Hassan, A., Ansari, S., Ghouzali, S., & Abdul, W. (2026). Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification. Diagnostics, 16(21), 3551. https://doi.org/10.3390/diagnostics16213551
Vancouver
Hussain A, Masood S, Batool S, Hassan A, Ansari S, Ghouzali S, et al. Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification. Diagnostics. 2026;16(21):3551. doi:10.3390/diagnostics16213551
BibTeX
@article{Hussain2026,
author = {Hussain, Aqeeb and Masood, Sabeen and Batool, Samera and Hassan, Ali and Ansari, Shuja and Ghouzali, Sanaa and Abdul, Wadood},
title = {Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification},
journal = {Diagnostics},
year = {2026},
volume = {16},
number = {21},
pages = {3551},
doi = {10.3390/diagnostics16213551}
}
EUVIMED