EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification

Aqeeb Hussain iD, Sabeen Masood iD, Samera Batool iD, Ali Hassan iD, Shuja Ansari, Sanaa Ghouzali iD, Wadood Abdul iD

Diagnostics · 28.10.2026 · 16 (21) · 3551

DOI: 10.3390/diagnostics16213551 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 28.10.2026. Verfasst von Aqeeb Hussain und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Software Engineering, Capital University of Science and Technology, Islamabad 44000, Pakistan. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Saud University, Ongoing Research Funding Program, King Saud University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives: Medical vision–language learning integrates radiographic images and textual information for chest X-ray (CXR) classification. However, in long-tailed multi-label CXR classification, modality imbalance can cause models to rely disproportionately on radiology reports while underutilizing visual information. This study investigates text dominance and proposes a modality-balanced framework to strengthen visual representation while preserving multimodal performance.

Methods

Experiments used MIMIC-CXR with patient-level training, validation, and test splits of 32,942, 4024, and 3955 paired samples, respectively. Swin-Tiny and PubMedBERT served as visual and textual encoders. The proposed framework combines adaptive gated fusion, Asymmetric Loss (ASL), and training-time modality dropout. It was compared with Binary Cross-Entropy (BCE) concatenation, ASL concatenation, gated fusion, and cross-attention fusion.

Results

The BCE baseline achieved a Macro AUROC of 0.9957 under Image + Text evaluation and 0.5804 under Image-Only evaluation, while Text-Only evaluation yielded 0.9955, indicating substantial modality disparity. Other alternatives achieved Image-Only Macro AUROC values of 0.5610–0.5720. The proposed framework improved Image-Only Macro AUROC to 0.7697 and Macro PR-AUC from 0.1814 to 0.3428 while maintaining an Image + Text Macro AUROC of 0.9958. The modality-dominance gap decreased from 0.4151 to 0.2260, a 45.6% reduction in performance disparity. Grad-CAM and Integrated Gradients provided insights into visual evidence and textual attribution, revealing reliance on explicit disease-related terminology in reports.

Conclusions

These findings suggest that combining adaptive gated fusion, ASL, and modality dropout can improve Image-Only performance while preserving multimodal discrimination in long-tailed multi-label CXR classification.

Schlagwörter

Stichwörter der Autor:innen

chest X-raymedical vision–language learningmodality dropoutmodality imbalancemulti-label classificationmultimodal learningtext dominance

Autor:innen und Institutionen

  • Department of Software Engineering, Capital University of Science and Technology, Islamabad 44000, Pakistan.
  • Department of Computer Science, COMSATS University, Islamabad 44000, Pakistan.
  • Department of Computer and Software Engineering, College of Electrical and Mechanical Engineering, National University of Sciences and Technology (NUST), Islamabad 44800, Pakistan.
  • James Watt School of Engnireeing, University of Glasgow, Glasgow G12 8QQ, UK.
  • Department of Electrical and Communication Engineering, College of Engineering, UAE University, Al Ain P.O. Box 15551, United Arab Emirates.
  • LyRICA Laboratory, Meridian Team, School of Information Sciences, Rabat 10100, Morocco.

ORCID-Profile: Aqeeb Hussain, Sabeen Masood, Samera Batool, Ali Hassan, Sanaa Ghouzali, Wadood Abdul

Förderung

  • King Saud University
  • Ongoing Research Funding Program, King Saud University

Zitieren

APA

Hussain, A., Masood, S., Batool, S., Hassan, A., Ansari, S., Ghouzali, S., & Abdul, W. (2026). Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification. Diagnostics, 16(21), 3551. https://doi.org/10.3390/diagnostics16213551

Vancouver

Hussain A, Masood S, Batool S, Hassan A, Ansari S, Ghouzali S, et al. Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification. Diagnostics. 2026;16(21):3551. doi:10.3390/diagnostics16213551

BibTeX
@article{Hussain2026,
  author = {Hussain, Aqeeb and Masood, Sabeen and Batool, Samera and Hassan, Ali and Ansari, Shuja and Ghouzali, Sanaa and Abdul, Wadood},
  title = {Mitigating Text Dominance in Medical Vision–Language Learning for Long-Tailed Multi-Label Chest X-Ray Classification},
  journal = {Diagnostics},
  year = {2026},
  volume = {16},
  number = {21},
  pages = {3551},
  doi = {10.3390/diagnostics16213551}
}
Kostenlos starten →