Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub
Journal of Biomedical Research & Environmental Sciences · 2026-08 · 7 (8) · 1-13
DOI: 10.37871/jbres2321 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Biomedical Research & Environmental Sciences, 2026-08. Verfasst von Hana Yoo und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Background
Automated lesion classification and interpretation of pediatric abdominal X-rays is clinically important but constrained by data scarcity and strict privacy regulations. Synthetic image-caption pair datasets offer a practical solution.
Methods
We used the AI Hub synthetic pediatric abdominal X-ray dataset (10,000 image-caption pairs, five disease classes, stratified 8:1:1 split with atomic pair splitting to prevent data leakage) to develop a BioMedCLIP-based multimodal system. The system integrates a ViT-B/16 visual encoder fine-tuned with InfoNCE contrastive loss, an MLP classification head with MC Dropout for uncertainty quantification (T = 20 forward passes), and a three-tier confidence-conditioned report generation module. Clinical performance was assessed using accuracy, sensitivity, specificity, precision, and F1-score; report generation was assessed with BLEU-4, ROUGE-L, and CIDEr. ResNet-50 (B1) and DenseNet-121 (B2) served as classification baselines; template-based (B_tmpl) and LLaVA-Med (B3) as generation baselines.
Results
The proposed model achieved accuracy 97.95%, sensitivity 91.57%, specificity 96.28%, and macro F1-score 93.49%, outperforming B1 (+5.04 pp) and B2 (+2.55 pp) in accuracy. For report generation, BLEU-4 6.46, ROUGE-L 20.51, and CIDEr 49.26 exceeded both baselines on all metrics. Grad-CAM analysis confirmed anatomically plausible attention patterns.
Conclusions
Multimodal vision-language contrastive learning on synthetic image-caption data yields consistent gains over single-modality baselines. The sim-to-real distribution gap and the need for external clinical validation are identified as the primary barriers to clinical translation.
Zitieren
APA
Yoo, H., & Cho, Y. (2026). Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub. Journal of Biomedical Research & Environmental Sciences, 7(8), 1-13. https://doi.org/10.37871/jbres2321
Vancouver
Yoo H, Cho Y. Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub. Journal of Biomedical Research & Environmental Sciences. 2026;7(8):1-13. doi:10.37871/jbres2321
BibTeX
@article{Yoo2026,
author = {Yoo, Hana and Cho, Youngbok},
title = {Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub},
journal = {Journal of Biomedical Research & Environmental Sciences},
year = {2026},
volume = {7},
number = {8},
pages = {1--13},
doi = {10.37871/jbres2321}
}
EUVIMED