EUVIMEDEUVIMED entdecken
Crossref

Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub

Hana Yoo, Youngbok Cho

Journal of Biomedical Research & Environmental Sciences · 2026-08 · 7 (8) · 1-13

DOI: 10.37871/jbres2321 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Biomedical Research & Environmental Sciences, 2026-08. Verfasst von Hana Yoo und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Automated lesion classification and interpretation of pediatric abdominal X-rays is clinically important but constrained by data scarcity and strict privacy regulations. Synthetic image-caption pair datasets offer a practical solution.

Methods

We used the AI Hub synthetic pediatric abdominal X-ray dataset (10,000 image-caption pairs, five disease classes, stratified 8:1:1 split with atomic pair splitting to prevent data leakage) to develop a BioMedCLIP-based multimodal system. The system integrates a ViT-B/16 visual encoder fine-tuned with InfoNCE contrastive loss, an MLP classification head with MC Dropout for uncertainty quantification (T = 20 forward passes), and a three-tier confidence-conditioned report generation module. Clinical performance was assessed using accuracy, sensitivity, specificity, precision, and F1-score; report generation was assessed with BLEU-4, ROUGE-L, and CIDEr. ResNet-50 (B1) and DenseNet-121 (B2) served as classification baselines; template-based (B_tmpl) and LLaVA-Med (B3) as generation baselines.

Results

The proposed model achieved accuracy 97.95%, sensitivity 91.57%, specificity 96.28%, and macro F1-score 93.49%, outperforming B1 (+5.04 pp) and B2 (+2.55 pp) in accuracy. For report generation, BLEU-4 6.46, ROUGE-L 20.51, and CIDEr 49.26 exceeded both baselines on all metrics. Grad-CAM analysis confirmed anatomically plausible attention patterns.

Conclusions

Multimodal vision-language contrastive learning on synthetic image-caption data yields consistent gains over single-modality baselines. The sim-to-real distribution gap and the need for external clinical validation are identified as the primary barriers to clinical translation.

Zitieren

APA

Yoo, H., & Cho, Y. (2026). Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub. Journal of Biomedical Research & Environmental Sciences, 7(8), 1-13. https://doi.org/10.37871/jbres2321

Vancouver

Yoo H, Cho Y. Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub. Journal of Biomedical Research & Environmental Sciences. 2026;7(8):1-13. doi:10.37871/jbres2321

BibTeX
@article{Yoo2026,
  author = {Yoo, Hana and Cho, Youngbok},
  title = {Multimodal Lesion Classification and Automated Interpretation Algorithm using Image-Caption Pairs of Pediatric Abdominal X-ray Synthetic Data Provided by AI-Hub},
  journal = {Journal of Biomedical Research & Environmental Sciences},
  year = {2026},
  volume = {7},
  number = {8},
  pages = {1--13},
  doi = {10.37871/jbres2321}
}
Kostenlos starten →