EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation

Jinnian Zhang, Weijie Chen iD, Tanmayee Joshi, Xiaomin Zhang, Po-Ling Loh, Varun Jog, Richard J Bruce iD, John W Garrett iD, Alan B McMillan iD

Tomography · 13.12.2024 · 10 (12) · 2058-2072

DOI: 10.3390/tomography10120146 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Tomography, online erschienen am 13.12.2024. Verfasst von Jinnian Zhang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA. Hauptthemen laut MEDLINE: Age Determination by Skeleton. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, Engineering and Physical Sciences Research Council, United States National Library of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This research introduces BAE-ViT, a specialized vision transformer model developed for bone age estimation (BAE). This model is designed to efficiently merge image and sex data, a capability not present in traditional convolutional neural networks (CNNs). BAE-ViT employs a novel data fusion method to facilitate detailed interactions between visual and non-visual data by tokenizing non-visual information and concatenating all tokens (visual or non-visual) as the input to the model. The model underwent training on a large-scale dataset from the 2017 RSNA Pediatric Bone Age Machine Learning Challenge, where it exhibited commendable performance, particularly excelling in handling image distortions compared to existing models. The effectiveness of BAE-ViT was further affirmed through statistical analysis, demonstrating a strong correlation with the actual ground-truth labels. This study contributes to the field by showcasing the potential of vision transformers as a viable option for integrating multimodal data in medical imaging applications, specifically emphasizing their capacity to incorporate non-visual elements like sex information into the framework. This tokenization method not only demonstrates superior performance in this specific task but also offers a versatile framework for integrating multimodal data in medical imaging applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Age Determination by SkeletonHumansChildFemaleMaleNeural Networks, ComputerMachine LearningChild, PreschoolImage Processing, Computer-AssistedAdolescentInfantTomography, X-Ray Computed

Stichwörter der Autor:innen

bone age regressiongender embeddingmachine learningmultimodal datavision transformer

Autor:innen und Institutionen

  • Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA.
  • Department of Computer Science, University of Wisconsin-Madison, Madison, WI 53706, USA.
  • Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge CB2 1TN, UK.
  • Department of Radiology, University of Wisconsin-Madison, Madison, WI 53706, USA.
  • Department of Medical Physics, University of Wisconsin-Madison, Madison, WI 53706, USA.
  • Department of Biomedical Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA.

ORCID-Profile: Weijie Chen, Richard J Bruce, John W Garrett, Alan B McMillan

Förderung

  • NLM NIH HHS
  • Engineering and Physical Sciences Research Council
  • United States National Library of Medicine
  • National Science Foundation
  • National Library of Medicine
  • Science and Technology Facilities Council
  • GE Healthcare

Zitieren

APA

Zhang, J., Chen, W., Joshi, T., Zhang, X., Loh, P. L., Jog, V., Bruce, R. J., Garrett, J. W., & McMillan, A. B. (2024). BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation. Tomography, 10(12), 2058-2072. https://doi.org/10.3390/tomography10120146

Vancouver

Zhang J, Chen W, Joshi T, Zhang X, Loh PL, Jog V, et al. BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation. Tomography. 2024;10(12):2058-2072. doi:10.3390/tomography10120146

BibTeX
@article{Zhang2024,
  author = {Zhang, Jinnian and Chen, Weijie and Joshi, Tanmayee and Zhang, Xiaomin and Loh, Po-Ling and Jog, Varun and Bruce, Richard J and Garrett, John W and McMillan, Alan B},
  title = {BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation},
  journal = {Tomography},
  year = {2024},
  volume = {10},
  number = {12},
  pages = {2058--2072},
  doi = {10.3390/tomography10120146}
}
Kostenlos starten →