BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation
Tomography · 13.12.2024 · 10 (12) · 2058-2072
DOI: 10.3390/tomography10120146 ↗Kurz gefasst
Fachartikel in der Zeitschrift Tomography, online erschienen am 13.12.2024. Verfasst von Jinnian Zhang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA. Hauptthemen laut MEDLINE: Age Determination by Skeleton. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, Engineering and Physical Sciences Research Council, United States National Library of Medicine.
Abstract (Originalsprache)
This research introduces BAE-ViT, a specialized vision transformer model developed for bone age estimation (BAE). This model is designed to efficiently merge image and sex data, a capability not present in traditional convolutional neural networks (CNNs). BAE-ViT employs a novel data fusion method to facilitate detailed interactions between visual and non-visual data by tokenizing non-visual information and concatenating all tokens (visual or non-visual) as the input to the model. The model underwent training on a large-scale dataset from the 2017 RSNA Pediatric Bone Age Machine Learning Challenge, where it exhibited commendable performance, particularly excelling in handling image distortions compared to existing models. The effectiveness of BAE-ViT was further affirmed through statistical analysis, demonstrating a strong correlation with the actual ground-truth labels. This study contributes to the field by showcasing the potential of vision transformers as a viable option for integrating multimodal data in medical imaging applications, specifically emphasizing their capacity to incorporate non-visual elements like sex information into the framework. This tokenization method not only demonstrates superior performance in this specific task but also offers a versatile framework for integrating multimodal data in medical imaging applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA.
- Department of Computer Science, University of Wisconsin-Madison, Madison, WI 53706, USA.
- Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge CB2 1TN, UK.
- Department of Radiology, University of Wisconsin-Madison, Madison, WI 53706, USA.
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI 53706, USA.
- Department of Biomedical Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA.
ORCID-Profile: Weijie Chen, Richard J Bruce, John W Garrett, Alan B McMillan
Förderung
- NLM NIH HHS
- Engineering and Physical Sciences Research Council
- United States National Library of Medicine
- National Science Foundation
- National Library of Medicine
- Science and Technology Facilities Council
- GE Healthcare
Zitieren
APA
Zhang, J., Chen, W., Joshi, T., Zhang, X., Loh, P. L., Jog, V., Bruce, R. J., Garrett, J. W., & McMillan, A. B. (2024). BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation. Tomography, 10(12), 2058-2072. https://doi.org/10.3390/tomography10120146
Vancouver
Zhang J, Chen W, Joshi T, Zhang X, Loh PL, Jog V, et al. BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation. Tomography. 2024;10(12):2058-2072. doi:10.3390/tomography10120146
BibTeX
@article{Zhang2024,
author = {Zhang, Jinnian and Chen, Weijie and Joshi, Tanmayee and Zhang, Xiaomin and Loh, Po-Ling and Jog, Varun and Bruce, Richard J and Garrett, John W and McMillan, Alan B},
title = {BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation},
journal = {Tomography},
year = {2024},
volume = {10},
number = {12},
pages = {2058--2072},
doi = {10.3390/tomography10120146}
}
EUVIMED