Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook
Journal of Medical Internet Research · 25.09.2024 · 26 · e59505
DOI: 10.2196/59505 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Medical Internet Research, online erschienen am 25.09.2024. Verfasst von Rawan AlSaad und 6 weiteren Autor:innen. Beteiligte Einrichtung: Weill Cornell Medicine-Qatar, Education City, Doha, Qatar. Hauptthemen laut MEDLINE: Delivery of Health Care. Die Arbeit wurde bisher 391-mal zitiert (Crossref) und nennt 199 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated, multimodal data–driven medical practice. We anticipate that this work will spark further discussion and inspire the development of innovative approaches in the next generation of medical M-LLM systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Weill Cornell Medicine-Qatar, Education City, Doha, Qatar.
- Qatar Computing Research Institute, Hamad Bin Khalifa University, Doha, Qatar.
- Department of Computer Science and Software Engineering, United Arab Emirates University, Al Ain, United Arab Emirates.
ORCID-Profile: Rawan AlSaad, Alaa Abd-Alrazaq, Sabri Boughorbel, Arfan Ahmed, Max-Antoine Renault, Rafat Damseh, Javaid Sheikh
Zitieren
APA
AlSaad, R., Abd-Alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M. A., Damseh, R., & Sheikh, J. (2024). Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. Journal of Medical Internet Research, 26, e59505. https://doi.org/10.2196/59505
Vancouver
AlSaad R, Abd-Alrazaq A, Boughorbel S, Ahmed A, Renault MA, Damseh R, et al. Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. Journal of Medical Internet Research. 2024;26:e59505. doi:10.2196/59505
BibTeX
@article{AlSaad2024,
author = {AlSaad, Rawan and Abd-Alrazaq, Alaa and Boughorbel, Sabri and Ahmed, Arfan and Renault, Max-Antoine and Damseh, Rafat and Sheikh, Javaid},
title = {Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook},
journal = {Journal of Medical Internet Research},
year = {2024},
volume = {26},
pages = {e59505},
doi = {10.2196/59505}
}
EUVIMED