EUVIMEDEUVIMED entdecken
Open Access

Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook

Rawan AlSaad iD, Alaa Abd-Alrazaq iD, Sabri Boughorbel iD, Arfan Ahmed iD, Max-Antoine Renault iD, Rafat Damseh iD, Javaid Sheikh iD

Journal of Medical Internet Research · 25.09.2024 · 26 · e59505

DOI: 10.2196/59505 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Medical Internet Research, online erschienen am 25.09.2024. Verfasst von Rawan AlSaad und 6 weiteren Autor:innen. Beteiligte Einrichtung: Weill Cornell Medicine-Qatar, Education City, Doha, Qatar. Hauptthemen laut MEDLINE: Delivery of Health Care. Die Arbeit wurde bisher 391-mal zitiert (Crossref) und nennt 199 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated, multimodal data–driven medical practice. We anticipate that this work will spark further discussion and inspire the development of innovative approaches in the next generation of medical M-LLM systems.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Delivery of Health CareHumansNatural Language ProcessingElectronic Health Records

Stichwörter der Autor:innen

artificial intelligencegenerative AIgenerative artificial intelligencehealth carelarge language modelsmultimodal generative AImultimodal generative artificial intelligencemultimodal large language modelsmultimodality

Autor:innen und Institutionen

  • Weill Cornell Medicine-Qatar, Education City, Doha, Qatar.
  • Qatar Computing Research Institute, Hamad Bin Khalifa University, Doha, Qatar.
  • Department of Computer Science and Software Engineering, United Arab Emirates University, Al Ain, United Arab Emirates.

ORCID-Profile: Rawan AlSaad, Alaa Abd-Alrazaq, Sabri Boughorbel, Arfan Ahmed, Max-Antoine Renault, Rafat Damseh, Javaid Sheikh

Zitieren

APA

AlSaad, R., Abd-Alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M. A., Damseh, R., & Sheikh, J. (2024). Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. Journal of Medical Internet Research, 26, e59505. https://doi.org/10.2196/59505

Vancouver

AlSaad R, Abd-Alrazaq A, Boughorbel S, Ahmed A, Renault MA, Damseh R, et al. Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. Journal of Medical Internet Research. 2024;26:e59505. doi:10.2196/59505

BibTeX
@article{AlSaad2024,
  author = {AlSaad, Rawan and Abd-Alrazaq, Alaa and Boughorbel, Sabri and Ahmed, Arfan and Renault, Max-Antoine and Damseh, Rafat and Sheikh, Javaid},
  title = {Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e59505},
  doi = {10.2196/59505}
}
Kostenlos starten →