EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Survey on Multimodal Large Language Models in Radiology for Report Generation and Visual Question Answering

Ziruo Yi, Ting Xiao, Mark V. Albert

Information · 12.02.2025 · 16 (2) · 136

DOI: 10.3390/info16020136 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 12.02.2025. Verfasst von Ziruo Yi und 2 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 215 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) and large vision models (LVMs) have driven significant advancements in natural language processing (NLP) and computer vision (CV), establishing a foundation for multimodal large language models (MLLMs) to integrate diverse data types in real-world applications. This survey explores the evolution of MLLMs in radiology, focusing on radiology report generation (RRG) and radiology visual question answering (RVQA), where MLLMs leverage the combined capabilities of LLMs and LVMs to improve clinical efficiency. We begin by tracing the history of radiology and the development of MLLMs, followed by an overview of MLLM applications in RRG and RVQA, detailing core datasets, evaluation metrics, and leading MLLMs that demonstrate their potential in generating radiology reports and answering image-based questions. We then discuss the challenges MLLMs face in radiology, including dataset scarcity, data privacy and security, and issues within MLLMs such as bias, toxicity, hallucinations, catastrophic forgetting, and limitations in traditional evaluation metrics. Finally, this paper proposes future research directions to address these challenges, aiming to help AI researchers and radiologists overcome these obstacles and advance the study of MLLMs in radiology.

Zitieren

APA

Yi, Z., Xiao, T., & Albert, M. V. (2025). A Survey on Multimodal Large Language Models in Radiology for Report Generation and Visual Question Answering. Information, 16(2), 136. https://doi.org/10.3390/info16020136

Vancouver

Yi Z, Xiao T, Albert MV. A Survey on Multimodal Large Language Models in Radiology for Report Generation and Visual Question Answering. Information. 2025;16(2):136. doi:10.3390/info16020136

BibTeX
@article{Yi2025,
  author = {Yi, Ziruo and Xiao, Ting and Albert, Mark V.},
  title = {A Survey on Multimodal Large Language Models in Radiology for Report Generation and Visual Question Answering},
  journal = {Information},
  year = {2025},
  volume = {16},
  number = {2},
  pages = {136},
  doi = {10.3390/info16020136}
}
Kostenlos starten →