EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Proactive Agent Collaborative Framework for Zero‐Shot Multimodal Medical Reasoning

Zishan Gu iD, Fenglin Liu, Jiayuan Chen, Changchang Yin, Ping Zhang iD

Advanced Intelligent Systems · 2025-08 · 7 (8)

DOI: 10.1002/aisy.202400840 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advanced Intelligent Systems, online erschienen am 05.02.2025. Verfasst von Zishan Gu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Engineering The Ohio State University Columbus OH USA. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIAID NIH HHS, National Science Foundation, Foundation for the National Institutes of Health.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The adoption of large language models (LLMs) in healthcare has garnered significant research interest, yet their performance remains limited due to a lack of domain‐specific knowledge, medical reasoning skills, and their unimodal nature, which restricts them to text‐only inputs. To address these limitations, we propose MultiMedRes, a multimodal medical collaborative reasoning framework that simulates human physicians’ communication by incorporating a learner agent to proactively acquire information from domain‐specific expert models. MultiMedRes addresses medical multimodal reasoning problems through three steps i) Inquire: The learner agent decomposes complex medical reasoning problems into multiple domain‐specific sub‐problems; ii) Interact: The agent engages in iterative “ask‐answer” interactions with expert models to obtain domain‐specific knowledge; and iii) Integrate: The agent integrates all the acquired domain‐specific knowledge to address the medical reasoning problems (e.g., identifying the difference of disease levels and abnormality sizes between medical images). We validate the effectiveness of our method on the task of difference visual question answering for X‐ray images. The experiments show that our zero‐shot prediction achieves state‐of‐the‐art performance, surpassing fully supervised methods, which demonstrates that MultiMedRes could offer trustworthy and interpretable assistance to physicians in monitoring the treatment progression of patients, paving the way for effective human–AI interaction and collaboration.

Schlagwörter

Stichwörter der Autor:innen

AI agentlarge language modelmultimodal medical reasoning

Autor:innen und Institutionen

  • Department of Computer Science and Engineering The Ohio State University Columbus OH USA.
  • Department of Biomedical Informatics The Ohio State University Columbus OH USA.
  • Department of Engineering Science, Institute of Biomedical Engineering University of Oxford Oxford UK.
  • Translational Data Analytics Institute The Ohio State University Columbus OH USA.

ORCID-Profile: Zishan Gu, Ping Zhang

Förderung

  • NIAID NIH HHS
  • National Science Foundation
  • Foundation for the National Institutes of Health

Zitieren

APA

Gu, Z., Liu, F., Chen, J., Yin, C., & Zhang, P. (2025). A Proactive Agent Collaborative Framework for Zero‐Shot Multimodal Medical Reasoning. Advanced Intelligent Systems, 7(8). https://doi.org/10.1002/aisy.202400840

Vancouver

Gu Z, Liu F, Chen J, Yin C, Zhang P. A Proactive Agent Collaborative Framework for Zero‐Shot Multimodal Medical Reasoning. Advanced Intelligent Systems. 2025;7(8). doi:10.1002/aisy.202400840

BibTeX
@article{Gu2025,
  author = {Gu, Zishan and Liu, Fenglin and Chen, Jiayuan and Yin, Changchang and Zhang, Ping},
  title = {A Proactive Agent Collaborative Framework for Zero‐Shot Multimodal Medical Reasoning},
  journal = {Advanced Intelligent Systems},
  year = {2025},
  volume = {7},
  number = {8},
  doi = {10.1002/aisy.202400840}
}
Kostenlos starten →