EUVIMEDEUVIMED entdecken
Open Access

MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning

Zaifu Zhan, Jun Wang, Shuang Zhou, Jiawen Deng, Rui Zhang iD

Journal of the American Medical Informatics Association · 01.10.2025 · 32 (10) · 1505-1516

DOI: 10.1093/jamia/ocaf128 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 04.08.2025. Verfasst von Zaifu Zhan und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, United States. Hauptthemen laut MEDLINE: Natural Language Processing, Information Storage and Retrieval, Machine Learning, Data Mining. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NCCIH NIH HHS, NIDDK NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To optimize in-context learning in biomedical natural language processing by improving example selection. Materials and Methods We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework, which integrates 4 retrieval strategies: (1) Random Mode, selecting examples arbitrarily; (2) Top Mode, retrieving the most relevant examples based on similarity; (3) Diversity Mode, ensuring variation in selected examples; and (4) Class Mode, selecting category-representative examples. This study evaluates MMRAG on 3 core biomedical NLP tasks: Named Entity Recognition (NER), Relation Extraction (RE), and Text Classification (TC). The datasets used include BC2GM for gene and protein mention recognition (NER), DDI for drug-drug interaction extraction (RE), GIT for general biomedical information extraction (RE), and HealthAdvice for health-related text classification (TC). The framework is tested with 2 large language models (Llama-2-7B and Llama-3-8B) and 3 retrievers (Contriever, MedCPT, and BGE-Large) to assess performance across different retrieval strategies.

Results

The results from the Random Mode indicate that providing more examples in the prompt improves the model’s generation performance. Meanwhile, Top Mode and Diversity Mode significantly outperform Random Mode on the RE (DDI) task, achieving an F1 score of 0.9669—a 26.4% improvement. Among the 3 retrievers tested, Contriever outperformed the other 2 in a greater number of experiments. Additionally, Llama 2 and Llama 3 demonstrated varying capabilities across different tasks, with Llama 3 showing a clear advantage in handling NER tasks.

Conclusion

MMRAG effectively enhances biomedical in-context learning by refining example selection, mitigating data scarcity issues, and demonstrating superior adaptability for NLP-driven healthcare applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingInformation Storage and RetrievalMachine LearningData MiningHumansLarge Language Models

Stichwörter der Autor:innen

in-context learninglarge language modelretrieval augmented generation

Autor:innen und Institutionen

  • Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, United States.
  • Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States.
  • Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN 55455, United States.

ORCID-Profile: Rui Zhang

Förderung

  • NIA NIH HHS
  • NCCIH NIH HHS
  • NIDDK NIH HHS
  • NIMHD NIH HHS
  • NIH HHS
  • NCI NIH HHS
  • National Institutes of Health's
  • National Institutes of Health’s

Zitieren

APA

Zhan, Z., Wang, J., Zhou, S., Deng, J., & Zhang, R. (2025). MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning. Journal of the American Medical Informatics Association, 32(10), 1505-1516. https://doi.org/10.1093/jamia/ocaf128

Vancouver

Zhan Z, Wang J, Zhou S, Deng J, Zhang R. MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning. Journal of the American Medical Informatics Association. 2025;32(10):1505-1516. doi:10.1093/jamia/ocaf128

BibTeX
@article{Zhan2025,
  author = {Zhan, Zaifu and Wang, Jun and Zhou, Shuang and Deng, Jiawen and Zhang, Rui},
  title = {MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning},
  journal = {Journal of the American Medical Informatics Association},
  year = {2025},
  volume = {32},
  number = {10},
  pages = {1505--1516},
  doi = {10.1093/jamia/ocaf128}
}
Kostenlos starten →