MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning
Journal of the American Medical Informatics Association · 01.10.2025 · 32 (10) · 1505-1516
DOI: 10.1093/jamia/ocaf128 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 04.08.2025. Verfasst von Zaifu Zhan und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, United States. Hauptthemen laut MEDLINE: Natural Language Processing, Information Storage and Retrieval, Machine Learning, Data Mining. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NCCIH NIH HHS, NIDDK NIH HHS.
Abstract (Originalsprache)
Objectives
To optimize in-context learning in biomedical natural language processing by improving example selection. Materials and Methods We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework, which integrates 4 retrieval strategies: (1) Random Mode, selecting examples arbitrarily; (2) Top Mode, retrieving the most relevant examples based on similarity; (3) Diversity Mode, ensuring variation in selected examples; and (4) Class Mode, selecting category-representative examples. This study evaluates MMRAG on 3 core biomedical NLP tasks: Named Entity Recognition (NER), Relation Extraction (RE), and Text Classification (TC). The datasets used include BC2GM for gene and protein mention recognition (NER), DDI for drug-drug interaction extraction (RE), GIT for general biomedical information extraction (RE), and HealthAdvice for health-related text classification (TC). The framework is tested with 2 large language models (Llama-2-7B and Llama-3-8B) and 3 retrievers (Contriever, MedCPT, and BGE-Large) to assess performance across different retrieval strategies.
Results
The results from the Random Mode indicate that providing more examples in the prompt improves the model’s generation performance. Meanwhile, Top Mode and Diversity Mode significantly outperform Random Mode on the RE (DDI) task, achieving an F1 score of 0.9669—a 26.4% improvement. Among the 3 retrievers tested, Contriever outperformed the other 2 in a greater number of experiments. Additionally, Llama 2 and Llama 3 demonstrated varying capabilities across different tasks, with Llama 3 showing a clear advantage in handling NER tasks.
Conclusion
MMRAG effectively enhances biomedical in-context learning by refining example selection, mitigating data scarcity issues, and demonstrating superior adaptability for NLP-driven healthcare applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, United States.
- Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States.
- Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN 55455, United States.
ORCID-Profile: Rui Zhang
Förderung
- NIA NIH HHS
- NCCIH NIH HHS
- NIDDK NIH HHS
- NIMHD NIH HHS
- NIH HHS
- NCI NIH HHS
- National Institutes of Health's
- National Institutes of Health’s
Zitieren
APA
Zhan, Z., Wang, J., Zhou, S., Deng, J., & Zhang, R. (2025). MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning. Journal of the American Medical Informatics Association, 32(10), 1505-1516. https://doi.org/10.1093/jamia/ocaf128
Vancouver
Zhan Z, Wang J, Zhou S, Deng J, Zhang R. MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning. Journal of the American Medical Informatics Association. 2025;32(10):1505-1516. doi:10.1093/jamia/ocaf128
BibTeX
@article{Zhan2025,
author = {Zhan, Zaifu and Wang, Jun and Zhou, Shuang and Deng, Jiawen and Zhang, Rui},
title = {MMRAG: multi-mode retrieval-augmented generation with large language models for biomedical in-context learning},
journal = {Journal of the American Medical Informatics Association},
year = {2025},
volume = {32},
number = {10},
pages = {1505--1516},
doi = {10.1093/jamia/ocaf128}
}
EUVIMED