Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering
International Journal of Artificial Intelligence and Cognitive Psychology · 28.06.2026 · 2 (1) · 79
DOI: 10.66419/ijaicp.5996 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Artificial Intelligence and Cognitive Psychology, online erschienen am 28.06.2026. Verfasst von BUHARI Bashir und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study improves and extends a thesis-derived investigation by providing a focused comparative evaluation of three embedding models - SFR Embedding Mistral, BGE-large-en-v1.5 and GTE-Qwen2 − 7B-instruct - within a biomedical RAG pipeline. The evaluation used the CovidQA and BioASQ datasets and assessed retrieval effectiveness using hit rate and mean reciprocal rank, while answer quality was measured using semantic similarity, faithfulness, relevancy and F1-score. The findings show that SFR Embedding Mistral achieved the strongest overall performance on CovidQA, recording a hit rate of 0.90, MRR of 0.65, semantic similarity of 0.73 and faithfulness of 0.82 at top-k = 5. On BioASQ, SFR and BGE produced an identical hit rate (0.91), MRR (0.82), relevancy (0.93) and F1-score (0.22), but SFR achieved a higher semantic similarity and slightly higher faithfulness. Nevertheless, BGE-large-en-v1.5 required substantially lower storage and memory resources than SFR and GTE, making it more suitable for resource-constr
Zitieren
APA
Bashir, B., Audu, S. A., Abdullahi, R. A., Muhammad, J., & Faruk, U. U. (2026). Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering. International Journal of Artificial Intelligence and Cognitive Psychology, 2(1), 79. https://doi.org/10.66419/ijaicp.5996
Vancouver
Bashir B, Audu SA, Abdullahi RA, Muhammad J, Faruk UU. Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering. International Journal of Artificial Intelligence and Cognitive Psychology. 2026;2(1):79. doi:10.66419/ijaicp.5996
BibTeX
@article{Bashir2026,
author = {Bashir, BUHARI and Audu, SALIHU Abdullahi and Abdullahi, RABIU Asmau and Muhammad, JIBRIL and Faruk, USMAN Umar},
title = {Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering},
journal = {International Journal of Artificial Intelligence and Cognitive Psychology},
year = {2026},
volume = {2},
number = {1},
pages = {79},
doi = {10.66419/ijaicp.5996}
}
EUVIMED