EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering

BUHARI Bashir, SALIHU Abdullahi Audu, RABIU Asmau Abdullahi, JIBRIL Muhammad, USMAN Umar Faruk

International Journal of Artificial Intelligence and Cognitive Psychology · 28.06.2026 · 2 (1) · 79

DOI: 10.66419/ijaicp.5996 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Artificial Intelligence and Cognitive Psychology, online erschienen am 28.06.2026. Verfasst von BUHARI Bashir und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study improves and extends a thesis-derived investigation by providing a focused comparative evaluation of three embedding models - SFR Embedding Mistral, BGE-large-en-v1.5 and GTE-Qwen2 − 7B-instruct - within a biomedical RAG pipeline. The evaluation used the CovidQA and BioASQ datasets and assessed retrieval effectiveness using hit rate and mean reciprocal rank, while answer quality was measured using semantic similarity, faithfulness, relevancy and F1-score. The findings show that SFR Embedding Mistral achieved the strongest overall performance on CovidQA, recording a hit rate of 0.90, MRR of 0.65, semantic similarity of 0.73 and faithfulness of 0.82 at top-k = 5. On BioASQ, SFR and BGE produced an identical hit rate (0.91), MRR (0.82), relevancy (0.93) and F1-score (0.22), but SFR achieved a higher semantic similarity and slightly higher faithfulness. Nevertheless, BGE-large-en-v1.5 required substantially lower storage and memory resources than SFR and GTE, making it more suitable for resource-constr

Zitieren

APA

Bashir, B., Audu, S. A., Abdullahi, R. A., Muhammad, J., & Faruk, U. U. (2026). Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering. International Journal of Artificial Intelligence and Cognitive Psychology, 2(1), 79. https://doi.org/10.66419/ijaicp.5996

Vancouver

Bashir B, Audu SA, Abdullahi RA, Muhammad J, Faruk UU. Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering. International Journal of Artificial Intelligence and Cognitive Psychology. 2026;2(1):79. doi:10.66419/ijaicp.5996

BibTeX
@article{Bashir2026,
  author = {Bashir, BUHARI and Audu, SALIHU Abdullahi and Abdullahi, RABIU Asmau and Muhammad, JIBRIL and Faruk, USMAN Umar},
  title = {Comparative Evaluation of Embedding Models for Retrieval-Augmented Generation in Biomedical Question Answering},
  journal = {International Journal of Artificial Intelligence and Cognitive Psychology},
  year = {2026},
  volume = {2},
  number = {1},
  pages = {79},
  doi = {10.66419/ijaicp.5996}
}
Kostenlos starten →