EUVIMEDEUVIMED entdecken
Open Access

Clinical Context Variables Collectively Rival Model Choice in Embedding-Based Retrieval: Multi-Corpus Benchmark Study

Yngve Mikkelsen iD

JMIR Medical Informatics · 07.05.2026 · 14 · e94241

DOI: 10.2196/94241 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 07.05.2026. Verfasst von Yngve Mikkelsen. Beteiligte Einrichtung: Saïd Business School, University of Oxford, Oxford, England, United Kingdom. Hauptthemen laut MEDLINE: Benchmarking, Information Storage and Retrieval, Natural Language Processing. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Retrieval-augmented generation (RAG) systems increasingly support clinical decision-making by grounding large language model outputs in verifiable evidence. The retrieval component is foundational: if the correct document is not retrieved, downstream generation cannot recover it. Despite this, embedding model selection for clinical RAG remains guided by general-domain benchmarks with limited clinical coverage. Given the heterogeneity of clinical documentation across institutions, specialties, and electronic health record systems, it is unclear whether general-domain model rankings generalize to clinical retrieval tasks.

Objective

This study evaluated whether clinical context variables, corpus type (encompassing differences in document length, medical specialty, and structural characteristics), and query format have effects on retrieval performance comparable to or exceeding those of embedding model choice.

Methods

Ten primary embedding models plus two ablation variants and a BM25 lexical baseline (13 retrieval configurations total) were benchmarked on three clinical corpora (MTSamples medical transcriptions, n=500; PMC-Patients case reports, n=500; Mistral-7B-generated synthetic clinical notes, n=500). Twelve embedding configurations were evaluated across 3 corpora × 2 query formats (keyword vs natural language) × 4 chunking strategies, yielding 294 experimental conditions. Primary metrics included MRR@10, P@1, Recall@10/20/50/100, and NDCG@10, with bootstrap confidence intervals. Relative factor contributions were quantified using factorial ANOVA with η² effect sizes, including all two-way interactions.

Results

In a factorial ANOVA across 288 balanced embedding conditions, embedding model choice explained 40.8% of variance in MRR@10 (η²=0.408), corpus type 24.6%, and query format 19.2%. Chunking strategy explained minimal variance (η²=0.002). The model × query format interaction (η²=0.029, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BenchmarkingInformation Storage and RetrievalNatural Language ProcessingHumansElectronic Health RecordsLarge Language Models

Stichwörter der Autor:innen

BM25benchmarkclinical documentationclinical informaticsdense retrievalembedding modelsretrieval-augmented generation

Autor:innen und Institutionen

  • Saïd Business School, University of Oxford, Oxford, England, United Kingdom.

ORCID-Profile: Yngve Mikkelsen

Zitieren

APA

Mikkelsen, Y. (2026). Clinical Context Variables Collectively Rival Model Choice in Embedding-Based Retrieval: Multi-Corpus Benchmark Study. JMIR Medical Informatics, 14, e94241. https://doi.org/10.2196/94241

Vancouver

Mikkelsen Y. Clinical Context Variables Collectively Rival Model Choice in Embedding-Based Retrieval: Multi-Corpus Benchmark Study. JMIR Medical Informatics. 2026;14:e94241. doi:10.2196/94241

BibTeX
@article{Mikkelsen2026,
  author = {Mikkelsen, Yngve},
  title = {Clinical Context Variables Collectively Rival Model Choice in Embedding-Based Retrieval: Multi-Corpus Benchmark Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  volume = {14},
  pages = {e94241},
  doi = {10.2196/94241}
}
Kostenlos starten →