Effects of Model Choice, Corpus Context, and Post Hoc Correction on Layer-Level Embedding Degradation in Clinical Document Retrieval: Experimental Study
JMIR Medical Informatics · 29.07.2026 · 14 · e99639
DOI: 10.2196/99639 ↗Kurz gefasst
Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 29.07.2026. Verfasst von Yngve Mikkelsen. Beteiligte Einrichtung: Saïd Business School, University of Oxford, Oxford, England, United Kingdom. Hauptthemen laut MEDLINE: Information Storage and Retrieval, Electronic Health Records. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Clinical retrieval-augmented generation depends on embedding models. A companion study found that non–retrieval-trained encoders underperformed retrieval-trained general-purpose embeddings and produced near-degenerate embedding geometry, but did not localize the architectural origin, separate training-domain from training-objective effects, or test whether the degradation can be corrected without retraining.
Objective
This study aimed to (1) characterize layer-wise retrieval and geometric trajectories across 13 transformer configurations on clinical documents, (2) separate training-objective from training-domain effects through matched architectural comparisons, (3) reanalyze the panel under a per-query layer-wise linear mixed-effects (LME) framework, and (4) evaluate deployment-relevant post hoc geometric correction.
Methods
Layer-wise embeddings were extracted from 13 transformer configurations on 3 clinical corpora (n=100 documents each: MTSamples, PMC-Patients, and Mistral-7B-Instruct–generated synthetic notes) under 2 query formats (keyword and natural-language via GPT-4o). Retrieval performance (mean reciprocal rank at cutoff 10 [MRR@10] and recall at cutoff 10 [recall@10]) and geometric properties (participation ratio, average pairwise cosine, and anisotropy) were measured at every layer. A per-query layer-wise LME model was fit independently per configuration. Corpus-only zero-phase component analysis (ZCA) whitening was evaluated as the primary deployment-relevant intervention, with 5-fold cross-validation, an epsilon sweep, a lexical-overlap audit, and a chunking sensitivity analysis.
Results
Document embeddings clustered into 3 anisotropy tiers: extreme (average pairwise cosine >0.92) for non–retrieval-trained encoders and large language models (LLMs), moderate (0.65-0.92) for general retrievers and most LLMs, and reduced (
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Saïd Business School, University of Oxford, Oxford, England, United Kingdom.
ORCID-Profile: Yngve Mikkelsen
Zitieren
APA
Mikkelsen, Y. (2026). Effects of Model Choice, Corpus Context, and Post Hoc Correction on Layer-Level Embedding Degradation in Clinical Document Retrieval: Experimental Study. JMIR Medical Informatics, 14, e99639. https://doi.org/10.2196/99639
Vancouver
Mikkelsen Y. Effects of Model Choice, Corpus Context, and Post Hoc Correction on Layer-Level Embedding Degradation in Clinical Document Retrieval: Experimental Study. JMIR Medical Informatics. 2026;14:e99639. doi:10.2196/99639
BibTeX
@article{Mikkelsen2026,
author = {Mikkelsen, Yngve},
title = {Effects of Model Choice, Corpus Context, and Post Hoc Correction on Layer-Level Embedding Degradation in Clinical Document Retrieval: Experimental Study},
journal = {JMIR Medical Informatics},
year = {2026},
volume = {14},
pages = {e99639},
doi = {10.2196/99639}
}
EUVIMED