Probabilistic Coherence of Stated Diagnostic Beliefs in Large Language Models: Anchored Tests on a Synthetic Clinical Corpus
Machine Learning and Knowledge Extraction · 2026-11-07 · 8 (11) · 369
DOI: 10.3390/make8110369 ↗At a glance
Journal article in Machine Learning and Knowledge Extraction, published online on 2026-11-07. Written by Govind Arun and 2 co-authors. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
Clinical evidence arrives in sequence, yet a coherent diagnostician’s belief should depend on what was found, not on the order of finding it. Bayesian belief revision is order-invariant as a theorem, which yields an exact, annotation-free test, with target zero, of the distribution a clinical language model reports. Three further axioms cover update fidelity, redundancy insensitivity, and positional anchoring. Eleven open-weight configurations of 4B to 32B parameters are evaluated on 1.12 million responses over 1956 patients and 49 pathologies of the synthetic DDXPlus corpus. Every divergence is anchored to the model’s own test–retest floor and between-patient ceiling, and to a corpus replicate floor of 0.049. On the Qwen3 ladder, raw permutation divergence rises from 0.128 to 0.291 with scale, while the order effect as a fraction of each model’s floor-to-ceiling range falls from 0.215 to 0.127. Belief updates do not track the corpus evidence: direction agreement is 0.454 to 0.549, even on findings the corpus moves by more than two log odds. Every model names the correct diagnosis 1.8 to 5.6 times more often than its own stated posterior ranks it first, locating the deficit in the probabilistic interface, not in clinical knowledge.
Cite
APA
Arun, G., George, A. B., & Bhasuran, B. (2026). Probabilistic Coherence of Stated Diagnostic Beliefs in Large Language Models: Anchored Tests on a Synthetic Clinical Corpus. Machine Learning and Knowledge Extraction, 8(11), 369. https://doi.org/10.3390/make8110369
Vancouver
Arun G, George AB, Bhasuran B. Probabilistic Coherence of Stated Diagnostic Beliefs in Large Language Models: Anchored Tests on a Synthetic Clinical Corpus. Machine Learning and Knowledge Extraction. 2026;8(11):369. doi:10.3390/make8110369
BibTeX
@article{Arun2026,
author = {Arun, Govind and George, Ajo Babu and Bhasuran, Balu},
title = {Probabilistic Coherence of Stated Diagnostic Beliefs in Large Language Models: Anchored Tests on a Synthetic Clinical Corpus},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {11},
pages = {369},
doi = {10.3390/make8110369}
}
EUVIMED