When a validation corpus cannot decide: Design effect, calibration invariance and anchor leakage in defence simulation model comparison
The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology · 11.11.2026
DOI: 10.1177/15485129261501944 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, online erschienen am 11.11.2026. Verfasst von Adrián Klapp Fritz und 2 weiteren Autor:innen. Die Arbeit nennt 27 Literaturangaben.
Abstract (Originalsprache)
Validation studies comparing two computational models over a fleet of physical assets usually treat every record as one observation, and rarely state what prior calibration leaves comparable. We ask what such a corpus can decide. In an interior-ballistics case study (266 muzzle velocities from 19 chambering groups, two lumped-parameter engines), we characterise the calibration operator algebraically, remove anchors at the group level and measure the cluster design effect with the exact record-weighted multiplier. A symmetric family-level correction is exactly invariant to a positive rescaling constant within a family: degrading one model by 11 percentage points leaves its corrected output unchanged to round-off. Anchor leakage, bounded by enumerating 159,936 admissible configurations, was measured once they were reconstructed: it had reversed the historical estimate’s sign. The intraclass correlation is 0.54, and the effective sample size, 21.9. All 12 nominal intervals contain parity, as does the mean absolute percentage error comparison after simulation-calibrated widening; no procedure examined reaches nominal coverage. Under the observed design model, detecting a 1-percentage-point difference needs about 380 chambering groups; 19 cap effective sample size at about 35 regardless of records per group. Validation campaigns should be budgeted in groups, not records.
Zitieren
APA
Fritz, A. K., Rivera, J. L. V., & Ketterer, C. G. (2026). When a validation corpus cannot decide: Design effect, calibration invariance and anchor leakage in defence simulation model comparison. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology. https://doi.org/10.1177/15485129261501944
Vancouver
Fritz AK, Rivera JLV, Ketterer CG. When a validation corpus cannot decide: Design effect, calibration invariance and anchor leakage in defence simulation model comparison. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology. 2026. doi:10.1177/15485129261501944
BibTeX
@article{Fritz2026,
author = {Fritz, Adrián Klapp and Rivera, José Luis Valín and Ketterer, Cristóbal Galleguillos},
title = {When a validation corpus cannot decide: Design effect, calibration invariance and anchor leakage in defence simulation model comparison},
journal = {The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology},
year = {2026},
doi = {10.1177/15485129261501944}
}
EUVIMED