EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Fixed-Sequence Confidence-Interval Calibration for Finite-Sample Selective Risk Control

Sijin Dong, Xinyu Hao, Hiroyuki Shinnou

Stats · 02.10.2026 · 9 (5) · 109

DOI: 10.3390/stats9050109 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Stats, online erschienen am 02.10.2026. Verfasst von Sijin Dong und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Selective prediction requires a statistically reliable rule for deciding which model outputs can be returned while controlling the error rate among accepted predictions. In large language model (LLM) applications, uncertainty scores provide useful ranking information, but directly thresholding such scores does not yield finite-sample risk guarantees. We propose confidence-interval calibration (CIC), a general calibration framework that converts an arbitrary uncertainty score into a risk-controlled selective decision rule. Given a held-out calibration sample, CIC estimates the acceptance-conditioned error rate for a pre-specified sequence of candidate thresholds and constructs one-sided upper confidence bounds using either Hoeffding’s inequality or the exact Clopper–Pearson procedure. Candidate thresholds are examined sequentially, and CIC returns the most permissive threshold within the consecutively certified prefix. Under independent and identically distributed calibration and deployment samples, we establish a finite-sample, high-probability guarantee that any returned threshold controls the acceptance-conditioned error rate at the prescribed level. In our experiments, calibration samples on the order of 103 examples already provide useful certification in favorable regimes, although the required sample size depends on the target risk level, model error rate, and quality of the uncertainty ranking. Experiments on CommonsenseQA and TriviaQA across seven LLMs demonstrate reliable empirical risk control with favorable acceptance rates across a range of target risk levels.

Zitieren

APA

Dong, S., Hao, X., & Shinnou, H. (2026). Fixed-Sequence Confidence-Interval Calibration for Finite-Sample Selective Risk Control. Stats, 9(5), 109. https://doi.org/10.3390/stats9050109

Vancouver

Dong S, Hao X, Shinnou H. Fixed-Sequence Confidence-Interval Calibration for Finite-Sample Selective Risk Control. Stats. 2026;9(5):109. doi:10.3390/stats9050109

BibTeX
@article{Dong2026,
  author = {Dong, Sijin and Hao, Xinyu and Shinnou, Hiroyuki},
  title = {Fixed-Sequence Confidence-Interval Calibration for Finite-Sample Selective Risk Control},
  journal = {Stats},
  year = {2026},
  volume = {9},
  number = {5},
  pages = {109},
  doi = {10.3390/stats9050109}
}
Kostenlos starten →