EUVIMEDEUVIMED entdecken
Crossref

Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective

Osvaldo Simeone, Yaniv Romano

IEEE Journal on Selected Areas in Information Theory · 2026 · 7 · 290-301

DOI: 10.1109/jsait.2026.3705302 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Journal on Selected Areas in Information Theory, 2026. Verfasst von Osvaldo Simeone und 1 weiteren Autor:innen. Die Arbeit nennt 83 Literaturangaben. Gefördert von: European Research Council (ERC) through European Union’s Horizon Europe Program, Open Fellowship of EPSRC, Engineering and Physical Sciences Research Council (EPSRC) Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In context-specific applications such as robotics, telecommunications, and healthcare, artificial intelligence systems often face the challenge of limited training data. This scarcity introduces epistemic uncertainty, i.e., reducible uncertainty stemming from incomplete knowledge of the underlying data distribution, which fundamentally limits predictive performance. This review paper examines formal methodologies that address data-limited regimes through two complementary approaches: quantifying epistemic uncertainty and mitigating data scarcity via synthetic data augmentation. We begin by reviewing generalized Bayesian learning frameworks that characterize epistemic uncertainty through generalized posteriors in the model parameter space, as well as “post-Bayes” learning frameworks. We continue by presenting information-theoretic generalization bounds that formalize the relationship between training data quantity and predictive uncertainty, providing a theoretical justification for generalized Bayesian learning. Moving beyond methods with asymptotic statistical validity, we survey uncertainty quantification methods that provide finite-sample statistical guarantees, including conformal prediction and conformal risk control. Finally, we examine recent advances in data efficiency by combining limited labeled data with abundant model predictions or synthetic data. Throughout, we take an information-theoretic perspective, highlighting the role of information measures in quantifying the impact of data scarcity.

Förderung

  • European Research Council (ERC) through European Union’s Horizon Europe Program
  • Open Fellowship of EPSRC
  • Engineering and Physical Sciences Research Council (EPSRC) Project
  • European Union

Zitieren

APA

Simeone, O., & Romano, Y. (2026). Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective. IEEE Journal on Selected Areas in Information Theory, 7, 290-301. https://doi.org/10.1109/jsait.2026.3705302

Vancouver

Simeone O, Romano Y. Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective. IEEE Journal on Selected Areas in Information Theory. 2026;7:290-301. doi:10.1109/jsait.2026.3705302

BibTeX
@article{Simeone2026,
  author = {Simeone, Osvaldo and Romano, Yaniv},
  title = {Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective},
  journal = {IEEE Journal on Selected Areas in Information Theory},
  year = {2026},
  volume = {7},
  pages = {290--301},
  doi = {10.1109/jsait.2026.3705302}
}
Kostenlos starten →