EUVIMEDEUVIMED entdecken
Crossref

Using Item Parameter Predictions for Reducing Calibration Sample Requirements—A Case Study Based on a High‐Stakes Admission Test

Esther Ulitzsch, Dmitry Belov, Oliver Lüdtke, Alexander Robitzsch

Journal of Educational Measurement · 2026-03 · 63 (1)

DOI: 10.1111/jedm.12426 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Educational Measurement, online erschienen am 16.02.2025. Verfasst von Esther Ulitzsch und 3 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 57 Literaturangaben. Gefördert von: Norges Forskningsråd.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In item difficulty modeling (IDM), item parameters are predicted from the items' linguistic features, aiming to ultimately render item calibration redundant. Current IDM applications, however, commonly do not yield the required prediction accuracy. To immediately exploit even somewhat inaccurate IDM predictions, we blend IDM with established Bayesian estimation techniques. We propose a two‐step approach where (1) IDM predictions are obtained and (2) employed to construct informative prior distributions. We evaluate the approach in a case study on small‐sample calibration of the 3PL in a high‐stakes test. First, concerning implementation, we find computationally efficient penalized maximum likelihood estimation to be comparable to the best‐performing MCMC‐based approach. Second, we investigate sample size reductions achievable with state‐of‐the‐art IDM predictions, finding negligible gains compared to merely considering the historical distribution of parameters. Third, we evaluate the prediction accuracy required for a targeted sample size reduction by gradually increasing simulated IDM prediction accuracies. We find that required accuracies can counterbalance each other, allowing calibration sample size to be reduced when either high‐quality item difficulty predictions or good predictions of item discriminations and pseudo‐guessing parameters are available. We argue that these evaluations provide new, portable IDM benchmarks quantifying performance in terms of achievable sample size reductions.

Förderung

  • Norges Forskningsråd

Zitieren

APA

Ulitzsch, E., Belov, D., Lüdtke, O., & Robitzsch, A. (2026). Using Item Parameter Predictions for Reducing Calibration Sample Requirements—A Case Study Based on a High‐Stakes Admission Test. Journal of Educational Measurement, 63(1). https://doi.org/10.1111/jedm.12426

Vancouver

Ulitzsch E, Belov D, Lüdtke O, Robitzsch A. Using Item Parameter Predictions for Reducing Calibration Sample Requirements—A Case Study Based on a High‐Stakes Admission Test. Journal of Educational Measurement. 2026;63(1). doi:10.1111/jedm.12426

BibTeX
@article{Ulitzsch2026,
  author = {Ulitzsch, Esther and Belov, Dmitry and Lüdtke, Oliver and Robitzsch, Alexander},
  title = {Using Item Parameter Predictions for Reducing Calibration Sample Requirements—A Case Study Based on a High‐Stakes Admission Test},
  journal = {Journal of Educational Measurement},
  year = {2026},
  volume = {63},
  number = {1},
  doi = {10.1111/jedm.12426}
}
Kostenlos starten →