EUVIMEDEUVIMED entdecken
Open Access

Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations

Shan Huang iD, David Goretzko

Educational and Psychological Measurement · 2026-12 · 86 (6) · 1193-1222

DOI: 10.1177/00131644261433236 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Educational and Psychological Measurement, online erschienen am 16.04.2026. Verfasst von Shan Huang und 1 weiteren Autor:innen. Beteiligte Einrichtung: Yew Chung Yew Wah Education Network, Hong Kong SAR, China. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: JST SPRING.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study presents the first application of e-value–based false discovery rate (FDR) control to Differential Item Functioning (DIF) detection, addressing long-standing limitations of p -value-based approaches when model assumptions are violated—for example, under multidimensionality, local item dependence, or extreme sample sizes. Two comprehensive simulation studies were conducted to evaluate e-BH (the e-value analogue of BH) procedures, using K-fold and Multisplit likelihood-ratio e-values, under (a) multidimensional contamination and (b) testlet-based local dependence. Across both scenarios, e-BH consistently provided stronger and more stable control of Type I error, FDR, and family-wise error rate (FWER) than classical procedures such as Benjamini–Hochberg (BH) and Holm. Even under severe model misspecification, e-BH maintained substantially lower false-positive rates while remaining relatively competitive in terms of Type II error. A key finding concerns sample size: classical p -value methods exhibited inflation of Type I error as N increased, whereas e-BH preserved stable error control due to its model-agnostic calibration. An empirical application using Progress in International Reading Literacy Study (PIRLS) data further demonstrated that e-BH produces a more defensible and operationally sustainable set of DIF flags than traditional approaches. Together, these results establish e-values as a powerful and robust evidential tool for DIF detection in modern assessment contexts.

Schlagwörter

Stichwörter der Autor:innen

differential item functioninge-valuesfalse discovery ratemodel misspecificationmultiple testing

Autor:innen und Institutionen

  • Yew Chung Yew Wah Education Network, Hong Kong SAR, China.
  • Goethe University Frankfurt, Germany.

ORCID-Profile: Shan Huang

Förderung

  • JST SPRING

Zitieren

APA

Huang, S., & Goretzko, D. (2026). Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations. Educational and Psychological Measurement, 86(6), 1193-1222. https://doi.org/10.1177/00131644261433236

Vancouver

Huang S, Goretzko D. Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations. Educational and Psychological Measurement. 2026;86(6):1193-1222. doi:10.1177/00131644261433236

BibTeX
@article{Huang2026,
  author = {Huang, Shan and Goretzko, David},
  title = {Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations},
  journal = {Educational and Psychological Measurement},
  year = {2026},
  volume = {86},
  number = {6},
  pages = {1193--1222},
  doi = {10.1177/00131644261433236}
}
Kostenlos starten →