Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations
Educational and Psychological Measurement · 2026-12 · 86 (6) · 1193-1222
DOI: 10.1177/00131644261433236 ↗Kurz gefasst
Fachartikel in der Zeitschrift Educational and Psychological Measurement, online erschienen am 16.04.2026. Verfasst von Shan Huang und 1 weiteren Autor:innen. Beteiligte Einrichtung: Yew Chung Yew Wah Education Network, Hong Kong SAR, China. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: JST SPRING.
Abstract (Originalsprache)
This study presents the first application of e-value–based false discovery rate (FDR) control to Differential Item Functioning (DIF) detection, addressing long-standing limitations of p -value-based approaches when model assumptions are violated—for example, under multidimensionality, local item dependence, or extreme sample sizes. Two comprehensive simulation studies were conducted to evaluate e-BH (the e-value analogue of BH) procedures, using K-fold and Multisplit likelihood-ratio e-values, under (a) multidimensional contamination and (b) testlet-based local dependence. Across both scenarios, e-BH consistently provided stronger and more stable control of Type I error, FDR, and family-wise error rate (FWER) than classical procedures such as Benjamini–Hochberg (BH) and Holm. Even under severe model misspecification, e-BH maintained substantially lower false-positive rates while remaining relatively competitive in terms of Type II error. A key finding concerns sample size: classical p -value methods exhibited inflation of Type I error as N increased, whereas e-BH preserved stable error control due to its model-agnostic calibration. An empirical application using Progress in International Reading Literacy Study (PIRLS) data further demonstrated that e-BH produces a more defensible and operationally sustainable set of DIF flags than traditional approaches. Together, these results establish e-values as a powerful and robust evidential tool for DIF detection in modern assessment contexts.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Yew Chung Yew Wah Education Network, Hong Kong SAR, China.
- Goethe University Frankfurt, Germany.
ORCID-Profile: Shan Huang
Förderung
- JST SPRING
Zitieren
APA
Huang, S., & Goretzko, D. (2026). Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations. Educational and Psychological Measurement, 86(6), 1193-1222. https://doi.org/10.1177/00131644261433236
Vancouver
Huang S, Goretzko D. Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations. Educational and Psychological Measurement. 2026;86(6):1193-1222. doi:10.1177/00131644261433236
BibTeX
@article{Huang2026,
author = {Huang, Shan and Goretzko, David},
title = {Controlling the False Discovery Rate in DIF Detection With e-Values: Evidence From Multidimensional and Testlet Simulations},
journal = {Educational and Psychological Measurement},
year = {2026},
volume = {86},
number = {6},
pages = {1193--1222},
doi = {10.1177/00131644261433236}
}
EUVIMED