Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework
Educational and Psychological Measurement · 07.07.2026
DOI: 10.1177/00131644261451746 ↗Kurz gefasst
Fachartikel in der Zeitschrift Educational and Psychological Measurement, online erschienen am 07.07.2026. Verfasst von Rabia Karatoprak Ersen und 2 weiteren Autor:innen. Beteiligte Einrichtung: GESIS-Leibniz Institute for the Social Sciences, Cologne, Germany. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The purpose of this study is to examine the reliability and dependability of difference scores computed as the change between a pretest and a posttest administered to assess the effectiveness of an intervention. The data-collection design involved a nested structure, with persons ( p ) nested within groups ( g ), and groups nested within sites ( s ). Multivariate generalizability theory was employed to estimate the reliability and dependability of difference scores at the levels of persons, groups, and sites. The G study designs included ( p • : g • : s • ) x i • , ( p • : s • ) x i • , ( p • : g • ) x i • , g • x i • , p • x i • , and s • x i • , with pretest and posttest serving as the two levels of the multivariate facet. In the designs with sites as the object of measurement, ( P • : G • : s • ) x I • , ( P • : s • ) x I • , and s • x I • , omitting groups within sites or persons within groups led to an underestimation of error variances and inflated generalizability and dependability coefficients. The relative error correlations increased, and the absolute error and universe score correlations decreased across s • x I • , ( P • : s • ) x I • , and ( P • : G • : s • ) x I • . Across all designs, generalizability and dependability coefficients were similar in magnitude, primarily due to the relatively small variance in items. Compared across different objects of measurement, both the generalizability and dependability coefficients were highest when the object of measurement was persons, and lowest when it was sites.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- GESIS-Leibniz Institute for the Social Sciences, Cologne, Germany.
- The University of Iowa, Iowa City, USA.
ORCID-Profile: Rabia Karatoprak Ersen, Won-Chan Lee
Zitieren
APA
Karatoprak Ersen, R., Lee, W. C., & Yarbrough, D. B. (2026). Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework. Educational and Psychological Measurement. https://doi.org/10.1177/00131644261451746
Vancouver
Karatoprak Ersen R, Lee WC, Yarbrough DB. Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework. Educational and Psychological Measurement. 2026. doi:10.1177/00131644261451746
BibTeX
@article{KaratoprakErsen2026,
author = {Karatoprak Ersen, Rabia and Lee, Won-Chan and Yarbrough, Donald B},
title = {Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework},
journal = {Educational and Psychological Measurement},
year = {2026},
doi = {10.1177/00131644261451746}
}
EUVIMED