EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework

Rabia Karatoprak Ersen iD, Won-Chan Lee iD, Donald B Yarbrough

Educational and Psychological Measurement · 07.07.2026

DOI: 10.1177/00131644261451746 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Educational and Psychological Measurement, online erschienen am 07.07.2026. Verfasst von Rabia Karatoprak Ersen und 2 weiteren Autor:innen. Beteiligte Einrichtung: GESIS-Leibniz Institute for the Social Sciences, Cologne, Germany. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The purpose of this study is to examine the reliability and dependability of difference scores computed as the change between a pretest and a posttest administered to assess the effectiveness of an intervention. The data-collection design involved a nested structure, with persons ( p ) nested within groups ( g ), and groups nested within sites ( s ). Multivariate generalizability theory was employed to estimate the reliability and dependability of difference scores at the levels of persons, groups, and sites. The G study designs included ( p • : g • : s • ) x i • , ( p • : s • ) x i • , ( p • : g • ) x i • , g • x i • , p • x i • , and s • x i • , with pretest and posttest serving as the two levels of the multivariate facet. In the designs with sites as the object of measurement, ( P • : G • : s • ) x I • , ( P • : s • ) x I • , and s • x I • , omitting groups within sites or persons within groups led to an underestimation of error variances and inflated generalizability and dependability coefficients. The relative error correlations increased, and the absolute error and universe score correlations decreased across s • x I • , ( P • : s • ) x I • , and ( P • : G • : s • ) x I • . Across all designs, generalizability and dependability coefficients were similar in magnitude, primarily due to the relatively small variance in items. Compared across different objects of measurement, both the generalizability and dependability coefficients were highest when the object of measurement was persons, and lowest when it was sites.

Schlagwörter

Stichwörter der Autor:innen

dependabilitydifference scoreseducational standardsmultilevel datamultivariate generalizability theoryreliability

Autor:innen und Institutionen

  • GESIS-Leibniz Institute for the Social Sciences, Cologne, Germany.
  • The University of Iowa, Iowa City, USA.

ORCID-Profile: Rabia Karatoprak Ersen, Won-Chan Lee

Zitieren

APA

Karatoprak Ersen, R., Lee, W. C., & Yarbrough, D. B. (2026). Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework. Educational and Psychological Measurement. https://doi.org/10.1177/00131644261451746

Vancouver

Karatoprak Ersen R, Lee WC, Yarbrough DB. Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework. Educational and Psychological Measurement. 2026. doi:10.1177/00131644261451746

BibTeX
@article{KaratoprakErsen2026,
  author = {Karatoprak Ersen, Rabia and Lee, Won-Chan and Yarbrough, Donald B},
  title = {Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework},
  journal = {Educational and Psychological Measurement},
  year = {2026},
  doi = {10.1177/00131644261451746}
}
Kostenlos starten →