More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance
Frontiers in Psychology · 15.05.2026 · 17
DOI: 10.3389/fpsyg.2026.1815225 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Psychology, online erschienen am 15.05.2026. Verfasst von Levent Ertuna und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Educational Measurement and Evaluation, Faculty of Education, Sakarya University, Sakarya, Türkiye. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Careless responding, in which survey participants fail to attend to item content, is a well-recognized threat to the quality of self-report data. Although prevalence estimates commonly fall between 8 and 12% in student samples, the extent to which careless responding distorts the psychometric properties of attitude measures has received limited attention, particularly outside Western samples.
Methods
The present study investigated the prevalence and psychometric consequences of careless responding in a paper-and-pencil sample of 1,112 Turkish university students who completed the sustainable development awareness scale. An instructed response item embedded within the scale identified 126 respondents (11.33%) as careless, and two post-hoc indicators (longstring and even-odd consistency) converged with this classification. Parallel analyses on the unscreened and screened samples were conducted to evaluate effects on internal consistency, confirmatory factor analysis, multigroup measurement invariance, criterion-related validity correlations, and an item-level Composite Sensitivity Index (CSI).
Results
Internal consistency was higher in the screened sample, with the gains concentrated in the two subscales containing reverse-coded items. Confirmatory factor analysis indices trended in the direction of better fit after screening, and criterion-validity correlations with related constructs were modestly larger in the screened sample at both the manifest and latent levels. Multigroup measurement invariance testing across attentive and careless responders supported metric but not scalar invariance, pointing to systematic intercept differences consistent with acquiescent responding among careless respondents. The composite sensitivity index combining changes in item means, item-total correlations, and factor loadings was used to rank items by vulnerability to careless responding, with results robust to an alternative standardized aggregation. All six reverse-coded items appeared among the ten most sensitive items, despite constituting only one-sixth of the scale, and the item immediately following the attention check showed a notably elevated sensitivity pattern that we interpret as a tentative hypothesis worth further investigation.
Discussion
These findings underscore the importance of routine screening for careless responding and offer practical guidance on the placement of attention checks and the use of reverse-coded items.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Educational Measurement and Evaluation, Faculty of Education, Sakarya University, Sakarya, Türkiye.
Zitieren
APA
Ertuna, L., Kaya-Uyanik, G., & Gençaslan, D. (2026). More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance. Frontiers in Psychology, 17. https://doi.org/10.3389/fpsyg.2026.1815225
Vancouver
Ertuna L, Kaya-Uyanik G, Gençaslan D. More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance. Frontiers in Psychology. 2026;17. doi:10.3389/fpsyg.2026.1815225
BibTeX
@article{Ertuna2026,
author = {Ertuna, Levent and Kaya-Uyanik, Gülden and Gençaslan, Duygu},
title = {More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance},
journal = {Frontiers in Psychology},
year = {2026},
volume = {17},
doi = {10.3389/fpsyg.2026.1815225}
}
EUVIMED