EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance

Levent Ertuna, Gülden Kaya-Uyanik, Duygu Gençaslan

Frontiers in Psychology · 15.05.2026 · 17

DOI: 10.3389/fpsyg.2026.1815225 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Psychology, online erschienen am 15.05.2026. Verfasst von Levent Ertuna und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Educational Measurement and Evaluation, Faculty of Education, Sakarya University, Sakarya, Türkiye. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Careless responding, in which survey participants fail to attend to item content, is a well-recognized threat to the quality of self-report data. Although prevalence estimates commonly fall between 8 and 12% in student samples, the extent to which careless responding distorts the psychometric properties of attitude measures has received limited attention, particularly outside Western samples.

Methods

The present study investigated the prevalence and psychometric consequences of careless responding in a paper-and-pencil sample of 1,112 Turkish university students who completed the sustainable development awareness scale. An instructed response item embedded within the scale identified 126 respondents (11.33%) as careless, and two post-hoc indicators (longstring and even-odd consistency) converged with this classification. Parallel analyses on the unscreened and screened samples were conducted to evaluate effects on internal consistency, confirmatory factor analysis, multigroup measurement invariance, criterion-related validity correlations, and an item-level Composite Sensitivity Index (CSI).

Results

Internal consistency was higher in the screened sample, with the gains concentrated in the two subscales containing reverse-coded items. Confirmatory factor analysis indices trended in the direction of better fit after screening, and criterion-validity correlations with related constructs were modestly larger in the screened sample at both the manifest and latent levels. Multigroup measurement invariance testing across attentive and careless responders supported metric but not scalar invariance, pointing to systematic intercept differences consistent with acquiescent responding among careless respondents. The composite sensitivity index combining changes in item means, item-total correlations, and factor loadings was used to rank items by vulnerability to careless responding, with results robust to an alternative standardized aggregation. All six reverse-coded items appeared among the ten most sensitive items, despite constituting only one-sixth of the scale, and the item immediately following the attention check showed a notably elevated sensitivity pattern that we interpret as a tentative hypothesis worth further investigation.

Discussion

These findings underscore the importance of routine screening for careless responding and offer practical guidance on the placement of attention checks and the use of reverse-coded items.

Schlagwörter

Stichwörter der Autor:innen

careless respondingcomposite sensitivity indexdata qualityinstructed response itemsinsufficient effort respondingmeasurement invariancepsychometric propertiesreverse-coded items

Autor:innen und Institutionen

  • Department of Educational Measurement and Evaluation, Faculty of Education, Sakarya University, Sakarya, Türkiye.

Zitieren

APA

Ertuna, L., Kaya-Uyanik, G., & Gençaslan, D. (2026). More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance. Frontiers in Psychology, 17. https://doi.org/10.3389/fpsyg.2026.1815225

Vancouver

Ertuna L, Kaya-Uyanik G, Gençaslan D. More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance. Frontiers in Psychology. 2026;17. doi:10.3389/fpsyg.2026.1815225

BibTeX
@article{Ertuna2026,
  author = {Ertuna, Levent and Kaya-Uyanik, Gülden and Gençaslan, Duygu},
  title = {More than just noise: careless responding and its systematic effects on reliability, validity, and measurement invariance},
  journal = {Frontiers in Psychology},
  year = {2026},
  volume = {17},
  doi = {10.3389/fpsyg.2026.1815225}
}
Kostenlos starten →