Boolean Analysis of Interobserver Agreement: Formal and Functional Evidence Sampling in Complex Coding Endeavors
Educational Measurement: Issues and Practice · 2021-06 · 40 (2) · 26-36
DOI: 10.1111/emip.12409 ↗Kurz gefasst
Fachartikel in der Zeitschrift Educational Measurement: Issues and Practice, online erschienen am 24.11.2020. Verfasst von Guillermo Solano‐Flores. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 37 Literaturangaben.
Abstract (Originalsprache)
This article proposes a Boolean approach to representing and analyzing interobserver agreement in dichotomous coding. Building on the notion that observations are samples of a universe of observations, it submits that coding can be viewed as a process in which observers sample pieces of evidence on constructs. It distinguishes between formal and functional interobserver agreement based on whether events coded by observers are, respectively, the same or different but equivalent samples of the same classes of events. Formal and functional agreement are defined as Boolean functions. The article reports on an investigation in which a Boolean approach was used to examine agreement in the coding of teacher–student social interactions. Agreement maps (devices that represent the relative frequencies of coding cases—different combinations of 1s and 0s in the observers’ coding) allowed detail examination of how interobserver agreement was shaped by the nature of the events coded and the characteristics of the individuals whose behaviors were coded. Results provide empirical evidence of formal and functional agreement as distinguishable forms of interobserver agreement. Results also show that Boolean coding analysis allows detection of ways in which coding procedures confound absence of evidence with evidence of absence. Practical and methodological implications are discussed.
Zitieren
APA
Solano‐Flores, G. (2021). Boolean Analysis of Interobserver Agreement: Formal and Functional Evidence Sampling in Complex Coding Endeavors. Educational Measurement: Issues and Practice, 40(2), 26-36. https://doi.org/10.1111/emip.12409
Vancouver
Solano‐Flores G. Boolean Analysis of Interobserver Agreement: Formal and Functional Evidence Sampling in Complex Coding Endeavors. Educational Measurement: Issues and Practice. 2021;40(2):26-36. doi:10.1111/emip.12409
BibTeX
@article{SolanoFlores2021,
author = {Solano‐Flores, Guillermo},
title = {Boolean Analysis of Interobserver Agreement: Formal and Functional Evidence Sampling in Complex Coding Endeavors},
journal = {Educational Measurement: Issues and Practice},
year = {2021},
volume = {40},
number = {2},
pages = {26--36},
doi = {10.1111/emip.12409}
}
EUVIMED