Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework
Journal of Official Statistics · 12.07.2026
DOI: 10.1177/0282423x261455656 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Official Statistics, online erschienen am 12.07.2026. Verfasst von Olga Kononykhina und 2 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Deutsche Forschungsgemeinschaft.
Abstract (Originalsprache)
Survey research increasingly relies on automatic tools to classify open-ended occupational data. With the rise of machine learning (ML) and large language models (LLMs), this task is shifting from post-survey coding to live classification during interviews. Respondents actively evaluate algorithmic suggestions, creating a joint human–machine decision process where data quality cannot be assessed by accuracy metrics alone. Drawing on human–computer interaction and survey methodology, we propose the COARSE (Classification Outcomes and Respondent Engagement) evaluation framework, which distinguishes five outcomes: accurate classification, misclassification, omission error, commission error, and appropriate rejection. Evidence from a representative German survey deploying an interactive ML-based instrument (OccuCoDe) shows that respondents cannot reliably safeguard data quality when algorithms fail. Instead of rejecting poor suggestions, they often settle for “good enough” answers. When correct options are present, errors reflect misjudgments of subtle distinctions. When respondents reject machine suggestions, they report higher task difficulty afterward, especially when valid options were overlooked. These results show that data errors extend to human cognition and survey interaction, going beyond established machine learning metrics for algorithmic decision-making. As humans and machines collaborate in the survey-answering process, the COARSE framework offers a new lens to evaluate data quality and improve automated coding systems.
Förderung
- Deutsche Forschungsgemeinschaft
Zitieren
APA
Kononykhina, O., Schierholz, M., & Kreuter, F. (2026). Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework. Journal of Official Statistics. https://doi.org/10.1177/0282423x261455656
Vancouver
Kononykhina O, Schierholz M, Kreuter F. Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework. Journal of Official Statistics. 2026. doi:10.1177/0282423x261455656
BibTeX
@article{Kononykhina2026,
author = {Kononykhina, Olga and Schierholz, Malte and Kreuter, Frauke},
title = {Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework},
journal = {Journal of Official Statistics},
year = {2026},
doi = {10.1177/0282423x261455656}
}
EUVIMED