EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework

Olga Kononykhina, Malte Schierholz, Frauke Kreuter

Journal of Official Statistics · 12.07.2026

DOI: 10.1177/0282423x261455656 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Official Statistics, online erschienen am 12.07.2026. Verfasst von Olga Kononykhina und 2 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Deutsche Forschungsgemeinschaft.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Survey research increasingly relies on automatic tools to classify open-ended occupational data. With the rise of machine learning (ML) and large language models (LLMs), this task is shifting from post-survey coding to live classification during interviews. Respondents actively evaluate algorithmic suggestions, creating a joint human–machine decision process where data quality cannot be assessed by accuracy metrics alone. Drawing on human–computer interaction and survey methodology, we propose the COARSE (Classification Outcomes and Respondent Engagement) evaluation framework, which distinguishes five outcomes: accurate classification, misclassification, omission error, commission error, and appropriate rejection. Evidence from a representative German survey deploying an interactive ML-based instrument (OccuCoDe) shows that respondents cannot reliably safeguard data quality when algorithms fail. Instead of rejecting poor suggestions, they often settle for “good enough” answers. When correct options are present, errors reflect misjudgments of subtle distinctions. When respondents reject machine suggestions, they report higher task difficulty afterward, especially when valid options were overlooked. These results show that data errors extend to human cognition and survey interaction, going beyond established machine learning metrics for algorithmic decision-making. As humans and machines collaborate in the survey-answering process, the COARSE framework offers a new lens to evaluate data quality and improve automated coding systems.

Förderung

  • Deutsche Forschungsgemeinschaft

Zitieren

APA

Kononykhina, O., Schierholz, M., & Kreuter, F. (2026). Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework. Journal of Official Statistics. https://doi.org/10.1177/0282423x261455656

Vancouver

Kononykhina O, Schierholz M, Kreuter F. Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework. Journal of Official Statistics. 2026. doi:10.1177/0282423x261455656

BibTeX
@article{Kononykhina2026,
  author = {Kononykhina, Olga and Schierholz, Malte and Kreuter, Frauke},
  title = {Beyond Algorithmic Accuracy: Understanding Data Quality in Interactive Occupational Coding with the COARSE Framework},
  journal = {Journal of Official Statistics},
  year = {2026},
  doi = {10.1177/0282423x261455656}
}
Kostenlos starten →