Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis
PLOS Digital Health · 25.02.2026 · 5 (2) · e0000576
DOI: 10.1371/journal.pdig.0000576 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 25.02.2026. Verfasst von Sam Martin und 10 weiteren Autor:innen. Beteiligte Einrichtung: Oxford Vaccine Group, Department of Paediatrics, Oxford University, Oxford, United Kingdom. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Institute for Health Research, National Institute of Health Research.
Abstract (Originalsprache)
Background The exponential growth of Big Qualitative (Big Qual) data in healthcare research presents methodological challenges for traditional analysis approaches. This study evaluates the effectiveness of machine-assisted analysis using artificial intelligence (AI) tools compared to human-only analysis for processing large-scale qualitative datasets, using the Royal College of Anaesthetists’ 7th National Audit Project (NAP7) baseline survey as a test case. Methodology/Principal Findings We conducted a comparative methodological study analysing 5,196 free-text responses about peri-operative cardiac arrest experiences. Three researchers established a human-coded reference standard following SRQR guidelines. We then applied machine-assisted analysis using Pulsar for exploratory analysis and Caplena for sentiment and thematic analysis, evaluating performance against the human gold standard using STARD-AI reporting standards. Performance metrics included accuracy, precision, recall, F1-scores, and Cohen’s Kappa, with confidence intervals calculated using bootstrap resampling. Machine-assisted analysis substantially reduced analysis time, with particularly dramatic improvements in theme identification speed. The machine-assisted approach achieved good thematic and sentiment classification accuracy compared to the human reference standard, though human analysis identified an emergent ‘ambiguous’ sentiment category that current AI tools cannot accommodate, highlighting limitations in commercial platforms’ flexibility for inductive analysis. Conclusions/Significance Machine-assisted analysis offers substantial efficiency gains with acceptable accuracy trade-offs for large-scale qualitative data analysis. However, human expertise remains essential for capturing nuanced meanings, identifying emergent categories, and providing domain-specific interpretation. This hybrid approach represents a viable methodology for Big Qual research, though current AI tools’ constraints in accommodating emergent classification schemes remain a limitation. Our findings establish benchmarks for future development of more flexible AI systems adapted to qualitative research paradigms.
Autor:innen und Institutionen
- Oxford Vaccine Group, Department of Paediatrics, Oxford University, Oxford, United Kingdom.
- Department of Targeted Intervention, Rapid Research Evaluation and Appraisal Lab (RREAL), University College London (UCL), London, United Kingdom.
- Department of Anaesthesia, Royal United Hospitals Bath NHS Foundation Trust, Bath, United Kingdom.
- Health Services Research Centre, Royal College of Anaesthetists, London, United Kingdom.
- Department of Anaesthesia, Severn Deanery, Bristol, United Kingdom.
- Department of Anaesthesia, Royal United Hospital, Bath, United Kingdom.
ORCID-Profile: Emma Beecham, Richard A Armstrong, Noémie Déom, Andrew D Kane
Förderung
- National Institute for Health Research
- National Institute of Health Research
Zitieren
APA
Martin, S., Beecham, E., Kursumovic, E., Armstrong, R. A., Cook, T. M., Déom, N., Kane, A. D., Moniz, S., Soar, J., Vindrola-Padros, C., & Collaborators (2026). Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis. PLOS Digital Health, 5(2), e0000576. https://doi.org/10.1371/journal.pdig.0000576
Vancouver
Martin S, Beecham E, Kursumovic E, Armstrong RA, Cook TM, Déom N, et al. Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis. PLOS Digital Health. 2026;5(2):e0000576. doi:10.1371/journal.pdig.0000576
BibTeX
@article{Martin2026,
author = {Martin, Sam and Beecham, Emma and Kursumovic, Emira and Armstrong, Richard A and Cook, Tim M and Déom, Noémie and Kane, Andrew D and Moniz, Sophie and Soar, Jasmeet and Vindrola-Padros, Cecilia and Collaborators, Collaborators},
title = {Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis},
journal = {PLOS Digital Health},
year = {2026},
volume = {5},
number = {2},
pages = {e0000576},
doi = {10.1371/journal.pdig.0000576}
}
EUVIMED