EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis

Sam Martin, Emma Beecham iD, Emira Kursumovic, Richard A Armstrong iD, Tim M Cook, Noémie Déom iD, Andrew D Kane iD, Sophie Moniz, Jasmeet Soar, Cecilia Vindrola-Padros, Collaborators

PLOS Digital Health · 25.02.2026 · 5 (2) · e0000576

DOI: 10.1371/journal.pdig.0000576 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 25.02.2026. Verfasst von Sam Martin und 10 weiteren Autor:innen. Beteiligte Einrichtung: Oxford Vaccine Group, Department of Paediatrics, Oxford University, Oxford, United Kingdom. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Institute for Health Research, National Institute of Health Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background The exponential growth of Big Qualitative (Big Qual) data in healthcare research presents methodological challenges for traditional analysis approaches. This study evaluates the effectiveness of machine-assisted analysis using artificial intelligence (AI) tools compared to human-only analysis for processing large-scale qualitative datasets, using the Royal College of Anaesthetists’ 7th National Audit Project (NAP7) baseline survey as a test case. Methodology/Principal Findings We conducted a comparative methodological study analysing 5,196 free-text responses about peri-operative cardiac arrest experiences. Three researchers established a human-coded reference standard following SRQR guidelines. We then applied machine-assisted analysis using Pulsar for exploratory analysis and Caplena for sentiment and thematic analysis, evaluating performance against the human gold standard using STARD-AI reporting standards. Performance metrics included accuracy, precision, recall, F1-scores, and Cohen’s Kappa, with confidence intervals calculated using bootstrap resampling. Machine-assisted analysis substantially reduced analysis time, with particularly dramatic improvements in theme identification speed. The machine-assisted approach achieved good thematic and sentiment classification accuracy compared to the human reference standard, though human analysis identified an emergent ‘ambiguous’ sentiment category that current AI tools cannot accommodate, highlighting limitations in commercial platforms’ flexibility for inductive analysis. Conclusions/Significance Machine-assisted analysis offers substantial efficiency gains with acceptable accuracy trade-offs for large-scale qualitative data analysis. However, human expertise remains essential for capturing nuanced meanings, identifying emergent categories, and providing domain-specific interpretation. This hybrid approach represents a viable methodology for Big Qual research, though current AI tools’ constraints in accommodating emergent classification schemes remain a limitation. Our findings establish benchmarks for future development of more flexible AI systems adapted to qualitative research paradigms.

Autor:innen und Institutionen

  • Oxford Vaccine Group, Department of Paediatrics, Oxford University, Oxford, United Kingdom.
  • Department of Targeted Intervention, Rapid Research Evaluation and Appraisal Lab (RREAL), University College London (UCL), London, United Kingdom.
  • Department of Anaesthesia, Royal United Hospitals Bath NHS Foundation Trust, Bath, United Kingdom.
  • Health Services Research Centre, Royal College of Anaesthetists, London, United Kingdom.
  • Department of Anaesthesia, Severn Deanery, Bristol, United Kingdom.
  • Department of Anaesthesia, Royal United Hospital, Bath, United Kingdom.

ORCID-Profile: Emma Beecham, Richard A Armstrong, Noémie Déom, Andrew D Kane

Förderung

  • National Institute for Health Research
  • National Institute of Health Research

Zitieren

APA

Martin, S., Beecham, E., Kursumovic, E., Armstrong, R. A., Cook, T. M., Déom, N., Kane, A. D., Moniz, S., Soar, J., Vindrola-Padros, C., & Collaborators (2026). Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis. PLOS Digital Health, 5(2), e0000576. https://doi.org/10.1371/journal.pdig.0000576

Vancouver

Martin S, Beecham E, Kursumovic E, Armstrong RA, Cook TM, Déom N, et al. Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis. PLOS Digital Health. 2026;5(2):e0000576. doi:10.1371/journal.pdig.0000576

BibTeX
@article{Martin2026,
  author = {Martin, Sam and Beecham, Emma and Kursumovic, Emira and Armstrong, Richard A and Cook, Tim M and Déom, Noémie and Kane, Andrew D and Moniz, Sophie and Soar, Jasmeet and Vindrola-Padros, Cecilia and Collaborators, Collaborators},
  title = {Comparing human vs. machine-assisted analysis to develop a new approach for Big Qualitative Data Analysis},
  journal = {PLOS Digital Health},
  year = {2026},
  volume = {5},
  number = {2},
  pages = {e0000576},
  doi = {10.1371/journal.pdig.0000576}
}
Kostenlos starten →