EUVIMEDEUVIMED entdecken
Validierungsstudie

AI-first, expert-verified: validating generative AI for HFACS-based coding of healthcare RCA transcripts with governance considerations

Jiun-Yih Lee iD, Chien-Hsien Huang, Chung-Ho Hsieh, Chin-Shu Lin, Pei-Ching Yang, Yan-Liang Lai, Chen-An Yeh, Chin-Jui Tien, Jui-Ting Chang

International Journal for Quality in Health Care · 13.04.2026 · 38 (2)

DOI: 10.1093/intqhc/mzag045 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift International Journal for Quality in Health Care, online erschienen am 09.04.2026. Verfasst von Jiun-Yih Lee und 8 weiteren Autor:innen. Beteiligte Einrichtung: Center for Quality Management, Shin Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan. Hauptthemen laut MEDLINE: Generative Artificial Intelligence, Root Cause Analysis, Artificial Intelligence, Clinical Coding. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 13 Literaturangaben. Gefördert von: Shin Kong Wu Ho-Su Memorial Hospital.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Root cause analysis (RCA) is widely used in healthcare incident investigation, but its outputs can be limited by inconsistent causal framing and variable integration of human factors. The Human Factors Analysis and Classification System (HFACS) offers a structured taxonomy for causal attribution, yet manual coding is resource-intensive. Empirical validation of generative artificial intelligence (AI) for document-level HFACS coding from complete healthcare RCA transcripts remains limited.

Methods

We conducted a cross-sectional validation study at an 829-bed medical center in Taiwan. Thirty-five de-identified RCA interview transcripts (2024–5) with verbatim transcription were analyzed using SKH-AI, an in-house platform integrating an Azure OpenAI–hosted GPT-4o model with deterministic decoding (temperature = 0; top_p = 1.0). The model processed each transcript holistically to identify salient narrative segments and assign HFACS codes with evidence-linked rationales without rule-based post-processing. Outputs were compared with dual-expert HFACS coding with adjudicated consensus. Performance was assessed using precision, recall, micro-/macro-F1, and Cohen’s κ with bootstrapped 95% confidence intervals.

Results

Across 562 AI-derived segments, micro-F1 was 0.66 [95% confidence interval (CI): 0.63–0.69] and macro-F1 was 0.68 (95% CI: 0.64–0.72), with moderate agreement versus expert coding (κ = 0.56, 95% CI: 0.52–0.60). Performance was higher for text-anchored categories (Level 2 Preconditions F1 = 0.70; Level 1 Unsafe Acts F1 = 0.69) than for more abstract domains (Level 3 F1 = 0.66; Level 4 F1 = 0.65). Subcategory analyses showed stronger detection of concrete cues (e.g. decision errors, physical environment, communication) and weaker performance for latent constructs (e.g. process management). Bias analyses indicated a recall-leaning tendency at Levels 3–4, consistent with increased over-attribution risk.

Conclusions

Generative AI can produce auditable, evidence-linked candidate HFACS attributions from document-level RCA transcripts with moderate concordance to expert coding. Higher-level supervisory and organizational attributions remain vulnerable to overgeneralization and should be governed as decision support, with evidence-anchoring and mandatory expert sign-off for Level 3–4 codes.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Generative Artificial IntelligenceRoot Cause AnalysisArtificial IntelligenceClinical CodingHumansCross-Sectional StudiesTaiwan

Stichwörter der Autor:innen

HFACSgenerative artificial intelligencehuman factorsincident investigationpatient safetyroot cause analysis

Autor:innen und Institutionen

  • Center for Quality Management, Shin Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan.
  • Division of Infectious Disease, Department of Internal Medicine, Shin Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan.
  • College of Medicine, Fu Jen Catholic University, New Taipei, Taiwan.
  • Department of General Surgery, Shin Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan.
  • College of Science and Engineering, Fu Jen Catholic University, New Taipei City, Taiwan.
  • Department of Information Technology, Shin Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan.

ORCID-Profile: Jiun-Yih Lee

Förderung

  • Shin Kong Wu Ho-Su Memorial Hospital

Zitieren

APA

Lee, J. Y., Huang, C. H., Hsieh, C. H., Lin, C. S., Yang, P. C., Lai, Y. L., Yeh, C. A., Tien, C. J., & Chang, J. T. (2026). AI-first, expert-verified: validating generative AI for HFACS-based coding of healthcare RCA transcripts with governance considerations. International Journal for Quality in Health Care, 38(2). https://doi.org/10.1093/intqhc/mzag045

Vancouver

Lee JY, Huang CH, Hsieh CH, Lin CS, Yang PC, Lai YL, et al. AI-first, expert-verified: validating generative AI for HFACS-based coding of healthcare RCA transcripts with governance considerations. International Journal for Quality in Health Care. 2026;38(2). doi:10.1093/intqhc/mzag045

BibTeX
@article{Lee2026,
  author = {Lee, Jiun-Yih and Huang, Chien-Hsien and Hsieh, Chung-Ho and Lin, Chin-Shu and Yang, Pei-Ching and Lai, Yan-Liang and Yeh, Chen-An and Tien, Chin-Jui and Chang, Jui-Ting},
  title = {AI-first, expert-verified: validating generative AI for HFACS-based coding of healthcare RCA transcripts with governance considerations},
  journal = {International Journal for Quality in Health Care},
  year = {2026},
  volume = {38},
  number = {2},
  doi = {10.1093/intqhc/mzag045}
}
Kostenlos starten →