EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY-NC 4.0

The Great Scribe-Off: A Comparative Analysis of AI Scribes Versus Human Documentation in Simulated General Practice Consultations

Darran Foo, Janice Tan, Amandeep Hansra, Sean Stevens, Helen Wilcox

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti251572 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Darran Foo und 4 weiteren Autor:innen. Beteiligte Einrichtung: Digital Health & Innovation Specific Interest Group, Royal Australian College of General Practitioners. Hauptthemen laut MEDLINE: Documentation, General Practice, Artificial Intelligence, Electronic Health Records. Die Arbeit wurde bisher 2-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Clinical documentation burden remains a significant challenge in healthcare, particularly in primary care settings. Artificial intelligence (AI) scribes have emerged as potential solutions, but their effectiveness compared to human documentation lacks robust evidence, especially in community general practice environments. Documentation quality is compared between four commercial AI scribes and human-generated notes using four standardised clinical scenarios from the Royal Australian College of General Practitioners examination repository in simulated general practice consultations. Three experienced general practitioners, blinded to the source, assessed quality using a modified Physician Documentation Quality Instrument (PDQI-9). AI-generated notes outperformed human documentation across multiple quality domains. Top AI scribes scored a mean of 44.08/50 (SD = 3.32) vs. 37.42 (SD = 9.78) for humans, excelling in thoroughness (M = 4.92), accuracy (M = 4.67), and freedom from bias (M = 4.92). Inter-rater reliability was high for thoroughness (ICC = 0.879) and accuracy (ICC = 0.745), but lower for subjective areas like synthesis (ICC = 0.082). This study shows that AI scribes can outperform traditional documentation in simulated general practice. Successful implementation, however, depends on workflow integration and customisation. Standardised evaluation and balancing consistency with clinical context are key. Future research should explore real-world use, focusing on customisation and workflow impact.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

DocumentationGeneral PracticeArtificial IntelligenceElectronic Health RecordsHumansAustralia

Stichwörter der Autor:innen

AI scribesartificial intelligenceclinical documentationdigital healthgeneral practiceprimary care

Autor:innen und Institutionen

  • Digital Health & Innovation Specific Interest Group, Royal Australian College of General Practitioners.
  • Australian Institute of Health Innovation, Macquarie University.
  • Healthdirect Australia.
  • Telstra Health.
  • Australian Digital Health Agency.
  • Grove Medical.

Zitieren

APA

Foo, D., Tan, J., Hansra, A., Stevens, S., & Wilcox, H. (2025). The Great Scribe-Off: A Comparative Analysis of AI Scribes Versus Human Documentation in Simulated General Practice Consultations. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti251572

Vancouver

Foo D, Tan J, Hansra A, Stevens S, Wilcox H. The Great Scribe-Off: A Comparative Analysis of AI Scribes Versus Human Documentation in Simulated General Practice Consultations. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti251572

BibTeX
@article{Foo2025,
  author = {Foo, Darran and Tan, Janice and Hansra, Amandeep and Stevens, Sean and Wilcox, Helen},
  title = {The Great Scribe-Off: A Comparative Analysis of AI Scribes Versus Human Documentation in Simulated General Practice Consultations},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti251572}
}
Kostenlos starten →