Human medical documentation significantly outperforms ChatGPT‐4o in critical clinical dimensions: A blinded comparative assessment in paediatric orthopaedics
Knee Surgery, Sports Traumatology, Arthroscopy · 2026-05 · 34 (5) · 1910-1916
DOI: 10.1002/ksa.70357 ↗Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift Knee Surgery, Sports Traumatology, Arthroscopy, online erschienen am 28.02.2026. Verfasst von Carlo Camathias und 5 weiteren Autor:innen. Beteiligte Einrichtung: Praxis Momentum, St. Gallen, Switzerland. Hauptthemen laut MEDLINE: Documentation, Orthopedics, Medical Records. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Purpose
This study evaluated the quality of ChatGPT‐generated medical history summaries compared to human‐created documentation in a paediatric orthopaedic practice setting.
Methods
A prospective, randomised, blinded comparative study was conducted involving 20 consecutive paediatric patients (mean age 14.2 ± 2.3 years; 11 males, 9 females) presenting with knee problems. Audio recordings of medical consultations were transcribed and processed by ChatGPT‐4o (OpenAI) using standardised prompts. Three independent orthopaedic specialists evaluated both human‐generated and AI‐generated summaries using eight quality criteria: temporal consistency, spatial consistency, accident description, symptom accuracy, symptom specificity, previous interventions, writing style and overall impression. Each criterion was scored on a 6‐point Likert scale.
Results
Human‐created summaries received significantly higher overall ratings (5.2 ± 0.8) compared to ChatGPT‐generated summaries (4.5 ± 0.8, p < 0.001, Cohen's d = 0.80). After Bonferroni correction for multiple comparisons, statistically significant differences favouring human documentation were confirmed in four of eight criteria: temporal consistency ( p < 0.001), spatial consistency ( p < 0.001), accident description ( p < 0.001) and overall impression ( p < 0.001). No significant differences were observed for writing style and documentation of previous interventions. Inter‐rater reliability was moderate (ICC = 0.64). ChatGPT demonstrated frequent temporal inconsistencies (14 of 60 evaluations, 23%) and omission of relevant accident details (21 of 60 evaluations, 35%).
Conclusion
While AI‐generated summaries showed acceptable stylistic quality, human documentation significantly outperformed ChatGPT in critical clinical dimensions, including temporal consistency and accuracy of complex orthopaedic presentations. Current large language models are not ready to replace human medical documentation in paediatric orthopaedic practice without careful oversight. The findings support the implementation of hybrid workflows where AI assists but does not replace human clinical judgement. Level of Evidence Level I.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Praxis Momentum, St. Gallen, Switzerland.
- Medical Faculty, University of Basel, Basel, Switzerland.
- Orthopaedic Departement, Schulthess Clinic, Zürich, Switzerland.
- Urosteph, St. Gallen, Switzerland.
- Kinder- und Jugendorthopädie, Aarau, Switzerland.
- Swiss Ortho Center, Basel, Switzerland.
ORCID-Profile: Carlo Camathias, Kata Papp, Patrick Betschart, Victor Valderrabano, Elias Ammann
Zitieren
APA
Camathias, C., Papp, K., Betschart, P., Speth, B., Valderrabano, V., & Ammann, E. (2026). Human medical documentation significantly outperforms ChatGPT‐4o in critical clinical dimensions: A blinded comparative assessment in paediatric orthopaedics. Knee Surgery, Sports Traumatology, Arthroscopy, 34(5), 1910-1916. https://doi.org/10.1002/ksa.70357
Vancouver
Camathias C, Papp K, Betschart P, Speth B, Valderrabano V, Ammann E. Human medical documentation significantly outperforms ChatGPT‐4o in critical clinical dimensions: A blinded comparative assessment in paediatric orthopaedics. Knee Surgery, Sports Traumatology, Arthroscopy. 2026;34(5):1910-1916. doi:10.1002/ksa.70357
BibTeX
@article{Camathias2026,
author = {Camathias, Carlo and Papp, Kata and Betschart, Patrick and Speth, Berhard and Valderrabano, Victor and Ammann, Elias},
title = {Human medical documentation significantly outperforms ChatGPT‐4o in critical clinical dimensions: A blinded comparative assessment in paediatric orthopaedics},
journal = {Knee Surgery, Sports Traumatology, Arthroscopy},
year = {2026},
volume = {34},
number = {5},
pages = {1910--1916},
doi = {10.1002/ksa.70357}
}
EUVIMED