Transcription Accuracy of Automatic Speech Recognition for Orthodontic Clinical Records
Journal of Dental Research · 2026-06 · 105 (6) · 737-745
DOI: 10.1177/00220345251382452 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Dental Research, online erschienen am 03.11.2025. Verfasst von R O'Kane und 8 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, London, UK. Hauptthemen laut MEDLINE: Speech Recognition Software, Electronic Health Records, Natural Language Processing, Orthodontics. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: mrc, Medical Research Council, national institute for health and care research.
Abstract (Originalsprache)
Accurate clinical records are fundamental to dental practice. Automatic speech recognition (ASR) has the capacity to convert spoken clinical language into written text within the electronic health record; however, the accuracy of ASR in natural language processing for clinical dentistry remains uncertain. The aim of this study was to investigate the transcriptional accuracy of ASR systems using orthodontic clinical records as the experimental model. Specifically, we used 4 commercial ASR systems (Heidi Health, DigitalTCO, Dragon Medical One, Dragon Professional Anywhere), 5 application programming interfaces (Amazon, Google, Speechmatics, Whisper, GPT4oTranscribe), and a 2-stage pipeline coupling GPT4oTranscribe with the GPT4o large language model (LLM) for generative error correction (GPT4oTranscribeCorrected). Orthodontic diagnostic and treatment planning summaries ( n = 200; 10 subject domains; 43,408 words; 6 h of audio) were narrated and recorded for analysis. The primary outcome was domain word error rate (DWER), which investigates clinical terminological transcription errors against the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) database. Secondary outcomes included nondomain WER (N-DWER), lexical accuracy (Recall-Oriented Understudy for Gisting Evaluation [ROUGE] score), semantic similarity (Bidirectional Encoder Representations from Transformers [BERT] and Bidirectional and Auto-Regressive Transformer [BART] scores), hallucinations (transcribed text not in the spoken input), and qualitative error analysis. GPT4oTranscribeCorrected was transcriptionally most accurate (DWER = 3.5%; WER = 3.7%), with DWER decreasing by 54.9% versus GPT4oTranscribe. Heidi Health was the highest-performing commercial system (DWER = 6.2%; WER = 5.4%), with Dragon Professional Anywhere being the worst (WER = 33.9%). All systems were less accurate with technical vocabulary (DWER > N-DWER; P < 0.001), except GPT4oTranscribeCorrected. Significant differences were seen across systems for ROUGE, BERT, and BART scores ( P < 0.001). Based on post hoc pairwise comparisons, GPT4oTranscribeCorrected performed best and Dragon Professional Anywhere was consistently worst for lexical and semantic errors. Hallucinations were absent except for Whisper ( n = 57) and DigitalTCO ( n = 1). Across systems, background noise increased DWER and WER ( P < 0.001). Importantly, clinically significant errors were seen with all systems, ranging from 2% to 66% (GPT4oTranscribeCorrected clean; Dragon Medical One background noise, respectively). Variation in narrator accent had no effect in clean conditions ( P = 0.65) and a small effect with background noise ( P = 0.001). ASR systems deliver single-digit transcription error rates, particularly when coupled with LLM-based correction, but clinically significant errors persist. The verification of clinical records is essential when using current ASR systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Centre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, London, UK.
- Department of Orthodontics, Guy's and St Thomas' NHS Foundation Trust, London, UK.
- Department of Orthodontics, Royal Alexandra Children's Hospital, Brighton and Sussex University Hospitals NHS Foundation Trust, Sussex, UK.
- Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Switzerland.
ORCID-Profile: R O'Kane, D Stonehouse-Smith, J Seehra, S N Papageorgiou, M T Cobourne
Förderung
- mrc
- Medical Research Council
- national institute for health and care research
Zitieren
APA
O'Kane, R., Stonehouse-Smith, D., Ota, L. C. U., Patel, R., Johnson, N., Slipper, C., Seehra, J., Papageorgiou, S. N., & Cobourne, M. T. (2026). Transcription Accuracy of Automatic Speech Recognition for Orthodontic Clinical Records. Journal of Dental Research, 105(6), 737-745. https://doi.org/10.1177/00220345251382452
Vancouver
O'Kane R, Stonehouse-Smith D, Ota LCU, Patel R, Johnson N, Slipper C, et al. Transcription Accuracy of Automatic Speech Recognition for Orthodontic Clinical Records. Journal of Dental Research. 2026;105(6):737-745. doi:10.1177/00220345251382452
BibTeX
@article{OKane2026,
author = {O'Kane, R and Stonehouse-Smith, D and Ota, L C U and Patel, R and Johnson, N and Slipper, C and Seehra, J and Papageorgiou, S N and Cobourne, M T},
title = {Transcription Accuracy of Automatic Speech Recognition for Orthodontic Clinical Records},
journal = {Journal of Dental Research},
year = {2026},
volume = {105},
number = {6},
pages = {737--745},
doi = {10.1177/00220345251382452}
}
EUVIMED