EUVIMEDEUVIMED entdecken
BeobachtungsstudieOpen Access · CC BY-NC-ND 4.0

Automating sedation state assessments using natural language processing

Aaron Conway iD, Jack Li, Mohammad Goudarzi Rad, Blair Warren iD, Sebastian Mafeld, Babak Taati

Journal of Nursing Scholarship · 2025

DOI: 10.1111/jnu.12968 ↗

Kurz gefasst

Beobachtungsstudie in der Zeitschrift Journal of Nursing Scholarship, 2025. Verfasst von Aaron Conway und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Nursing, QUT (Queensland University of Technology), Brisbane, Queensland, Australia. Hauptthemen laut MEDLINE: Natural Language Processing, Procedural Sedation, Hypnotics and Sedatives. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: University of Toronto Temerty Centre for AI Research and Education in Medicine (T-CAIREM), University of Toronto Data Sciences Institute (DSI).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Common goals for procedural sedation are to control pain and ensure the patient is not moving to an extent that is impeding safe progress or completion of the procedure. Clinicians perform regular assessments of the adequacy of procedural sedation in accordance with these goals to inform their decision‐making around sedation titration and also for documentation of the care provided. Natural language processing could be applied to real‐time transcriptions of audio recordings made during procedures in order to classify sedation states that involve movement and pain, which could then be integrated into clinical documentation systems. The aim of this study was to determine whether natural language processing algorithms will work with sufficient accuracy to detect sedation states during procedural sedation.

Design

A prospective observational study was conducted.

Methods

Audio recordings from consenting participants undergoing elective procedures performed in the interventional radiology suite at a large academic hospital were transcribed using an automated speech recognition model. Sentences of transcribed text were used to train and evaluate several different NLP pipelines for a text classification task. The NLP pipelines we evaluated included a simple Bag‐of‐Words (BOW) model, an ensemble architecture combining a linear BOW model and a “token‐to‐vector” (Tok2Vec) component, and a transformer‐based architecture using the RoBERTa pre‐trained model.

Results

A total of 15,936 sentences from transcriptions of 82 procedures was included in the analysis. The RoBERTa model achieved the highest performance among the three models with an area under the ROC curve (AUC‐ROC) of 0.97, an F1 score of 0.87, a precision of 0.86, and a recall of 0.89. The Ensemble model showed a similarly high AUC‐ROC of 0.96, but lower F1 score of 0.79, precision of 0.83, and recall of 0.77. The BOW approach achieved an AUC‐ROC of 0.97 and the F1 score was 0.7, precision was 0.83 and recall was 0.66.

Conclusion

The transformer‐based architecture using the RoBERTa pre‐trained model achieved the best classification performance. Further research is required to confirm the that this natural language processing pipeline can accurately perform text classifications with real‐time audio data to allow for automated sedation state assessments. Clinical Relevance Automating sedation state assessments using natural language processing pipelines would allow for more timely documentation of the care received by sedated patients, and, at the same time, decrease documentation burden for clinicians. Downstream applications can also be generated from the classifications, including for example real‐time visualizations of sedation state, which may facilitate improved communication of the adequacy of the sedation between clinicians, who may be performing supervision remotely. Also, accumulation of sedation state assessments from multiple procedures may reveal insights into the efficacy of particular sedative medications or identify procedures where the current approach for sedation and analgesia is not optimal (i.e. a significant amount of time spent in “pain” or “movement” sedation states).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingProcedural SedationHypnotics and SedativesHumansProspective StudiesFemaleMaleMiddle AgedAlgorithmsAdult

Stichwörter der Autor:innen

artificial intelligenceclinical documentationmachine learningnatural language processingnursingprocedural sedation

Substanzen

Hypnotics and Sedatives

Autor:innen und Institutionen

  • School of Nursing, QUT (Queensland University of Technology), Brisbane, Queensland, Australia.
  • Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada.
  • Joint Department of Medical Imaging, University Health Network, Toronto, Ontario, Canada.
  • Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada.
  • KITE Research Institute - Toronto Rehabilitation Institute, University Health Network, Toronto, Ontario, Canada.
  • Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.

ORCID-Profile: Aaron Conway, Blair Warren

Förderung

  • University of Toronto Temerty Centre for AI Research and Education in Medicine (T-CAIREM)
  • University of Toronto Data Sciences Institute (DSI)

Zitieren

APA

Conway, A., Li, J., Rad, M. G., Warren, B., Mafeld, S., & Taati, B. (2025). Automating sedation state assessments using natural language processing. Journal of Nursing Scholarship. https://doi.org/10.1111/jnu.12968

Vancouver

Conway A, Li J, Rad MG, Warren B, Mafeld S, Taati B. Automating sedation state assessments using natural language processing. Journal of Nursing Scholarship. 2025. doi:10.1111/jnu.12968

BibTeX
@article{Conway2025,
  author = {Conway, Aaron and Li, Jack and Rad, Mohammad Goudarzi and Warren, Blair and Mafeld, Sebastian and Taati, Babak},
  title = {Automating sedation state assessments using natural language processing},
  journal = {Journal of Nursing Scholarship},
  year = {2025},
  doi = {10.1111/jnu.12968}
}
Kostenlos starten →