Interrater Agreement on National Institutes of Health Stroke Scale Between Paramedics and Stroke Physicians: Validation Study for the Digital Training Model in the Paramedic Norwegian Acute Stroke Prehospital Project
JMIR Neurotechnology · 2022
DOI: 10.2196/39444 ↗At a glance
Journal article in JMIR Neurotechnology, 2022. Written by Mona Guterud and 10 co-authors. Institution: Department of Research Norwegian Air Ambulance Foundation Oslo Norway. Cited 5 times so far (Crossref); 37 references. The full text is freely available in PubMed Central.
Topics: Schlaganfall
Abstract (original language)
Background
Time spent in the prehospital phase of acute stroke care is multifactorial and has an effect on the possibilities for acute treatment. Communication between paramedics and the in-hospital stroke team directly affects time to treatment. A mutual stroke scale such as the National Institutes of Health Stroke Scale (NIHSS) may improve communication quality. The Paramedic Norwegian Acute Stroke Prehospital Project (ParaNASPP) was a stepped-wedge, randomized trial of stroke screening using NIHSS in the ambulance where the intervention was training paramedics in stroke and the NIHSS, with the use of NIHSS made into a mobile app to guide the examination and facilitate communication with the in-hospital stroke team.
Objective
The aim of this study was to validate the digital training model from the ParaNASPP clinical trial.
Methods
In total, 24 paramedics were recruited from Oslo University Hospital in Norway to complete the ParaNASPP training model; 20 exclusive videos with predefined NIHSS scores were recorded; and 4 stroke physicians from Oslo University Hospital were included for reference. Bland-Altman plots with 95% limits of agreement (LoA) were calculated—first comparing paramedics and stroke physicians to the predefined scores and then with each other. The predefined LoA were set to 3 points. To align with clinical practice, NIHSS scores were also dichotomized into 2 categories: from 0-5 (minor stroke) or ≥6 (moderate and major stroke), and agreement was calculated using Cohen κ.
Results
The videos (n=20) had a median (range) NIHSS score of 7 (0-31). The paramedics’ scores were slightly higher than the predefined scores with a mean difference of –0.38 and the LoA ranging from –4.04 to 3.29. The paramedics scored higher than the stroke physicians with a mean difference of –0.39, with the LoA ranging from –4.58 to 3.80. When the NIHSS scores were dichotomized, Cohen κ was 0.89 between the predefined scores and paramedics, 0.92 between the predefined scores and stroke physicians, and 0.81 between the paramedics and stroke physicians, all indicating very good agreement.
Conclusions
The paramedics scored higher than both the predefined scores and stroke physicians’ scores, hence the predefined LoA were not met. However, the width of the LoA was smaller than seen when experienced neurologists are compared. When the NIHSS scores were dichotomized, the paramedics achieved very good agreement with both the predefined scores and stroke physicians’ scores. This study demonstrates the possibilities for the transfer of clinical competence in digital simulation training.
Subjects
Author keywords
Authors and institutions
- Department of Research Norwegian Air Ambulance Foundation Oslo Norway.
- Division of Prehospital Services Oslo University Hospital Oslo Norway.
- Institute of Clinical Medicine University of Oslo Oslo Norway.
- Department of Neurology Stroke Unit Oslo University Hospital Oslo Norway.
- Institute of Basic Medical Sciences University of Oslo Oslo Norway.
- Research and Dissemination Østfold University College Halden Norway.
ORCID profiles: Mona Guterud, Helge Fagerheim Bugge, Jo Røislien, Karianne Larsen, Erik Eriksen, Svein Håkon Ingebretsen, Martin Lerstang Mikkelsen, Jo Kramer-Johansen, Kristi G Bache, Else Charlotte Sandset, Maren Ranhoff Hov
Cite
APA
Guterud, M., Bugge, H. F., Røislien, J., Larsen, K., Eriksen, E., Ingebretsen, S. H., Mikkelsen, M. L., Kramer-Johansen, J., Bache, K. G., Sandset, E. C., & Hov, M. R. (2022). Interrater Agreement on National Institutes of Health Stroke Scale Between Paramedics and Stroke Physicians: Validation Study for the Digital Training Model in the Paramedic Norwegian Acute Stroke Prehospital Project. JMIR Neurotechnology. https://doi.org/10.2196/39444
Vancouver
Guterud M, Bugge HF, Røislien J, Larsen K, Eriksen E, Ingebretsen SH, et al. Interrater Agreement on National Institutes of Health Stroke Scale Between Paramedics and Stroke Physicians: Validation Study for the Digital Training Model in the Paramedic Norwegian Acute Stroke Prehospital Project. JMIR Neurotechnology. 2022. doi:10.2196/39444
BibTeX
@article{Guterud2022,
author = {Guterud, Mona and Bugge, Helge Fagerheim and Røislien, Jo and Larsen, Karianne and Eriksen, Erik and Ingebretsen, Svein Håkon and Mikkelsen, Martin Lerstang and Kramer-Johansen, Jo and Bache, Kristi G and Sandset, Else Charlotte and Hov, Maren Ranhoff},
title = {Interrater Agreement on National Institutes of Health Stroke Scale Between Paramedics and Stroke Physicians: Validation Study for the Digital Training Model in the Paramedic Norwegian Acute Stroke Prehospital Project},
journal = {JMIR Neurotechnology},
year = {2022},
doi = {10.2196/39444}
}
EUVIMED