Acute stroke care greatly depends on saving time to treatment by the early identification of patients who may benefit from cerebral revascularization, either as intravenous thrombolysis or mechanical thrombectomy, or as a combination of these two options. New treatment options call for restructuring of both prehospital and in-hospital settings. Recent publications suggest that new models for stroke identification should be tested in the prehospital setting and preferably in mobile stroke units (MSU) . However, prehospital models solely dedicated to acute stroke care and run by in-hospital specialists may be difficult and costly to implement.
The ideal prehospital stroke scale should be accurate and rapid to perform. Several different easy-to-use scales, such as the Cincinnati prehospital stroke scale , the Face Arm Speech Test , and the Los Angeles Prehospital Stroke Scale , have been assessed in a prehospital setting. Unfortunately, these scales show low sensitivity and specificity and fail to identify up to 30% of patients with acute stroke . Recent literature suggests that the in-hospital scale, National Institute of Health Stroke Scale (NIHSS), may be used in the prehospital setting if carried out by a stroke specialist via telemedicine . Moreover, NIHSS may likely be the best nonradiological tool to identify patients with cerebral large vessel occlusion for direct triage to an invasive stroke center .
This study was carried out in a MSU staffed like the national Norwegian helicopter emergency medical services (HEMS) with an anesthesiologist trained to provide prehospital critical care to all critically ill patients including acute stroke. We investigated their ability and accuracy to perform real-time NIHSS scoring in a pre-existing nationwide emergency medical service (EMS), and the main aim of the study was to analyze the level of agreement between the prehospital and in-hospital NIHSS scores in the acute phase of stroke.
Patients and methods
The study is part of the Norwegian Acute Stroke Prehospital Project (NASPP), a pilot study conducted in the county of Østfold Norway, inhabited by ~285 000 people. It was carried out using a MSU (Mercedes Springer, Stuttgart, Germany), operating on weekdays from 8: 00 a.m. to 8: 00 p.m. for 85 days between October 2014 and January 2016. As NASPP was a pilot study, patients within a 15-min driving time to the hospital were not included to avoid the possibility of a prehospital delay.
The emergency medical communication center used the Norwegian index of emergency medicine as a decision tool for determination of the appropriate response . Patients included in the study met the following inclusion criteria: age older than 18 years, not pregnant, clinical symptoms of acute stroke, and symptom onset within 4 h.
The NIHSS quantifies the severity of neurological symptoms in stroke by the functionality of 11 physical parameters and a scoring system ranging from 0–42 points (Supplementary Table 1, Supplemental digital content 1, http://links.lww.com/EJEM/A190). High scores correspond with increased severity . We aimed to explore the possible benefits of prehospital NIHSS and to group patients in a three-category scale indicating the clinical relevance of the prehospital score (Table 1).
The critical care physicians completed a 1-day course in the clinical assessment of acute stroke. This included a one-hour introductory course in NIHSS assessment followed by practical training including simulation with vibrant markers mimicking different neurological deficits. After the practical NIHSS session, all physicians completed a web-based certification program .
On arrival at patient’s site, the critical care physician performed a short patient history evaluation, a rapid assessment of vital signs, a NIHSS scoring, blood testing, and a cerebral computed tomography (CT) scan. The NIHSS score and the tentative diagnosis were noted in an online study form and kept blinded to the other physicians involved in the study. The patient’s history and the time of symptom onset were reported directly to the on-call resident neurologist who performed a new NIHSS scoring immediately after hospital arrival. The in-hospital physicians were responsible for the final diagnostic and therapeutic decisions.
The Norwegian regional ethics committee approved the study (project ID: 2098/2013). The patients gave their initial oral consent in the MSU, and a deferred written consent. In situations where a written consent could not be completed by the patient, the next of kin provided consent.
The prehospital and in-hospital NIHSS scores are presented in a Bland–Altman plot, where the difference between the two measurements are plotted against their mean . The corresponding limits of agreement (LoA) are the limits for 95% of the observed differences, representing the actual variation in the data . The LoA enables a comparison between the actual variation in the collected data and the clinically acceptable variation. In this study, a NIHSS score variability that led to a change in clinical category was considered of relevance to patient care, as a change in category may result in altered treatment options.
Inter-rater agreement for the corresponding categorized NIHSS data was calculated with Cohen’s κ. κ value less than or equal to 0.2 represents poor agreement, 0.21–0.4 fair agreement, 0.41–0.6 moderate agreement, 0.61–0.80 good agreement, and 0.81–1.0 excellent agreement .
Continuous data are presented as mean (SD) for Gaussian distributed and median (quartiles) for skewed data and data with outliers.
Statistical analyses were performed in SPSS, version 23 (IBM Corp., Armonk, New York, USA) and R 3.3.1 (University of Auckland, Auckland, New Zealand).
Of 68 patients examined with a prehospital cerebral CT scan in the NASPP MSU, 40 patients were finally included for statistical analyses (Fig. 1). Fourteen patients were excluded from the analyses (Supplementary Table 2, Supplemental digital content 2, http://links.lww.com/EJEM/A191). Patients were predominantly female (65%) with a mean age of 67 years (range: 21–91 years).
The Bland–Altman plot is shown in Fig. 2. The mean difference between prehospital and in-hospital NIHSS score measurements was 0.85, and LoAs were − 5.94–7.64 (95% confidence interval: − 7.86–4.02 and 5.72–9.56), respectively. The wide LoAs are, however, mainly owing to a few patients with large differences in prehospital and in-hospital NIHSS scores. Patient’s prehospital and in-hospital NIHSS scores and clinical categorization are presented in Table 1.
Inter-rater agreement between anesthesiologist and neurologist in defining patient’s NIHSS values to clinical categories was fair (κ = 0.38). Most of the misclassifications were owing to 10 patients scored as NIHSS 2–5 in the MSU, but NIHSS 0 or 1 at hospital admission.
Time from first encounter with the MSU to a complete diagnostic workup including NIHSS, blood tests, and CT examination was median (quartiles) 10 min (range: 7–14 min). Time spent to carry out NIHSS specifically was not reported. Median time (quartiles) from symptom onset to prehospital NIHSS score was 1 h and 06 min (range: 47 min–1 h 59 min). The time difference between prehospital and in-hospital NIHSS scores was median (quartiles) 40 min (range: 32–48 min).
This pilot study shows that critical care physicians in a MSU independently may use the NIHSS as a reliable clinical tool for quantification of neurological symptoms in acute stroke. This opens for more specific prehospital clinical stroke assessment and hopefully a better and faster selection to revascularization therapy already in the very early prehospital phase.
After the breakthrough thrombectomy studies published in 2015 , early identification and selection of patients who may benefit from thrombectomy has become increasingly important. By incorporating quantitative acute stroke diagnostics in the existing EMS or HEMS system, many more ‘of the right patients’ may be offered revascularization therapy.
An exact prehospital notification and triage in acute stroke seems efficient , and stroke symptom quantification with NIHSS is relatively simple, fast, and well validated. NIHSS may help to identify patients with a probable large vessel occlusion, which is essential when making the decision for direct triage to an invasive stroke center for thrombectomy . A Danish study suggested that a NIHSS score of 6 or more would identify most patients with large vessel occlusion . This study grouped NIHSS values from prehospital and in-hospital assessment in the same patients and found that inter-rater agreement was fair, and the highest level of variability was in patients with mild symptoms. In patients with moderate to severe symptoms (NIHSS ≥ 6), there was little variability between prehospital and in-hospital scores. These findings are comparable to the results presented in this manuscript.
We found slightly higher NIHSS scores prehospital than in-hospital (Fig. 2). This tendency may primarily be interpreted as a systematic difference and could be explained by the often seen fluctuating clinical presentation, where spontaneous recanalization occur in up to 17% of patients with acute ischemic stroke . A recent in-hospital study by Naess et al.  showed that the mean of NIHSS scores in the acute phase (first 3 h after symptom onset) improved by more than 3 score points.
Critical care physicians are trained to observe patients in a very systematically and quantitative manner. A prehospital incorporation of the NIHSS scale may allow prehospital and in-hospital physicians to assess their patients similarly during the first hours and days after a stroke. Furthermore, by incorporating NIHSS in the existing EMS and HEMS, the need to train neurologists and radiologists in prehospital critical care will be reduced [21–23]. Our NASPP model, combining a prehospital NIHSS scoring with a prehospital cerebral CT, may allow both initiation of prehospital thrombolysis and a high-quality triage to revascularization therapy .
This study has some limitations. First, the number of patients included was rather low as the MSU operated in a rural area, and this results in a large confidence interval in the analysis. In addition, patients with intracranial bleeding were admitted directly from the MSU to the regional neurosurgical department. Second, the critical care physicians and the neurologists conducted the NIHSS with a mean time difference of 40 min, which creates a bias for a direct comparison of prehospital and in-hospital NIHSS scores. However, the real-time setting in our study makes time intervals impossible to avoid.
Incorporation of NIHSS in the EMS may result in higher level of prehospital stroke competence, by establishing a ‘common language’ throughout the acute phase. We will explore stroke quantification using NIHSS in a real-time EMS run by paramedics in our future studies.
The authors thank Ann Kristin Wiik, Head of Research Department at the Norwegian Air Ambulance Foundation, for administrative support, and Professor Hans Morten Lossius, General Secretary of the Norwegian Air Ambulance Foundation, for making the Norwegian MSU concept a reality.
The Norwegian Air Ambulance Foundation funded this study.
Conflicts of interest
There are no conflicts of interest.
1. Scheitz JF, Abdul-Rahim AH, MacIsaac RL, Cooray C, Sucharew H, Kleindorfer D, et al. Clinical selection strategies to identify ischemic stroke patients with large anterior vessel occlusion: results from SITS-ISTR (Safe Implementation of Thrombolysis in Stroke International Stroke Thrombolysis Registry). Stroke 2017; 48:290–297.
2. Kothari R, Hall K, Brott T, Broderick J. Early stroke recognition: developing an out-of-hospital NIH Stroke Scale. Acad Emerg Med 1997; 4:986–990.
3. Harbison J, Hossain O, Jenkinson D, Davis J, Louw SJ, Ford GA. Diagnostic accuracy of stroke referrals from primary care, emergency room physicians, and ambulance staff using the face arm speech test. Stroke 2003; 34:71–76.
4. Kidwell CS, Saver JL, Schubert GB, Eckstein M, Starkman S. Design and retrospective analysis of the Los Angeles Prehospital
Stroke Screen (LAPSS). Prehosp Emerg Care 1998; 2:267–273.
5. Brandler ES, Sharma M, Sinert RH, Levine SR. Prehospital
stroke scales in urban environments: a systematic review. Neurology
6. Kesinger MR, Sequeira DJ, Buffalini S, Guyette FX. Comparing National Institutes of Health Stroke Scale among a stroke team and helicopter emergency medical service providers. Stroke 2015; 46:575–578.
7. Vanacker P, Heldner MR, Amiguet M, Faouzi M, Cras P, Ntaios G, et al. Prediction of large vessel occlusions in acute stroke
: National Institute of Health Stroke Scale
is hard to beat. Crit Care Med 2016; 44:e336–e343.
8. Dewey HM, Donnan GA, Freeman EJ, Sharples CM, Macdonell RA, McNeil JJ, Thrift AG. Interrater reliability of the National Institutes of Health Stroke Scale: rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovasc Dis 1999; 9:323–327.
9. Brott T, Adams HP Jr, Olinger CP, Marler JR, Barsan WG, Biller J, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989; 20:864–870.
11. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1:307–310.
12. Giavarina D. Understanding Bland–Altman analysis. Biochem Med (Zagreb) 2015; 25:141–151.
13. Brennan P, Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ 1992; 304:1491–1494.
14. Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 2015; 372:11–20.
15. Goyal M, Menon BK, van Zwam WH, Dippel DW, Mitchell PJ, Demchuk AM, et al. Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet 2016; 387:1723–1731.
16. Kim SK, Lee SY, Bae HJ, Lee YS, Kim SY, Kang MJ, Cha JK. Pre-hospital notification reduced the door-to-needle time for iv t-PA in acute ischaemic stroke. Eur J Neurol 2009; 16:1331–1335.
17. Wendt M, Ebinger M, Kunz A, Rozanski M, Waldschmidt C, Weber JE, et al. Improved prehospital
triage of patients with stroke in a specialized stroke ambulance: results of the pre-hospital acute neurological therapy and optimization of medical care in stroke study. Stroke 2015; 46:740–745.
18. Hansen CK, Christensen A, Ovesen C, Havsteen I, Christensen H. Stroke severity and incidence of acute large vessel occlusions in patients with hyper-acute cerebral ischemia: results from a prospective cohort study based on CT-angiography (CTA). Int J Stroke 2015; 10:336–342.
19. Kassem-Moussa H, Graffagnino C. Nonocclusion and spontaneous recanalization rates in acute ischemic stroke: a review of cerebral angiography studies. Arch Neurol 2002; 59:1870–1873.
20. Naess H, Kurtz M, Thomassen L, Waje-Andreassen U. Serial NIHSS scores in patients with acute cerebral infarction. Acta Neurol Scand 2016; 133:415–420.
21. Bowry R, Parker S, Rajan SS, Yamal JM, Wu TC, Richardson L, et al. Benefits of stroke treatment using a mobile stroke unit
compared with standard management: the BEST-MSU Study Run-In Phase. Stroke 2015; 46:3370–3374.
22. Walter S, Kostopoulos P, Haass A, Keller I, Lesmeister M, Schlechtriemen T, et al. Diagnosis and treatment of patients with stroke in a mobile stroke unit
versus in hospital: a randomised controlled trial. Lancet Neurol 2012; 11:397–404.
23. Ebinger M, Rozanski M, Waldschmidt C, Weber J, Wendt M, Winter B, et al. PHANTOM-S: the prehospital
acute neurological therapy and optimization of medical care in stroke patients – study. Int J Stroke 2012; 7:348–353.
24. Hov MR, Nome T, Zakariassen E, Russell D, Roislien J, Lossius HM, Lund CG. Assessment of acute stroke
cerebral CT examinations by anaesthesiologists. Acta Anaesthesiol Scand 2015; 59:1179–1186.