Secondary Logo

Journal Logo

Stroke severity quantification by critical care physicians in a mobile stroke unit

Hov, Maren R.a,g; Røislien, Joa,b; Lindner, Thomasa,c; Zakariassen, Erika,d,e; Bache, Kristi C.G.g; Solyga, Volker M.f; Russell, Davidg,h; Lund, Christian G.a,h

European Journal of Emergency Medicine: June 2019 - Volume 26 - Issue 3 - p 194–198
doi: 10.1097/MEJ.0000000000000529
Original articles

Background Cerebral revascularization in acute stroke requires robust diagnostic tools close to symptom onset. The quantitative National Institute of Health Stroke Scale (NIHSS) is widely used in-hospital, whereas shorter and less specific stroke scales are used in the prehospital field. This study explored the accuracy and potential clinical benefit of using NIHSS prehospitally.

Patients and methods Thirteen anesthesiologists trained in prehospital critical care enrolled patients with suspected acute stroke in a mobile stroke unit. NIHSS was completed twice in the acute phase: first prehospitally and then by an on-call resident neurologist at the receiving hospital. The agreement between prehospital and in-hospital NIHSS scores was assessed by a Bland–Altman plot, and inter-rater agreement for predefined clinical categories was tested using Cohen’s κ.

Results This Norwegian Acute Stroke Prehospital Project study included 40 patients for analyses. The mean numerical difference between prehospital and in-hospital NIHSS scores was 0.85, with corresponding limits of agreement from − 5.94 to 7.64. Inter-rater agreement (κ) for the corresponding clinical categories was 0.38. A prehospital diagnostic workup (NIHSS and computed tomographic examination) was completed in median (quartiles) 10 min (range: 7–14 min). Time between the prehospital and in-hospital NIHSS scores was median (quartiles) 40 min (32–48 min).

Conclusion Critical care physicians in a mobile stroke unit may use the NIHSS as a clinical tool in the assessment of patients experiencing acute stroke. The disagreement in NIHSS scores was mainly for very low values and would not have changed the handling of the patients.

aDepartment of Research and Development, The Norwegian Air Ambulance Foundation, Drøbak

bDepartment of Health Studies, University of Stavanger

cThe Regional Centre for Emergency Medical Research and Development, Stavanger

dDepartment of Global Public Health and Primary Care, University of Bergen

eNational Centre for Emergency Primary Health Care, Uni Research Health, Bergen

fDepartment of Neurology, Østfold Hospital, Kalnes

gDepartment of Clinical Medicine, University of Oslo

hDepartment of Neurology, Oslo University Hospital, Oslo, Norway

Received 27 July 2017 Accepted 19 November 2017

Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website,

Correspondence to Maren R. Hov, MD, Department of Research and Development, The Norwegian Air Ambulance Foundation, Holterveien 24, 1448 Drøbak, Norway Tel: +47 934 58492; fax: +47 649 04445; e-mail:

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Back to Top | Article Outline


Acute stroke care greatly depends on saving time to treatment by the early identification of patients who may benefit from cerebral revascularization, either as intravenous thrombolysis or mechanical thrombectomy, or as a combination of these two options. New treatment options call for restructuring of both prehospital and in-hospital settings. Recent publications suggest that new models for stroke identification should be tested in the prehospital setting and preferably in mobile stroke units (MSU) [1]. However, prehospital models solely dedicated to acute stroke care and run by in-hospital specialists may be difficult and costly to implement.

The ideal prehospital stroke scale should be accurate and rapid to perform. Several different easy-to-use scales, such as the Cincinnati prehospital stroke scale [2], the Face Arm Speech Test [3], and the Los Angeles Prehospital Stroke Scale [4], have been assessed in a prehospital setting. Unfortunately, these scales show low sensitivity and specificity and fail to identify up to 30% of patients with acute stroke [5]. Recent literature suggests that the in-hospital scale, National Institute of Health Stroke Scale (NIHSS), may be used in the prehospital setting if carried out by a stroke specialist via telemedicine [6]. Moreover, NIHSS may likely be the best nonradiological tool to identify patients with cerebral large vessel occlusion for direct triage to an invasive stroke center [7].

This study was carried out in a MSU staffed like the national Norwegian helicopter emergency medical services (HEMS) with an anesthesiologist trained to provide prehospital critical care to all critically ill patients including acute stroke. We investigated their ability and accuracy to perform real-time NIHSS scoring in a pre-existing nationwide emergency medical service (EMS), and the main aim of the study was to analyze the level of agreement between the prehospital and in-hospital NIHSS scores in the acute phase of stroke.

Back to Top | Article Outline

Patients and methods

The study is part of the Norwegian Acute Stroke Prehospital Project (NASPP), a pilot study conducted in the county of Østfold Norway, inhabited by ~285 000 people. It was carried out using a MSU (Mercedes Springer, Stuttgart, Germany), operating on weekdays from 8: 00 a.m. to 8: 00 p.m. for 85 days between October 2014 and January 2016. As NASPP was a pilot study, patients within a 15-min driving time to the hospital were not included to avoid the possibility of a prehospital delay.

The emergency medical communication center used the Norwegian index of emergency medicine as a decision tool for determination of the appropriate response [8]. Patients included in the study met the following inclusion criteria: age older than 18 years, not pregnant, clinical symptoms of acute stroke, and symptom onset within 4 h.

The NIHSS quantifies the severity of neurological symptoms in stroke by the functionality of 11 physical parameters and a scoring system ranging from 0–42 points (Supplementary Table 1, Supplemental digital content 1, High scores correspond with increased severity [9]. We aimed to explore the possible benefits of prehospital NIHSS and to group patients in a three-category scale indicating the clinical relevance of the prehospital score (Table 1).

Table 1

Table 1

The critical care physicians completed a 1-day course in the clinical assessment of acute stroke. This included a one-hour introductory course in NIHSS assessment followed by practical training including simulation with vibrant markers mimicking different neurological deficits. After the practical NIHSS session, all physicians completed a web-based certification program [10].

On arrival at patient’s site, the critical care physician performed a short patient history evaluation, a rapid assessment of vital signs, a NIHSS scoring, blood testing, and a cerebral computed tomography (CT) scan. The NIHSS score and the tentative diagnosis were noted in an online study form and kept blinded to the other physicians involved in the study. The patient’s history and the time of symptom onset were reported directly to the on-call resident neurologist who performed a new NIHSS scoring immediately after hospital arrival. The in-hospital physicians were responsible for the final diagnostic and therapeutic decisions.

The Norwegian regional ethics committee approved the study (project ID: 2098/2013). The patients gave their initial oral consent in the MSU, and a deferred written consent. In situations where a written consent could not be completed by the patient, the next of kin provided consent.

Back to Top | Article Outline

Statistical analysis

The prehospital and in-hospital NIHSS scores are presented in a Bland–Altman plot, where the difference between the two measurements are plotted against their mean [11]. The corresponding limits of agreement (LoA) are the limits for 95% of the observed differences, representing the actual variation in the data [12]. The LoA enables a comparison between the actual variation in the collected data and the clinically acceptable variation. In this study, a NIHSS score variability that led to a change in clinical category was considered of relevance to patient care, as a change in category may result in altered treatment options.

Inter-rater agreement for the corresponding categorized NIHSS data was calculated with Cohen’s κ. κ value less than or equal to 0.2 represents poor agreement, 0.21–0.4 fair agreement, 0.41–0.6 moderate agreement, 0.61–0.80 good agreement, and 0.81–1.0 excellent agreement [13].

Continuous data are presented as mean (SD) for Gaussian distributed and median (quartiles) for skewed data and data with outliers.

Statistical analyses were performed in SPSS, version 23 (IBM Corp., Armonk, New York, USA) and R 3.3.1 (University of Auckland, Auckland, New Zealand).

Back to Top | Article Outline


Of 68 patients examined with a prehospital cerebral CT scan in the NASPP MSU, 40 patients were finally included for statistical analyses (Fig. 1). Fourteen patients were excluded from the analyses (Supplementary Table 2, Supplemental digital content 2, Patients were predominantly female (65%) with a mean age of 67 years (range: 21–91 years).

Fig. 1

Fig. 1

The Bland–Altman plot is shown in Fig. 2. The mean difference between prehospital and in-hospital NIHSS score measurements was 0.85, and LoAs were − 5.94–7.64 (95% confidence interval: − 7.86–4.02 and 5.72–9.56), respectively. The wide LoAs are, however, mainly owing to a few patients with large differences in prehospital and in-hospital NIHSS scores. Patient’s prehospital and in-hospital NIHSS scores and clinical categorization are presented in Table 1.

Fig. 2

Fig. 2

Inter-rater agreement between anesthesiologist and neurologist in defining patient’s NIHSS values to clinical categories was fair (κ = 0.38). Most of the misclassifications were owing to 10 patients scored as NIHSS 2–5 in the MSU, but NIHSS 0 or 1 at hospital admission.

Time from first encounter with the MSU to a complete diagnostic workup including NIHSS, blood tests, and CT examination was median (quartiles) 10 min (range: 7–14 min). Time spent to carry out NIHSS specifically was not reported. Median time (quartiles) from symptom onset to prehospital NIHSS score was 1 h and 06 min (range: 47 min–1 h 59 min). The time difference between prehospital and in-hospital NIHSS scores was median (quartiles) 40 min (range: 32–48 min).

Back to Top | Article Outline


This pilot study shows that critical care physicians in a MSU independently may use the NIHSS as a reliable clinical tool for quantification of neurological symptoms in acute stroke. This opens for more specific prehospital clinical stroke assessment and hopefully a better and faster selection to revascularization therapy already in the very early prehospital phase.

After the breakthrough thrombectomy studies published in 2015 [1415], early identification and selection of patients who may benefit from thrombectomy has become increasingly important. By incorporating quantitative acute stroke diagnostics in the existing EMS or HEMS system, many more ‘of the right patients’ may be offered revascularization therapy.

An exact prehospital notification and triage in acute stroke seems efficient [1617], and stroke symptom quantification with NIHSS is relatively simple, fast, and well validated. NIHSS may help to identify patients with a probable large vessel occlusion, which is essential when making the decision for direct triage to an invasive stroke center for thrombectomy [7]. A Danish study suggested that a NIHSS score of 6 or more would identify most patients with large vessel occlusion [18]. This study grouped NIHSS values from prehospital and in-hospital assessment in the same patients and found that inter-rater agreement was fair, and the highest level of variability was in patients with mild symptoms. In patients with moderate to severe symptoms (NIHSS ≥ 6), there was little variability between prehospital and in-hospital scores. These findings are comparable to the results presented in this manuscript.

We found slightly higher NIHSS scores prehospital than in-hospital (Fig. 2). This tendency may primarily be interpreted as a systematic difference and could be explained by the often seen fluctuating clinical presentation, where spontaneous recanalization occur in up to 17% of patients with acute ischemic stroke [19]. A recent in-hospital study by Naess et al. [20] showed that the mean of NIHSS scores in the acute phase (first 3 h after symptom onset) improved by more than 3 score points.

Critical care physicians are trained to observe patients in a very systematically and quantitative manner. A prehospital incorporation of the NIHSS scale may allow prehospital and in-hospital physicians to assess their patients similarly during the first hours and days after a stroke. Furthermore, by incorporating NIHSS in the existing EMS and HEMS, the need to train neurologists and radiologists in prehospital critical care will be reduced [21–23]. Our NASPP model, combining a prehospital NIHSS scoring with a prehospital cerebral CT, may allow both initiation of prehospital thrombolysis and a high-quality triage to revascularization therapy [24].

This study has some limitations. First, the number of patients included was rather low as the MSU operated in a rural area, and this results in a large confidence interval in the analysis. In addition, patients with intracranial bleeding were admitted directly from the MSU to the regional neurosurgical department. Second, the critical care physicians and the neurologists conducted the NIHSS with a mean time difference of 40 min, which creates a bias for a direct comparison of prehospital and in-hospital NIHSS scores. However, the real-time setting in our study makes time intervals impossible to avoid.

Back to Top | Article Outline


Incorporation of NIHSS in the EMS may result in higher level of prehospital stroke competence, by establishing a ‘common language’ throughout the acute phase. We will explore stroke quantification using NIHSS in a real-time EMS run by paramedics in our future studies.

Back to Top | Article Outline


The authors thank Ann Kristin Wiik, Head of Research Department at the Norwegian Air Ambulance Foundation, for administrative support, and Professor Hans Morten Lossius, General Secretary of the Norwegian Air Ambulance Foundation, for making the Norwegian MSU concept a reality.

The Norwegian Air Ambulance Foundation funded this study.

Back to Top | Article Outline

Conflicts of interest

There are no conflicts of interest.

Back to Top | Article Outline


1. Scheitz JF, Abdul-Rahim AH, MacIsaac RL, Cooray C, Sucharew H, Kleindorfer D, et al. Clinical selection strategies to identify ischemic stroke patients with large anterior vessel occlusion: results from SITS-ISTR (Safe Implementation of Thrombolysis in Stroke International Stroke Thrombolysis Registry). Stroke 2017; 48:290–297.
2. Kothari R, Hall K, Brott T, Broderick J. Early stroke recognition: developing an out-of-hospital NIH Stroke Scale. Acad Emerg Med 1997; 4:986–990.
3. Harbison J, Hossain O, Jenkinson D, Davis J, Louw SJ, Ford GA. Diagnostic accuracy of stroke referrals from primary care, emergency room physicians, and ambulance staff using the face arm speech test. Stroke 2003; 34:71–76.
4. Kidwell CS, Saver JL, Schubert GB, Eckstein M, Starkman S. Design and retrospective analysis of the Los Angeles Prehospital Stroke Screen (LAPSS). Prehosp Emerg Care 1998; 2:267–273.
5. Brandler ES, Sharma M, Sinert RH, Levine SR. Prehospital stroke scales in urban environments: a systematic review. Neurology 2014; 82:2241–2249.
6. Kesinger MR, Sequeira DJ, Buffalini S, Guyette FX. Comparing National Institutes of Health Stroke Scale among a stroke team and helicopter emergency medical service providers. Stroke 2015; 46:575–578.
7. Vanacker P, Heldner MR, Amiguet M, Faouzi M, Cras P, Ntaios G, et al. Prediction of large vessel occlusions in acute stroke: National Institute of Health Stroke Scale is hard to beat. Crit Care Med 2016; 44:e336–e343.
8. Dewey HM, Donnan GA, Freeman EJ, Sharples CM, Macdonell RA, McNeil JJ, Thrift AG. Interrater reliability of the National Institutes of Health Stroke Scale: rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovasc Dis 1999; 9:323–327.
9. Brott T, Adams HP Jr, Olinger CP, Marler JR, Barsan WG, Biller J, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989; 20:864–870.
10. National Institutes of Health Stroke Scale. NIH stroke scale training campus. Available at: [Accessed 14 June 2017]
11. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1:307–310.
12. Giavarina D. Understanding Bland–Altman analysis. Biochem Med (Zagreb) 2015; 25:141–151.
13. Brennan P, Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ 1992; 304:1491–1494.
14. Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 2015; 372:11–20.
15. Goyal M, Menon BK, van Zwam WH, Dippel DW, Mitchell PJ, Demchuk AM, et al. Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet 2016; 387:1723–1731.
16. Kim SK, Lee SY, Bae HJ, Lee YS, Kim SY, Kang MJ, Cha JK. Pre-hospital notification reduced the door-to-needle time for iv t-PA in acute ischaemic stroke. Eur J Neurol 2009; 16:1331–1335.
17. Wendt M, Ebinger M, Kunz A, Rozanski M, Waldschmidt C, Weber JE, et al. Improved prehospital triage of patients with stroke in a specialized stroke ambulance: results of the pre-hospital acute neurological therapy and optimization of medical care in stroke study. Stroke 2015; 46:740–745.
18. Hansen CK, Christensen A, Ovesen C, Havsteen I, Christensen H. Stroke severity and incidence of acute large vessel occlusions in patients with hyper-acute cerebral ischemia: results from a prospective cohort study based on CT-angiography (CTA). Int J Stroke 2015; 10:336–342.
19. Kassem-Moussa H, Graffagnino C. Nonocclusion and spontaneous recanalization rates in acute ischemic stroke: a review of cerebral angiography studies. Arch Neurol 2002; 59:1870–1873.
20. Naess H, Kurtz M, Thomassen L, Waje-Andreassen U. Serial NIHSS scores in patients with acute cerebral infarction. Acta Neurol Scand 2016; 133:415–420.
21. Bowry R, Parker S, Rajan SS, Yamal JM, Wu TC, Richardson L, et al. Benefits of stroke treatment using a mobile stroke unit compared with standard management: the BEST-MSU Study Run-In Phase. Stroke 2015; 46:3370–3374.
22. Walter S, Kostopoulos P, Haass A, Keller I, Lesmeister M, Schlechtriemen T, et al. Diagnosis and treatment of patients with stroke in a mobile stroke unit versus in hospital: a randomised controlled trial. Lancet Neurol 2012; 11:397–404.
23. Ebinger M, Rozanski M, Waldschmidt C, Weber J, Wendt M, Winter B, et al. PHANTOM-S: the prehospital acute neurological therapy and optimization of medical care in stroke patients – study. Int J Stroke 2012; 7:348–353.
24. Hov MR, Nome T, Zakariassen E, Russell D, Roislien J, Lossius HM, Lund CG. Assessment of acute stroke cerebral CT examinations by anaesthesiologists. Acta Anaesthesiol Scand 2015; 59:1179–1186.

acute stroke; emergency medicine; mobile stroke unit; National Institute of Health Stroke Scale; neurology; Norwegian Acute Stroke Prehospital Project; prehospital; stroke diagnostics

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.