Institutional members access full text with Ovid®

Share this article on:

A Comparison of the Ability of the Physiologic Components of Medical Emergency Team Criteria and the U.K. National Early Warning Score to Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes*

Smith, Gary B. FRCA, FRCP; Prytherch, David R. PhD, MIPEM, CSci; Jarvis, Stuart PhD; Kovacs, Caroline BSc; Meredith, Paul PhD; Schmidt, Paul E. MRCP, BMedSc, MBA; Briggs, Jim BA, DPhil

doi: 10.1097/CCM.0000000000002000
Clinical Investigations

Objective: To compare the ability of medical emergency team criteria and the National Early Warning Score to discriminate cardiac arrest, unanticipated ICU admission and death within 24 hours of a vital signs measurement, and to quantify the associated workload.

Design: Retrospective cohort study.

Setting: A large U.K. National Health Service District General Hospital.

Patients: Adults hospitalized from May 25, 2011, to December 31, 2013.

Interventions: None.

Measurements and Main Results: We applied the National Early Warning Score and 44 sets of medical emergency team criteria to a database of 2,245,778 vital signs sets (103,998 admissions). The National Early Warning Score’s performance was assessed using the area under the receiver-operating characteristic curve and compared with sensitivity/specificity for different medical emergency team criteria. Area under the receiver-operating characteristic curve (95% CI) for the National Early Warning Score for the combined outcome (i.e., death, cardiac arrest, or unanticipated ICU admission) was 0.88 (0.88–0.88). A National Early Warning Score value of 7 had sensitivity/specificity values of 44.5% and 97.4%, respectively. For the 44 sets of medical emergency team criteria studied, sensitivity ranged from 19.6% to 71.2% and specificity from 71.5% to 98.5%. For all outcomes, the position of the National Early Warning Score receiver-operating characteristic curve was above and to the left of all medical emergency team criteria points, indicating better discrimination. Similarly, the positions of all medical emergency team criteria points were above and to the left of the National Early Warning Score efficiency curve, indicating higher workloads (trigger rates).

Conclusions: When medical emergency team systems are compared to a National Early Warning Score value of greater than or equal to 7, some medical emergency team systems have a higher sensitivity than National Early Warning Score values of greater than or equal to 7. However, all of these medical emergency team systems have a lower specificity and would generate greater workloads.

1Centre of Postgraduate Medical Research and Education, Faculty of Health and Social Sciences, University of Bournemouth, Bournemouth, UK.

2Department of Research & Development, Portsmouth Hospitals NHS Trust, Portsmouth, UK.

3Centre for Healthcare Modelling & Informatics, School of Computing, University of Portsmouth, Portsmouth, United Kingdom.

4Department of Health Sciences, University of York, York, United Kingdom.

*See also p. 2283.

This work was performed at Portsmouth Hospitals NHS Trust & University of Portsmouth.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal).

Dr. Smith consulted for Laerdal Medical, United Kingdom (Independent contractor to Laerdal Medical, Orpington, Kent on continuous monitoring of vital signs); received grant support from the National Institute for Health Research (NIHR) (grant relating to research into nurse staffing and vital signs observations commenced in April 2015); lectured for the Swedish Medical Association (Honorarium from the Swedish Medical Association for lecture in 2014) and for Huddersfield University, United Kingdom (Honorarium from Huddersfield University re Masterclass on National Early Warning Scores [NEWS]); and received support for travel from The Learning Clinic (TLC). (He was an unpaid research advisor to TLC until May 2016 and has received reimbursement of travel expenses for attending symposia in the UK). He disclosed other relationships: His wife was a shareholder in TLC (manufacturer of VitalPAC, the system used to collect data used in the current research) until October 2015; he acts as an expert advisor to the National Institute for Health and Clinical Excellence during the development of the NICE clinical guideline (50: “Acutely ill patients in hospital: recognition of and response to acute illness in adults in hospital” development group). He was a member of the National Patient Safety Agency committee that wrote the two reports on “Recognizing and responding appropriately to early signs of deterioration in hospitalized patients” and “Safer care for the acutely ill patient: learning from serious incidents.” He is a member of the Royal College of Physicians of London’s NEWS Development and Implementation Group. His institution received royalties and other support from TLC. (At the time of the research, his previous employers, Portsmouth Hospitals NHS Trust [PHT], had a royalty agreement with TLC to pay for the use of PHT intellectual property within TLC VitalPAC product [VitalPAC is the system used to collect data used in the current research]. He worked for PHT from 1985 to 2011 [until May 31, 2011].) Dr. Prytherch is employed by PHT, received support for travel (he was an unpaid research advisor to TLC until May 2016. In the past they have reimbursed his travel expenses for attending meetings in the UK), and disclosed other relationships (His wife was a minor shareholder in TLC until October 2015 and he assisted Royal College of Physicians [London] in the analysis of data to validate the NEWS.) His institution received other support (at the time of the research, PHT had a royalty agreement with TLC. TLC paid royalties to PHT for IP within their VitalPAC product. VitalPAC was used to collect the vital signs data used in this research), grant support from the Wellcome Trust (grant to start August 2015) and the NIHR (grant started April 2015), and royalties (at the time of his research, PHT had a royalty agreement with TLC. TLC paid royalties to PHT for IP within their VitalPAC product). Dr. Meredith is employed by PHT and received royalties from TLC (at the time of his research, my employer PHT had a royalty agreement with TLC. TLC paid royalties to PHT for IP within their VitalPAC product. VitalPAC was used to collect the vital signs data used in this research). His institution received other support from TLC until October 2015. (At the time of the research, PHT had a royalty agreement TLC. TLC paid royalties to PHT for IP within their VitalPAC product. VitalPAC was used to collect the vital signs data used in this research), grant support from Wellcome Trust (grant to start August 2015) and NIHR (grant started April 2015). Dr. Schmidt received royalties from TLC (at the time of his research, PHT had a royalty agreement with TLC. TLC paid royalties to PHT for IP within their VitalPAC product. VitalPAC was used to collect the vital signs data used in this research). His institution received royalties from TLC (related to development of VitalPAC, the information system used to collect data for this study). Dr. Briggs received support for article research from the Technology Strategy Board. His institution received grant support from TLC and Technology Strategy Board (some of the preliminary work was funded by a Knowledge Transfer Partnership award). The remaining authors have disclosed that they do not have any potential conflicts of interest.

For information regarding this article, E-mail: gbsresearch@virginmedia.com

Copyright © by 2016 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.