McGillis Hall, Linda PhD, RN, FAAN, FCAHS; Wodchis, Walter P. PhD; Ma, Xiaomu MSc; Johnson, Stacey MN, RN
EFFORTS TO SUPPORT evidence-based management in the health care system have resulted in the development of clinical databases to enhance managerial decision making. The need for governments, policy makers, health care executives, and leaders to have data that provide a clearer understanding of patient health outcomes has long been identified as a need for the planning, measurement, and effective allocation of health care resources. Across North America, a number of initiatives have emerged to create databases that assist in determining the contribution that nurses make to patient health outcomes.1–3 This research provides evidence from a large health outcomes database developed in Canada that includes measures of patient clinical health status on admission and discharge in acute care hospitals.
Nursing interventions and patient outcome databases
The contribution of research-based nursing care interventions to improved patient health outcomes was established in a meta-analysis conducted more than 2 decades ago demonstrating that they led to improved patient outcomes.4 This research gave rise to increased efforts to measure patient health outcomes at the point of care to further evaluate nurse staffing and nursing care interventions. The establishment of standardized databases that include indicators to assess nurse-sensitive outcomes emerged in the early 1990s when the American Nurses Association developed the National Database of Nursing Quality Indicators (NDNQI).5
On the basis of preliminary work conducted by the American Nurses Association in a nursing report card aimed at identifying empirical links between nurse staffing and patient outcomes, 21 indicators were proposed for inclusion in the NDNQI.6,7 Pilot studies conducted in various settings across the United States evaluated the effectiveness of these initial indicators and resulted in a final set of 10 indicators that are currently in use.6–9 This database was designed to create a mechanism for monitoring the impact of nurse staffing on patient care quality and safety while providing individual hospitals with a quality improvement tool that allowed staffing and outcome comparisons across sites.10 Hospital participation in the NDNQI is voluntary. From a research perspective, the NDNQI produces national data comparisons that relate nurse staffing to the clinical patient outcomes of falls, pressure ulcers, pain assessment in pediatrics, intravenous infiltration in pediatrics, psychiatric patient assaults, restraint use, and health care–associated infections (ventilator-associated pneumonia, central line–associated bloodstream infections, catheter-associated urinary tract infections).11 In addition, the national database provides the individual participating sites with quarterly unit-level reports that can be used to compare performance in relation to patient outcomes. Data collection for the NDNQI began in 1998 with 1834 US hospitals currently participating.12
Further work in the United States led to the development of a national set of standardized performance indicators aimed at assessing nursing's contribution to the quality of health care, patient safety, and safe, professional work environments in acute care settings.13 The National Quality Forum (NQF) was developed in 1999 to promote patient and health care quality through publicly reported performance measurement. Fifteen indicators form the basis for measurement categorized into 3 areas: patient, nursing intervention, and system centered. There are 8 patient-centered indicators including death among surgical inpatients with treatable serious complications (failure to rescue), pressure ulcer prevalence, falls prevalence, falls with injury, restraint prevalence, urinary catheter–associated urinary tract infections in intensive care patients, central line catheter–associated bloodstream infection rate for intensive care and high-risk neonatal patients, and ventilator-associated pneumonia for these same patient groups. In addition, 3 nursing-centered intervention indicators are measured including smoking cessation counseling for acute myocardial infarction, heart failure, and pneumonia patients. The 4 system-oriented indicators are skill mix, nursing care hours per patient day, voluntary turnover, and an index of the nursing practice environment.13
Nine of the NDNQI indicators are endorsed for use in the NQF initiative, which uses a consensus approach that includes standard development and review before voting and approval of use of the measure.3 Endorsement of these NDNQI indicators by the NQF demonstrates that standardized measures for evaluating nursing care are being employed.3 The concordance of the nursing indicators between NDNQI and NQF has been reported in a comparative analysis of the types, content, and intended purpose of nursing indicators available in the United States,14 potentially making them a preferred data source for researchers and quality managers. A number of research initiatives are currently funded that support study of these indicators in exploring the relationships between nursing care and patient outcomes.2,15
Health outcomes for better information and care
In Canada, the Health Outcomes for Better Information and Care (HOBIC) database emerged following research initiated in 1999 aimed at identifying nurse-sensitive outcomes that could be abstracted from patient records and retained in an administrative database.16,17 An expert panel conducted consultations with stakeholders to determine which outcomes were considered relevant before commissioning an analysis of outcomes that related to nurses' contributions to patient care.16–19 A subsequent feasibility study provided evidence that the outcomes of functional status (activities of daily living [ADL]); symptom control (fatigue, nausea, dyspnea, and pain); therapeutic self-care (TSC); falls; and pressure ulcers could be collected reliably and were relevant to nursing practice.20,21 As a result, the expert panel identified these 8 nurse-sensitive outcomes for ongoing data collection, using existing reliable and valid measures.19,22
Provincial government support was received in 2006 to implement collection of data on this standardized set of indicators across the health sectors including 62 acute, 123 long-term, and 16 community home care settings17,19 to address the need for data to inform government and health care leaders in decision making related to patient outcomes across the spectrum of health care delivery. The HOBIC data are collected by nursing staff using outcome measures that have been embedded in existing online clinical documentation systems.19 The HOBIC data collection takes place on admission and discharge in all sectors with the exception of long-term care, where assessments are completed quarterly or when there is a significant change in the resident's condition.18 An implementation team worked with the health care settings to provide needed support with the process of data collection.19
Aims of the study
The development of large clinical databases provides the opportunity to identify the contribution of nurses and nursing care interventions to patient and health outcomes that can be attributed to nursing care. To examine these relationships, it is important to assess the sensitivity of these clinical data to the concepts being measured. This research describes the results of a study aimed at determining whether there are changes in health outcome scores from admission to discharge for medical and surgical patients in acute care settings that could be related to nursing interventions. The specific research question was: Are there changes in HOBIC outcome mean scores, specifically ADL, pain, fatigue, bladder continence, dypsnea, nausea, falls, pressure ulcers, and TSC, in acute care settings between admission and discharge?
This retrospective study included 59 157 adult acute-care hospital-based patient records from 44 hospitals in Ontario, Canada, between December 9, 2006, and March 31, 2010, who had assessment records in the HOBIC database. The HOBIC data are collected electronically on a set of standardized measures of patient status during the admission and discharge assessments conducted by nurses as part of their day-to-day care. Admission assessments are made within 48 hours of patient admission to capture the patient's status at the time of entry into the facility, whereas the final assessment is made at discharge. The HOBIC data are housed at the Institute for Clinical and Evaluative Sciences, a secure data warehouse that holds health information used solely for evaluation and research purposes. Data are available and anonymized with a unique identifier to protect individual privacy while enabling linkage across clinical administrative databases. The project received ethics approval from the Sunnybrook Health Sciences Research Ethics Board affiliated with the University of Toronto.
To avoid replication, the data and measures used to capture the health outcomes data for HOBIC come from existing clinical documentation instruments employed within health care facilities in Canada.19 Thus, for many of the acute care outcomes, interRAI Acute Care (AC) instrument systems form the basis for measurement in the HOBIC database. InterRAI instruments are derived from the Minimum Data Set Resident Assessment Instrument (RAI) originally used in nursing homes. InterRAI AC is a suite of data collection instruments that was developed through an international collaborative of researchers and clinicians aimed at providing standardized assessment and care planning for older patients admitted to acute hospitals (including medical and surgical wards) with conditions that may benefit from comprehensive assessment.23 Patient care assessment data are collected by care providers within 48 hours of admission, to capture the individual's pre–hospital admission status or status early in the hospital stay, and again shortly before discharge.23 InterRAI AC domains of care with an evidence base linked to nursing interventions formed the basis of the HOBIC measures including ADL, pain frequency, fatigue, bladder continence, dyspnea, falls, and pressure ulcers.
Activities of daily living self-performance assesses what the patient actually does within each activity category according to a performance-based scale. Self-performance is assessed from 0 to 6, with 0 indicating independence where little or no assistance is required and 6 referring to total dependence related for the activity. Individual assessments for each of 7 activities including bathing, personal hygiene, walking, toilet transfer, toilet use, bed mobility, and eating are collected in reference to a 24-hour period. Activities of daily living self-performance is calculated as a summary score ranging from 0 to 56, for the 7 individual ADL activities.
Pain frequency, an interRAI AC measure, captures the frequency with which the patient complained or showed evidence of pain in the past 24 hours on a 3-point scale ranging from 0 (no pain) to 2 (pain). In addition, a measure of pain intensity using an 11-point numeric visual analog scale was included.18 The scale ranges from 0 (no pain present) to 10 (the worst possible pain). Fatigue was assessed using the interRAI AC instrument, describing the gradations of fatigue or impaired stamina exhibited by the patient in the past 24 hours. The 4-point scale ranges from 0 (no fatigue present) to 4 (severe fatigue). Bladder continence assessed continence control over the previous 24 hours using the 5-point interRAI AC scale ranging from 0 (complete control) to 4 (incontinence). Dypsnea also was assessed using the 4-point interRAI AC instrument capturing data for the previous 24 hours, with 0 (absence of difficult breathing) and 3 (dyspnea while at rest).
As well, a measure of nausea was developed for use in HOBIC by the executive lead for the project. The nausea scale is scored similar to the ADL scales, with 0 (no nausea reported in the last 24 hours) and 4 (incapacitating nausea that causes the patient to remain in bed for part of the day interfering with eating and other ADL). Falls were assessed using the interRAI AC instrument, which is aimed at determining whether the person has a history of falling and thus is at risk of falling while in hospital. Falls data are captured on a scale ranging from 0 (no falls in the last 90 days) to 3 (2 or more falls in the last 30 days). Pressure ulcers are assessed using the interRAI AC measure that captures the most severe pressure ulcer that the patient has at admission and discharge, on a 5-point scale from 0 (no pressure ulcer) to 4 (breaks in the skin that expose muscle or bone).
Finally, a 13-item instrument (TSC) assessing aspects of self-care readiness at discharge was employed.21,24 The TSC scale assesses how much a patient is able to do for each care-related activity on a 5-point scale ranging from 0 (not at all) to 5 (very much). Self-care readiness activities capture information about patient knowledge and ability to manage their medications (3 items), changes in symptoms and health condition (4 items), general health knowledge (1 item), support and emergency contacts (2 items), and understanding of their ability to perform and/or adjust regular activities (2 items).
The reliability of the outcome assessment tools used in this study has been determined in previous research in 2 ways.25 First, interrater reliability was conducted to assess the extent to which results obtained by different raters using the same outcome assessment agree and tested using kappa statistics. The weighted kappa results assessing the interrater reliability demonstrated that there was good agreement between the different assessments for the items measuring the outcomes. The kappa scores ranged from 0.68 to 0.89 for the individual ADL, 0.83 for pain frequency, 0.93 for pain intensity, 0.80 for fatigue, 0.87 for bladder continence, 0.93 for dyspnea, 0.76 for nausea, 0.86 for falls, and 0.93 for pressure ulcers.25 Also, for the 2 psychometric scales used in the HOBIC outcomes assessment (ADL and TSC), internal consistency reliability was assessed. The Cronbach α coefficients of the ADL scale on admission was 0.88 and at discharge was 0.91; the TSC scale was 0.93 in previous research.25 Cronbach α for the ADL scale in this study was 0.94 and for the TSC scale was 0.97, indicating that the scales demonstrated excellent internal consistency reliability.
Admission and discharge scores for each patient were matched, means and standard deviations were summarized, and differences in mean scores were tested. Most scales contain more than 4 levels, and all were treated as continuous variables in the initial data analysis. Normal probability plots were used to test for normality with t tests, and Wilcoxon signed rank (nonparametric) tests were used to compare differences in admission and discharge scores for normal and nonnormally distributed outcomes, respectively. The admission and discharge scores for ADL, pain intensity, and TSC were not normally distributed, therefore the Wilcoxon test was used in this analysis. The average percent score change for each HOBIC measure was calculated (mean score at discharge minus the mean score at admission, divided by the mean score at admission), and results are reported in the Table. Because the large population might assure statistical significance, effect size (the standardized difference of the mean—how many standard deviations by which the 2 groups differed) is reported to capture the relative magnitude of changes in health status. Data analysis was conducted using SAS (SAS Institute Inc, Cary, NC) at the Institute for Clinical and Evaluative Sciences.
Table. Changes in HO...Image Tools
The patients' age ranged from 14 to 106 years, with a mean of 65.2 years (SD = 17.9), demonstrating the majority of patients were older. Few young teenage patients were found in the database. Slightly more women (51.7%) than men (48.3%) comprised the sample. Patients in this sample were housed primarily in community hospitals not affiliated with academic health sciences settings.
Changes in health outcomes
The changes to admission and discharge mean scores on the HOBIC outcome measures as well as the magnitude and significance of changes over the course of the hospital stay are shown in the Table.
Activities of daily living
At time 1 (within 24 hours of admission), patients had a mean score of 5.84 (SD = 9.62), indicating that they had a high need for assistance with ADL. This suggests that maximal assistance was required, such as weight bearing support with more than 1 helper or weight bearing support for more than half of the ADL being performed. Patients required the most assistance with bathing, transferring on and off the toilet, walking, general toileting, and personal hygiene. They needed less assistance with mobility while in bed and eating. By discharge, patients showed a statistically significant improvement in their ability to perform all of these ADL, with the mean score decreasing to 4.39 (SD = 8.53), although still needing extensive assistance requiring 1 helper.
Pain frequency and intensity
At admission, patients had a mean score of 1.06 (SD = 0.95) indicating that symptoms of pain were present and being exhibited to the nurse. By discharge, a statistically significant change was noted, with pain symptoms no longer being present and visible (see the Table). In addition, statistically significant changes in the intensity of pain that patients reported to nurses were noted between admission and discharge, with the observed mean scores decreasing over time from 3.39 (SD = 3.74) to 1.69 (SD = 1.69).
Overall, patients in this study had some minimal fatigue on admission, reporting diminished energy but retaining the ability to complete normal day-to-day activities. On discharge, significant improvements were noted with patients reporting little or no fatigue symptoms (see the Table).
Bladder continence, dyspnea, and nausea
For several of the health outcomes studied, patients were not assessed as experiencing the symptom either at admission or discharge. Despite this, the mean scores showed a change, indicating that an improvement in the outcome took place over the duration of the hospitalization (ie, bladder continence, dyspnea, nausea). This suggests that the measure was capturing some degree of change in patient conditions.
Falls and pressure ulcers
On admission, patients in this study did not report having a fall in the 90 days prior to admission, and no falls were reported at discharge. In addition, patients did not have any reported pressure ulcers either on admission or discharge. The HOBIC scores for pressure ulcers and falls are not reported, as their incidence was low.
The mean score on TSC showed a small but statistically significant change from 4.17 (SD = 1.26) on admission to 4.32 (SD = 1.21) on discharge as reported in the Table. This suggests that patients' ability and knowledge needed to manage their own health, symptoms, and prescribed health plan demonstrated improvement during their time in the hospital.
Magnitude of change in health outcomes
Several of the patient health outcomes examined in this study showed substantial differences in the effect sizes when mean scores from admission to discharge were compared. This may indicate that nursing care interventions are one of the factors contributing to the desired effect on clinical outcomes. Specifically, the effect sizes for pain intensity and frequency were moderately high at 0.52 and 0.41, respectively, suggesting that nursing management of pain symptoms through interventions including effective pain medication management and patient education may have contributed to an improvement in these outcomes from the time of admission to discharge. This is consistent with evidence reported in a recent quantitative systematic review of literature that identifies the importance of patient education in promoting effective self-care management of pain.26
A moderately high effect size (0.46) was also found for nausea, whereas moderate effect sizes were noted for fatigue (0.36) and dyspnea (0.36) outcomes for patients in this study, indicating that nurses' assessment and treatment of these symptoms may be having a moderately positive impact on improving symptoms prior to discharge. Clinical trials employing behavioral nursing interventions that include problem-solving approaches for symptom management and improved physical functioning targeted to patients with pain and fatigue have been found to reduce symptom burden such as fatigue levels, improve patient quality of the life, and demonstrate the “value-added” role of nursing care.27,28
Smaller effect sizes were noted for the outcomes of ADL (0.16), bladder continence (0.11), and TSC (0.12), suggesting that although nursing interventions had some impact on each of the outcomes, the magnitude of that effect was not as strong as for the other patient health outcomes reported earlier.
Acute care patients in this study experienced improved ADL; decreased pain frequency and intensity; a decline in symptoms of fatigue, dyspnea, and nausea; and improved bladder continence. In addition, patients' ability to manage their health and health care improved by the time they were discharged. These findings are similar to research evidence that links these health outcomes to nursing care and nursing interventions.7–9,11,18,25–28 No change in pressure ulcers was noted among patients in this study, which may reflect the characteristics of the acute care population examined in these data. In addition, the patients in this study had not experienced a fall within the 3 months leading up to their admission nor while hospitalized. This may indicate that the interRAI falls measure used in the HOBIC is not sensitive for patients in these settings as it does not appear to relate to treatment or observable health status within the acute care hospitalization.
Nursing interventions play a central role in acute care hospitals in the management of patient symptom control, the performance of ADL, and self-care ability at discharge. Improvements in ADL and pain control are recognized care achievements that result from nursing care interventions, highlighting the intended effects of the process of nursing care.24 On the basis of the findings in this study, interventions to improve symptom control and the performance of ADL and self-care ability at discharge in patients in acute-care settings could be hypothesized to improve both patients' quality of care and readiness for discharge. These specific nursing interventions also could improve patient overall length of stay and potentially reduce the costs of health care.
Sensitivity of HOBIC outcomes
The sensitivity of the health outcome measures to change was evaluated by comparing the mean scores obtained at 2 distinct points in time. The results of this study indicate that the HOBIC outcomes detect change in patient outcomes over time, specifically from admission to discharge. A moderate effect size was identified for the outcomes of pain intensity, pain frequency, and nausea, whereas a moderate to small effect size was noted for the remainder of outcomes observed in this study (ie, ADL, fatigue, dyspnea, TSC, and bladder continence). Similar patterns of change were reported in a pilot study using several of these outcomes,20 providing further evidence that with the exception of falls, these instruments are sensitive to change in outcome status in acute care patients. Patients in this sample from within acute care hospitals were older, further validating the interRAI AC as an appropriate measure of health outcomes for this population.
Standardized health outcome indicators offer a mechanism for evaluating both the effectiveness and quality of care provided in health care settings. These data provide the opportunity to determine the significance of nursing interventions and the care provided by different groups of nursing care providers on patients' health and health status. Clinical leaders involved with the HOBIC data collection in Canada describe a number of organizational benefits attained from having these data.29 These include the opportunity for data comparability from one admission to another and from admission to discharge, as well as having access to a standardized admission database for review of the effectiveness of nursing interventions used in their organizations.29 Having access to clinical databases also provides staff nurses with the ability to assess how a patient's status has improved over their stay and managers with information relating patient needs to nurse staffing.29
In an era of health care accountability and a rapidly changing health care environment, the need to demonstrate quality care is of central importance to health care administrators. Mechanisms that provide readily available information about patient care quality in terms of the process and outcomes of the delivery system are essential components of effective health care. These data highlight the achievements that have been evidenced from the implementation of a health outcomes database in Canada. Future studies should explore direct linkages between nursing resources and nursing care interventions and health outcomes across multiple health care sectors.
1. American Nurses Association. Improve Patient Safety and Quality Globally. The National Database of Nursing Quality Indicators™ (NDNQI). Washington, DC: American Nurses Association; 2010.
2. Brown DS, Donaldson N, Burnes Bolton L, Aydin CE. Nursing-sensitive benchmarks for hospitals to gauge high-reliability performance. J Healthc Qual. 2010;32(6): 9–17.
4. Heater BS, Becker AM, Olson RK. Nursing interventions and patient outcomes: a meta-analysis of studies. Nurs Res. 1988;37(5):303–307.
5. McGillis Hall L. Report cards: relevance for nursing and patient care safety. Int Nurs Rev. 2002;49:168–177.
6. American Nurses Association. Nursing Care Report Card for Acute Care. Washington, DC: American Nurses Association; 1995.
7. American Nurses Association. Implementing Nursing's Report Card: A Study of RN Staffing, Length of Stay and Patient Outcomes. Washington, DC: American Nurses Association; 1997.
8. American Nurses Association. Nurse Staffing and Patient Outcomes in the Inpatient Hospital Setting. Washington, DC: American Nurses Association; 2000.
9. Gallagher RM, Rowell PA. Claiming the future of nursing through nursing-sensitive quality indicators. Nurs Adm Q. 2003;27:273–284.
10. Dunton N, Gajewski B, Taunton RL, Moore J. Nurse staffing and patient falls on acute care hospital units. Nurs Outlook. 2004;52(1):53–59.
13. National Quality Forum. National Voluntary Consensus Standards for Nursing Sensitive Care: An Initial Performance Measure Set. Washington, DC: National Quality Forum; 2004.
14. Savitz L, Jones CB, Bernard S. Quality indicators sensitive to nurse staffing in acute care settings. In: Hendriksen K, Battles JB, Marks ES, Lewin DI. Advan-ces in Patient Safety: From Research to Implementation. 2005. http://www.ncbi.nlm.nih.gov/books/NBK20594/
. Accessed May 12, 2012.
15. Kurtzman ET, Corrigan JM. Measuring the contribution of nursing to quality, patient safety, and health care outcomes. Policy Polit Nurs Pract. 2007;8(1):20–36.
16. Pringle D, White P. Nursing and Health Outcomes Project. Phase 1 report. Toronto, Ontario, Canada: Ministry of Health and Long-Term Care; 2002.
17. White P, Pringle D, Doran D, McGillis Hall L. Accountability: the nursing and health outcomes project. Can Nurse. 2005;101(9):15–18.
18. Doran D. Nursing Sensitive Outcomes: State of the Science. Sudbury, MA: Jones & Bartlett; 2003.
19. Nagle LM, White P, Pringle D. Realizing the benefits of standardized measures of clinical outcomes. Electronic Healthcare. 2010;9(2):e3–e9.
20. Doran D, Harrison MB, Laschinger H, et al. Collecting Data on Nursing-sensitive Outcomes in Different Care Settings: Can it be Done? What are the benefits? Final report. Toronto, Ontario, Canada: Ministry of Health and Long-Term Care; 2004.
21. Doran D, Harrison MB, Laschinger H, et al. Nursing-sensitive outcomes data collection in acute care and long-term care settings. Nurs Res. 2006;55(2S):S75–S81.
22. Hannah KJ, White PA, Nagle LM, Pringle DM. Standardizing nursing information in Canada for inclusion in electronic health records: C-HOBIC. J Am Med Inform Assoc. 2009;16(4):524–530.
23. Hirdes JP, Ljunggren G, Morris JN, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an intergrated health information system. BMC Health Serv Res. 2008;8:277.
24. Doran D, Sidani S, Keatings M, Doidge D. An empirical test of the nursing role effectiveness model. J Adv Nurs. 2002;38(1):29–39.
25. Doran D, Harrison MB, Laschinger H, et al. An Evaluation of the Feasibility of Instituting Data Collection of Nursing-Sensitive Outcomes in Acute Care, Long-Term Care, Complex Continuing Care and Home Care. Technical report. Toronto, Ontario, Canada: Ministry of Health and Long-Term Care; 2004.
26. Ling CC, Lui LYY, So WKW. Do educational interventions improve cancer patients' quality of life and reduce pain intensity? Quantitative systematic review. J Adv Nurs. 2011;68(3):511–520.
27. Given B, Given CW, McCorkle R, et al. Pain and fatigue management: results of a nursing randomized clinical trial. Oncol Nurs Forum. 2002;29(6):949–956.
28. Borneman T, Koczywas M, Sun V, Piper BF, Uman G, Ferrell B. Reducing patient barriers to pain and fatigue management. J Pain Symptom Manage. 2010;39(3):486–501.
29. McGillis Hall L, Johnson S, Hemingway A, Pringle D, White P, Wodchis W. “The potential is unlimited!” Nurse leader perspectives on the integration of HOBIC in Ontario. Nurs Leadersh (Tor Ont). 2012;25(1):29–42.
clinical outcomes; databases; nurse-sensitive outcomes; outcomes assessment
© 2013 Lippincott Williams & Wilkins, Inc.