Hickey, Patricia A. PhD, MBA, RN, NEA-BC, FAAN; Gauvreau, Kimberlee ScD; Curley, Martha A. Q. PhD, RN, FAAN; Connor, Jean A. PhD, RN, CPNP
RNs remain the largest group of healthcare providers and typically account for the greatest expense in hospital operating budgets. The impact of nursing care on patient outcomes has been well established in adult hospitals. Specifically, the nursing and organizational characteristics of baccalaureate (BSN) education, increased RN staffing, healthy work environments, and Magnet® recognition are associated with improved patient outcomes in these hospitals.1-9
Less is known about the pediatric nursing workforce’s influence and impact on patient outcomes.10-13 The costs of patient services provided to children are substantially higher than those provided to adult patients in general hospitals. In a classic study by the Center for Health Policy Studies, differences were greatest in the provision of nursing care and respiratory therapy.14 The differences in costs continue to be explained today, by the inability of children to independently perform activities of daily living and by their limited developmental capacity to understand and communicate their illness and follow treatment instructions. Infants younger than 1 year and children with chronic and congenital diseases have also been found to have higher costs of nursing care.15,16
In pediatrics, congenital heart disease is the most commonly occurring birth defect requiring surgical intervention for survival. These patients also consume a disproportionate share of hospital resources because of their critical care needs. The purpose of this study was to explore pediatric critical care nursing and organizational characteristics that impact in-hospital mortality for cardiac surgery patients across children’s hospitals in the United States. Specifically, what nursing and organizational characteristics provide a protective effect on pediatric mortality for congenital heart surgery patients?
Donabedian’s17 structure-process-outcome (SPO) model of healthcare quality proposes that the assessment of outcomes requires the examination of relationships among all 3 components (SPO) for a comprehensive approach to quality improvement and provision of care. This model was used to examine the interrelationships among the structure and process variables on the patient outcome of in-hospital mortality. According to Donabedian’s17 framework, good organizational structures lead to improved processes, which lead to better outcomes.
For our purposes, the structure variables in the hospital’s critical care units (CCUs) included number of RNs, their education level, full- and part-time status, clinical experience, CCU experience, specialty certification, and indirect resources. Processes of care included nursing leadership roles, physician coverage, clinical and administrative support roles, standardized communication, procedural checklists, medication safety, simulation training, clinical practice guidelines, quality metrics, physical characteristics of the CCU, and organizational awards of national prominence including Magnet from the American Nurses Credentialing Center® (ANCC) and Beacon from the American Association of Critical Care Nurses® (AACN).
Approval was granted by the internal review board of Boston Children’s Hospital to conduct a primary analysis on new nursing and organizational characteristics data and a secondary analysis of existing data in the Pediatric Health Information System (PHIS) database for calendar years 2009 and 2010.
Nursing and Organizational Characteristics Data Sources
Senior nursing leaders from children’s hospitals with a cardiac surgery volume of at least 50 cases over 2 years and who also contributed data to the PHIS data set were invited to complete an electronic organizational assessment survey. All 38 centers meeting these criteria responded. To develop the survey content, the lead researcher led a series of conference calls with cardiovascular nurses and scientists from the children’s hospitals to solicit their expert opinions about nursing and organizational characteristics considered important to improving outcomes for patients with congenital heart surgery. More detailed survey methodology and summary analysis have been previously described.9 Nine domains of nursing were examined in the survey and included RN staffing; nursing education; specialty certification; overall clinical experience; CCU experience; indirect resources provided for nursing support including clinical coordinator, clinical nurse specialist (CNS), nurse educator, dedicated charge nurse, nurse director, nurse manager, nurse scientist, physician assistant, nurse practitioner, nurse resource, and physician roles; and hours of coverage. Processes of care included standardized communication, procedural checklists, unit-based medication safety, simulation training, clinical practice guidelines, quality metrics, and nurse-developed algorithms. Indicators of organizational excellence were recognized as the ANCC Magnet designation and the AACN Beacon award for excellence. The physical characteristics of the CCUs that were analyzed included whether these patients were cared for in a dedicated cardiac CCU, or a general pediatric CCU. The hospital data on each site’s survey were assigned a unique identification number. Nursing and organizational characteristics for the institutions were linked to patient-level data by hospital name and location.
Patient-level data were obtained from the PHIS administrative database for calendar years 2009-2010. The PHIS data set is a large, detailed clinical and financial database managed by the Performance Improvement Division of the Child Health Corporation of America, which collects information on 43 participating freestanding children’s hospitals in the United States. Thirty-eight of these hospitals were included in this study. More than 125 data elements are collected for each patient admission, including demographic information, admission and discharge dates, patient outcomes, and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes for all diagnoses and procedures. Institutions are labeled within the database but cannot be identified in public reporting; individual patient medical record numbers and identification codes are encrypted. The PHIS database was also used to calculate the average annual number of congenital heart surgical procedures performed at each institution for patients younger than 18 years.
The Risk Adjustment for Congenital Heart Surgery (RACHS-1) method was used to adjust for baseline differences in case mix complexity while evaluating associations between nursing and organizational characteristics and in-hospital mortality.18,19 To apply this method, cases of congenital heart surgery were assigned to 1 of 6 predefined risk categories based on the presence or absence of specific ICD-9-CM diagnosis and procedure codes. Risk category 1 has the lowest risk for in-hospital mortality and risk category 6 the highest. Cases with a combination of cardiac surgical procedures during the same admission were placed in the category corresponding to the single highest-risk procedure. Additional clinical variables included in the risk adjustment model are age at surgery, prematurity, presence of a major noncardiac structural anomaly, and combinations of cardiac surgical procedures.
Nursing and organizational data are summarized (Tables 1 and 2). Patient demographic information and clinical information are also summarized for cases of congenital heart surgery able to be assigned to a RACHS-1 risk category (Table 3). Associations between pediatric critical care nursing and organizational characteristics measured at the institution level and the patient outcome in-hospital mortality were evaluated using generalized estimating equations models (Table 5). These models account for the intra-institutional correlation among cases from the same hospital. Nursing characteristics were examined 1 at a time, and each model included all factors that are part of the RACHS-1 risk adjustment method; statistical significance was assessed using the likelihood ratio test (Table 4). Odds ratios (ORs) were estimated with 95% confidence intervals. For continuous characteristics such as percentage of RNs with 2 years of clinical experience or less within the hospital, ORs are expressed as the odds associated with a 10% increase (eg, from 20% to 30% of RNs having <2 years of clinical experience). Nursing and organizational characteristics that were significantly associated with in-hospital mortality at the 0.10 level after risk adjustment were considered for inclusion in a multivariable model. Average annual institutional volume of congenital heart surgical procedures was added to the final model to determine whether nursing characteristics remained significant after adjusting for volume.
For the 38 children’s hospitals represented in this analysis, survey responses included information on 3413 pediatric critical care nurses. A total of 26158 cases of congenital heart surgery were identified. Within this group, 20407 cases were both eligible for RACHS-1 risk adjustment and able to be assigned to a risk category.
Nursing characteristics (Table 1) included highest level of education, years of clinical experience, years of any CCU experience, and professional certification. Seventy-one percent of the nurses held a baccalaureate degree in nursing (BSN). The distribution of nursing experience included 52% of the nurses having less than or equal to 5 years of nursing experience and 48% of the nurses with more than 5 years. Sixty-two percent of the nurses had at most 5 years of nursing experience. Fifteen percent of the pediatric nurses were professionally certified in critical care (13%) or pediatric nursing (2%).
For the 38 institutions included in this analysis, 37 (97%) were academic teaching hospitals (Table 2). The median number of staffed beds was 288 (range, 118-512). Twenty-three of the 38 hospitals (61%) had achieved Magnet recognition, and 25 (66%) had a dedicated cardiac CCU. The nursing leadership models included a combination of directors, nurse managers, and clinical coordinators. Eighty-four percent of the units had a dedicated charge nurse. The median number of RN full-time equivalents in these ICUs was 68.4 (range, 23.3-150). Procedural checklists were present in 100% of the units. Given the nature and intensity of the CCU environment, checklists included a variety of standardized tools used before invasive procedures on patients including extracorporeal membrane oxygenation cannulation, chest tube insertion, and discontinuation of intracardiac line procedures, to name a few. Standardized communication was evident in 37 of the units (97%). A common form of standardized communication includes SBAR (situation-background-assessment-recommendation)20,21 as a method for clinicians to convey important patient care information using critical language.
Among the 38 study institutions, 26158 cases of congenital heart surgery were identified (Table 3). Within this group, of the 20407 eligible cases, 59% were younger than 1 year, and 55% were reported to be male. Prematurity was coded in 5.8% of patients, and 8.4% of the cohort had an additional major noncardiac anomaly. The RACHS-1 risk categories 2 and 3 accounted for 72.6% of cases, and 24.1% were noted to have multiple surgical procedures during the hospital admission. The average in-hospital mortality for the national cohort was 2.7% (range, 0.0%-5.1%).
The RACHS-1 risk adjustment model for this cohort is shown in Table 4. The area under the receiver operating characteristic curve for this model is 0.81, indicating excellent discrimination between patients who died in-hospital and those who did not. Table 5 reports the relationships between the nursing and organizational variables and odds of in-hospital death for each variable separately. After adjusting for baseline patient risk, a significant increase in the odds of in-hospital mortality was noted as the institutional percentage of pediatric ICU (PICU) nurses with 2 years of clinical experience or less increased (OR, 1.12 for each 10% increase in percentage of nurses with ≤2 years of experience; P < 0.001) (Table 5). Conversely, the odds of mortality decreased as the institutional percentage of nurses with 11 years’ clinical experience or more increased (OR, 0.89 for each 10% increase; P = 0.04), and as the percentage of nurses with 16 years of clinical experience or more increased (OR, 0.82 for each 10% increase; P = 0.006). A similar pattern was detected for years of unit experience, although these relationships were only borderline significant (≤2 years, P = 0.06; ≥11 years, P = 0.11; ≥16 years, P = 0.06).
The odds of mortality significantly decreased as the institutional percentage of nurses with a BSN or higher increased (OR, 0.91 for each 10% increase in percentage of nurses with a BSN or higher; P = 0.02).
Among 6 process of care indicators examined, the only 1 found to be significantly associated with decreased odds of in-hospital mortality was participation of the unit in national quality metric benchmarking (OR, 0.61; P < 0.001). Relationships with having a dedicated cardiac CCU and Magnet recognition were not statistically significant.
In multivariable analysis, after adjusting for institutional percentage of PICU nurses with 2 years of clinical experience or less and baseline patient risk, no other nursing characteristics remained statistically significant. There was still a trend for lower in-hospital mortality in units participating in national quality metric benchmarking, but this association did not reach statistical significance (P = 0.08). However, a higher percentage of PICU nurses with 2 years of clinical experience or less remained strongly associated with greater odds of in-hospital mortality even after adjusting for average annual institutional volume of congenital heart surgeries. We investigated whether there was a cut point associated with higher odds of mortality and found that when 20% of PICU nurses or more in an institution’s CCU had 2 years of clinical experience or less, the odds of in-hospital mortality are increased (OR, 1.30; P = 0.05). The effect is even greater when 25% of nurses or more have 2 years of clinical experience or less (OR, 1.52; P < 0.001) (Table 5). Data included in Tables, Supplemental Digital Content 1, http://links.lww.com/JONA/A274, Supplemental Digital Content 2, http://links.lww.com/JONA/A275, Supplemental Digital Content 3, http://links.lww.com/JONA/A276, Supplemental Digital Content 4, http://links.lww.com/JONA/A277, Supplemental Digital Content 5, http://links.lww.com/JONA/A278, and Supplemental Digital Content 6, http://links.lww.com/JONA/A279, illustrate other nursing and organizational characteristics.
This study describes new data supporting the relationships between pediatric critical care nursing and organizational characteristics and mortality. Years of nursing experience was significantly associated with improved patient outcomes. In these CCUs, a cut point of 20% appears to be the critical value at which the percentage of RNs with 2 years of clinical experience or less significantly increases the odds of death; when more than 25% of nurses have 2 years of experience or less, the odds of in-hospital mortality increase by a factor of 1.52 (Table 5). Nursing education was also associated with decreased odds of death for children undergoing cardiac surgery. Pediatric CCUs that subscribed to national quality metric benchmarking organizations were associated with improved outcomes. Overall, significant variation existed in structures and processes to support CCU nurses, and there were broad differences in mortality rates with a range of 0.0% to 5.1% for congenital heart surgery patients across the freestanding children’s hospitals.
Although it has been speculated that nursing experience does influence outcomes, this is the 1st known study to demonstrate that it actually is significant among a pediatric nursing population. Interestingly, years of CCU experience alone was not a predictor of mortality, but total clinical experience was a significant factor. This finding may reinforce the belief that experiential knowledge and skill acquisition are cumulative over time regardless of the clinical setting, as professional nurses advance their practice.22 A higher percentage of nurses with 2 years of experience or less was associated with higher odds of mortality. Cut points of 20% and higher achieved statistical significance; however, the cut point with the highest OR was 25%. Reciprocally, increased percentages of nurses with at least 11 years’ experience and more with at least 16 years of experience were associated with improved mortality outcomes. The nursing experience finding has important implications for hospital executives and policy makers as work environments are reengineered to transform care delivery. Hiring new-graduate nurses and nurses with 2 years of experience or less is necessary to provide an adequate supply of CCU nurses for the future. At the same time, these nurses must be supported during orientation and through the 1st couple of years of their practice to ensure their professional success as well as optimize outcomes for patients and families.
Findings from this study provide hospital leaders with data to justify nurse staffing and retention including a balance of new and experienced nurses to optimize patient outcomes and staff satisfaction. Appropriate nursing residency and mentorship programs have been identified as effective in supporting new-graduate nurses as they transition to practice.23 Based on the findings of this study, these programs are needed for the 1st 2 years of experience. Our data support the 2010 Institute of Medicine (IOM) report on The Future of Nursing Leading Change, Advancing Health recommendation that residency programs be implemented to support new graduates.24 Nursing experience is essential because nurses provide both high-quality patient care and teach and support those learning the intricacies of complex care, such as pediatric cardiovascular nursing. These data should be considered to serve as a guide for nurse leaders when evaluating experience and expertise in nurse staffing models.
Another important finding from the study is that organizations would benefit from focusing on retention strategies for experienced nurses. As every hospital is engaged in cost containment efforts, retention strategies for experienced nurses must be continued and reprioritized. This study demonstrates that experienced nurses contribute significantly to improved patient mortality.
The study findings support other research studies in adult hospitals demonstrating the association of at least a BSN and improved adult patient outcomes including mortality.25,26 This is the 1st known study to extend those findings to pediatric nursing. The formal education associated with obtaining the BSN is imperative for building the practice and science of nursing in the future. Care provided in hospitals will grow more complex, and nurses must engage in critical thinking and complex information management that will require skills in analysis and synthesis such as provided in BSN education programs to meet the increasing acuities of patients and the support needs of their families.13 These data also support the 2010 Institute of Medicine report, recommendation that 80% of practicing nurses earn a BSN degree or higher by 2020. The evidence in this study that 70% of the pediatric RNs held a BSN was encouraging and revealed that children’s hospitals in general have made significant progress toward actualizing this IOM recommendation.24
The odds of death were significantly decreased in CCUs that contributed data to national quality metric benchmarking programs, reinforcing the belief that participation in national registries such as the American Nurses Association’s National Database of Nursing Quality Indicators NDNQI® has a positive impact on patient outcomes. Organized quality improvement activities enable a national community of nursing professionals to benchmark, learn best practice, and implement these practices in their home institutions.
A number of the variables including nursing leadership, advanced practice nurses, attending physicians, and Magnet recognition that are often thought to be important for optimal patient outcomes were not found to be significant in this study. Although there were various nursing leadership models in operation, each of the units had either a nurse director, nurse manager, or both and either a CNS, nurse educator, or both. A common debate among specialists is whether pediatric cardiac surgery patients should be cared for in a dedicated cardiac CCU or a general pediatric CCU. In this study, there was no difference in mortality outcomes. Consistent with previously published data, there was a volume outcome relationship in this study.12,13 Patients cared for in higher-volume centers experienced lower odds of mortality (Table 4).
In adult hospitals, Magnet recognition has been associated with mixed results related to patient outcomes.26,27 This research was conducted solely in children’s hospitals, and there were no differences in mortality between Magnet and non-Magnet hospitals. However, a disproportionately high number of children’s hospitals have received Magnet recognition compared with adult hospitals.28 More than half of the children’s hospitals (24 hospitals) already had Magnet recognition, and the majority of others (63%) were on the journey to Magnet. Hence, there was not enough heterogeneity in Magnet status in our sample of 38 children’s hospitals to detect a difference.
The professional nursing credential of certified critical care nurse (CCRN) was aligned with decreased odds of death, which is clinically important for the profession. Many organizations have embraced the need for nursing certification because of patients’ and families’ expectations of certification and the education, knowledge, and experience required to achieve and maintain professional certification. The AACN awards CCRN certifications to nurses who have worked in critical care for at least 2 years and have successfully passed a rigorous examination and continue to demonstrate ongoing education and maintenance of critical care knowledge and skills.
Results of this study should be interpreted with consideration of limitations. This research was focused on pediatric critical care nurses and did not include nurses caring for pediatric cardiac surgery patients on the general ward, cardiac catheterization laboratory, operating room, or clinic. The PHIS database provides detailed demographic, clinical, and billing information from freestanding children’s hospitals, and these data may not be generalizable to other nonpediatric institutions. Although trained individuals oversee data collection and input, consistent with the use of secondary data, there may be missing data and coding errors that can occur in administrative databases. Although many nursing and organizational variables were included, there are dimensions of the work environment that were not examined.
The study findings revealed for the 1st time that higher levels of experience and education in the pediatric critical care nursing workforce are associated with fewer patient deaths. The identification of a cut point of 20% or less when nursing inexperience is associated with increased mortality may now be considered as nurse leaders, hospital executives, and policy makers continue the staffing conversation in these fiscally challenging times. In addition, CCUs that contribute data to national quality metric registries are associated with reduced mortality. Future research is required to determine other pediatric nursing and organizational factors that may have a protective effect on mortality for critically ill patients. Although this study involves secondary analysis of a large database with broad representation supporting generalizability to other academic institutions, replication of this research in adult hospitals may be warranted to determine generalizability in general hospitals and other patient populations.
1. Aiken LH, Smith HL, Lake ET. Lower Medicare mortality among a set of hospitals known for good nursing care. Med Care. 1994; 32: 771–787.
2. Aiken LH, Clarke SP, Sloane DM, et al. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002; 288: 1987–1993.
3. Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organizational support and quality of care: cross-national findings. Int J Qual Health Care. 2002; 14: 5–13.
4. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002; 346 (22): 1715–1722.
5. Aiken LH, Clarke SP, Sloane DM, et al. Effects of hospital care environment on patient mortality and nurse outcomes. J Nurs Admin. 2008; 38: 223–229.
6. Kutney-Lee A, McHugh MD, Sloane DM, et al. Nursing: a key to patient satisfaction. Health Aff (Millwood). 2009; 28: w669–w677.
7. Duffield C, Diers D, O’Brien-Pallas L, et al. Nursing staffing, nursing workload, the work environment and patient outcomes. Appl Nurs Res. 2010; 24: 244–255.
8. Aiken LH, Cimiotti JP, Sloane DM, et al. Effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Med Care. 2011; 49: 1047–1053.
9. Kelly L, McHugh M, Aiken L. Nurse outcomes in Magnet® and non-Magnet® hospitals. J Nurs Adm. 2011; 41: 428–433.
10. Mark B, Harless D, Berman W. Nurse staffing and adverse events in hospitalized children. Policy Polit Nurs Pract. 2007; 8: 83–92.
11. Hickey P, Gauvreau K, Connor J, Sporing E, Jenkins K. The relationship of nurse staffing, skill mix, and magnet recognition to institutional volume and mortality for congenital heart surgery. J Nurs Adm. 2010; 40 (5): 226–232.
12. Hickey P, Gauvreau K, Jenkins K, Fawcett J, Hayman L. Statewide and national impact of California’s staffing law on pediatric cardiac surgery outcomes. J Nurs Adm. 2011; 41 (5): 218–225.
13. Hickey P, Gauvreau K, Tong E, Schiffer N, Connor J. Who are we: nursing and organizational characteristics of pediatric cardiovascular critical care in the United States. Am J Crit Care. 2012; 21 (4): 242–250.
14. Pediatric Costing Study: Final Report. Columbia, MD: Center for Health Policy Studies; 1993.
15. Law SP, Kim JJ, Decker JA, et al. Hospital charges for pediatric heart transplant hospitalizations in the United States from 1997 to 2006. J Heart Lung Transplant. 2012; 31 (5): 485–491.
16. McHugh MD, Kelly LA, Smith HL, Wu ES, Vanak JM, Aiken LH. Lower mortality in Magnet hospitals. Med Care. 2013; 51 (5): 382–388.
17. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966; 44 (suppl 3): 166–206.
18. Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg. 2002; 123: 110–118.
19. Jenkins KJ, Gauvreau K. Center-specific differences in mortality: preliminary analyses using risk adjustment in congenital heart surgery (RACHS-1) method. J Thorac Cardiovasc Surg. 2002; 124: 97–104.
20. Kosnik LK, Brown J, Maund T. Patient safety; learning from the aviation industry. Nurs Manage. 2007; 38 (1): 25–30.
21. Hickey P, Connor JA, Trainor B, et al. Implementation of an organization-wide standardized communication initiative. J Commun Healthc. 2012; 5: 32–39.
22. Benner P. From Novice to Expert: Excellence and Power in Clinical Nursing Practice. Menlo Park, CA: Addison-Wesley Publishing Company; 1984.
23. Benner P. Educating Nurses: A Call for Radical Transformation. San Francisco, CA: Jossey-Bass Inc.; 2010.
24. Institute of Medicine Report. The Future of Nursing Leading Change, Advancing Health. Washington, DC: National Academies Press; 2010.
25. Aiken LH, Cimiotti J, Sloane DM, Smith HL, Flynn L, Neff D. The effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Med Care. 2011; 49: 1047–1053.
26. Goode C, Blegan M, Hye Park S, Vaughn T, Spetz J. Comparison of patient outcomes in Magnet® and non-Magnet hospitals. J Nursing Adm. 2011; 41 (12): 517–523.
27. Kelly LA, McHugh MD, Aiken LH. Nurse outcomes in Magnet® and non-Magnet hospitals. J Nurs Adm. 2011; 41: 428–433.