Beckel, Jean DNP, RN, MPH, CNML; Wolf, Gail PhD, RN, FAAN; Wilson, Roxanne PhD, RN; Hoolahan, Susan MSN, RN, NEA-BC
In 2002, hospitals began collecting data to demonstrate performance related to The Joint Commission’s core measures.1 This initiated many healthcare organizations to external data reporting and benchmarking. In 2005, the Centers for Medicare & Medicaid Services launched the “pay for performance” concept2 based on the Institute of Medicine report recommending safe, effective, patient-centered, timely, efficient, and equitable care for every patient.3 These initiatives contributed to an evolution toward required healthcare outcome trending and reporting.
The American Nurses Credentialing Center’s® (ANCC’s®) Magnet® designation, recognizes healthcare organizations for excellence in nursing services. In 2008, ANCC sponsored nursing research identified a new Magnet model. Implementation of the model transitioned the Magnet Recognition Program® from a focus on structures and processes to a focus on infrastructure and outcomes denoted as essential to a culture of excellence and innovation.4 Hospitals pursuing Magnet designation have experienced challenges in successfully demonstrating outcomes associated with provision of nursing care. It is reported that Magnet applications not scoring in the defined range of excellence required for a site visit have identified deficiencies related to the empirical outcome sources of evidence, especially related to demonstration of nurse-sensitive outcomes. The purpose of this study was to determine if differences exist in chief nursing officer (CNO), Magnet program director (MPD), nurse leader (NL), direct care RN (DC RN), and other perspectives of potential barriers to successfully demonstrating nurse-sensitive outcomes.
In 1855, Florence Nightingale conducted the 1st nursing outcomes research in a Scutari, Turkey, military hospital. Her documentation of nursing interventions demonstrated reduction of British soldier deaths from unsanitary and unsafe conditions.5 Outcome measurement and application of findings have challenged nursing since that time. Donabedian’s6 seminal article introduced concepts of structure, process, and outcomes, still used to evaluate healthcare today. His work acknowledged outcome assessment limitations, but held outcomes as the ultimate validators of medical care effectiveness and quality.6
Most hospitals use quality improvement performance measures; however, the degree and sophistication of use vary.7 The ability to link outcome achievement to specific nursing interventions is essential for determining appropriateness of care and identifying which nursing interventions are associated with patient outcomes.8 Nurses report perception of outcome collection as beneficial to practice.9 Recommendations consistently emphasize the need to develop standardized nursing outcome reporting formats with supporting data systems.7,9-12 This recommendation is bolstered by today’s increasing burden and complexity of data collection in healthcare settings.
Evidence supports demonstration of nursing intervention effectiveness as a required function and facilitates the transition from structure and process to outcomes as a fundamental conceptual shift.13 The need to monitor nurse-sensitive outcomes and benchmark performance has resulted in development of multiple national databases.14 Despite increased data access, a growing research base, and expanded use of performance improvement methodologies including Lean and continuous quality improvement, nursing has primarily relied on soft data and anecdotal proof of value for intervention and effectiveness measurement. Several previous studies explored barriers related to change, implementation of evidence-based practice, and use of research in practice.13,15-18 Barriers included limited resources13,15; limited access to findings13,18; unreliable or poorly tracked nurse staffing data15,16; difficulty recording data15; organizational politics16; insufficient skills to evaluate research18; and lack of staffing,16 time,16-18 training,16,18 management support,16 nursing autonomy,17 support for research as a priority,18 and understanding concerns on the part of the staff.16,17 No evidence currently addresses barriers to nursing practice outcome demonstration as defined above.
MacDavitt et al12 reported that organizational climate must be measured by employee perceptions. Using methodology outlined by the European Commission on development of health survey instruments,19 investigators designed a tool to identify nurses’ perception of potential barriers to nurse-sensitive outcome demonstration in their current practice environment. The instrument’s conceptual framework was derived from literature on recommended requirements for, and barriers to, outcome reporting.7,10,11,14,20,21 Investigators sought expert opinion on barrier themes from CNOs, MPDs, and DC RNs at 7 Midwestern Magnet-designated hospitals to design the initial tool. This instrument was reviewed and edited by 10 nurses with expertise in the Magnet process and nurse-sensitive outcome demonstration, with input resulting in a final survey. With a Cronbach’s α of .838, the scale was deemed to be a reasonably reliable and valid measure of potential barriers to nurse-sensitive outcome demonstration.
The instrument involves a 3-part, 29-item, self-report. Part 1 (see Document, Supplemental Digital Content 1, http://links.lww.com/JONA/A268) consists of 7 demographic questions related to individual role, education level, career length, hospital size, location, and Magnet status. Part 2 (see Document, Supplemental Digital Content 2, http://links.lww.com/JONA/A269) includes 21 questions designed to elicit perceptions regarding potential barriers to demonstration of nurse-sensitive quality outcomes. Responses were scored on a 5-point Likert scale from (1) = “strongly disagree” to (5) = “strongly agree.” Reverse-order question responses were reverse-coded prior to data analysis, so higher responses corresponded to more positive answers. Data were transformed into 3 classifications for χ2 analysis, with scores of 1 or 2 combined for “disagree,” scores of 4 or 5 combined for “agree,” and scores of 3 as “neutral.” Survey questions were grouped around 5 thematic categories including resource availability, process value to the organization, benchmarks, competing priorities, and understanding of the outcome demonstration process. These categories were utilized in χ2 and analysis of variance (ANOVA) testing. Part 3 of the survey is a single question seeking qualitative response identifying existing barrier-reduction best practices in any hospital. Identifying information was requested for follow-up communication.
Design, Setting, and Sample
The study was deemed exempt from review by the institutional review board of the University of Pittsburgh. The study used a descriptive, cross-sectional survey design with a self-administered questionnaire. The setting was the 2012 national Magnet conference held in a western United States metropolitan area. The target sample was composed of 526 nurses attending a concurrent session related to demonstration of Magnet nurse-sensitive quality outcomes. The survey tool was distributed to all conference session attendees and collected at the conclusion of the session. Of 526 attendees, 331 (62.9%) completed and returned the questionnaire. Completion of the questionnaire was voluntary and deemed as consent to participate. Confidentiality of individual results was ensured by exclusion of identifying information and response reporting in aggregate.
The Statistical Package for the Social Sciences for Windows version 16.0 (SPSS Inc, Chicago, Illinois) was used to analyze data. Minitab, version 16 (Minitab Inc, State College, Pennsylvania), was used to analyze thematically grouped χ2 results. The α level was set at .05. Descriptive statistics were compiled for all variables; χ2 and ANOVA tests were used to measure significant differences between groups. All significant ANOVA data were subject to post hoc Tukey honestly significant difference (HSD) analysis. There were between 12 (3.6%) and 16 (4.8%) missing responses for each survey question. Missing values were replaced by mean values calculated for each question for use in the χ2 and ANOVAs.
The majority of respondents completing the questionnaire were DC RNs (26.1%, n = 86) or MPDs (24.5%, n = 81) (Table 1). Most had a graduate level of nursing education (49.8%, n = 165) and 20 or more years of nursing experience (61%, n = 202). Just over half (53.8%, n = 178) were from Magnet hospitals. The majority of hospitals had more than 300 beds (65.8%, n = 208) and were located in the Midwestern region of the United States (28.7%).
Perception of Barriers Related to Role
χ2 Tests were used to compare the proportion of respondents in each role by thematically grouped survey questions. Table 2 illustrates question and thematic category response means by role. Chief nursing officers, MPDs, and NLs perceived less resource availability than did DC RNs and others (Table 2). More MPDs and fewer DC RNs perceived competing priorities in the organization as a barrier to outcome demonstration (Table 2). Direct care RNs perceived greater organizational understanding, and MPDs less organizational understanding of the outcome demonstration process than did CNOs, NLs, or others (Table 2). No significant differences were found between groups related to benchmarks or perception of outcome demonstration value to the organization (Table 2). A 1-way ANOVA revealed significant differences between groups. Means, SDs, and a summary of significant results are presented in Table 3.
Shared Best Practices
Survey respondents suggested 16 facilities as potential sources related to best practices for overcoming outcome demonstration barriers, identifying MPD contact persons. E-mails were sent to the designated contact at each of 6 international facilities, with no response received. Phone calls were made to the designated contact at each of the 10 US facilities. Six MPDs were interviewed, and responses were collated (Table 4) to questions related to organizational structures and processes supporting successful nurse-sensitive outcome demonstration and barrier reduction.
This study provides some of the 1st evidence demonstrating perception of barrier existence and difference in barrier perception related to nursing role. The literature supports that successful behavior change depends on barrier identification and strategy development to overcome barriers.16 Nurses are crucial to improving quality21 and increasingly accountable for demonstrating provision of high quality care. Magnet organizations are challenged to be innovators in healthcare. Nembhard et al22 cite data monitoring and process modification as essential to successful innovation.
Magnet program directors play a pivotal organizational role. Although each may function uniquely within the organizational structure, the MPD holds knowledge, insight, and expertise related to the Magnet designation process. Organizational assessment of MPD structural reporting, committee participation, quality plan development, and responsibilities for DC RN knowledge enhancement may be beneficial. Linking MPD knowledge to existing and developing organizational structures, processes, communication, and quality practices will enhance MPD influence and align organizational processes with Magnet requirements.
Study results show a gap between MPD and CNO perception of resources needed, and provided, to support nursing outcome demonstration. Chief nursing officers perceived the need for fewer personnel resources to analyze, report, and articulate Magnet outcome data and that too many hours were required of those personnel. As reimbursement declines and nursing leaders are challenged to reduce resources, budgets will be tight. Chief nursing officer advocacy for required resources is crucial. Outcome demonstration, although valuable and necessary, is resource-intensive. Organizations need to explore this gap to identify and solidify nursing’s resources.
One key resource is the electronic health record (EHR), facilitating quality data tracking and retrieval. Magnet program directors perceived EHR coordination and ease in obtaining needed outcome reports as a barrier. This indicates opportunity in EHR development to support production of required quality indicator reports. Template development must simplify data collection, management, and reporting processes to optimize available resources.
All roles except the MPD perceived current quality projects are initiated with meaningful “before” and “after” measures. Feedback from best practices organizations strongly reinforced this perception. Magnet program director awareness of data reporting requirements and complexity may not currently be translated throughout organizations. Nurses are familiar with traditional quality improvement processes, with internally established targets. Education focused on identification and use of external benchmarks, Magnet requirements, and how to obtain meaningful change measurement may establish understanding and integration of effective outcome demonstration. Nurses need to understand process versus clinical outcomes and how to discern the impact of nursing interventions on patients. For example, nurses who observe a skin care concern may develop an educational intervention. Current outcome measurements, often process oriented, reflect the number of education attendees or attendee perception of the class. More effective outcome measurements are clinically oriented and may include pre and post number of hospital-acquired pressure ulcers or percentage of patients with the required skin assessment completed.
Nurses have ranked at the top of Gallup’s “most trusted professions” list in 12 of the last 13 years.23 Today’s healthcare and safety environment does not allow reliance solely on trust for demonstration of effectiveness, efficiency, and value. Nurses need to shift their paradigm to one of quality performance validation. This shift needs to occur in undergraduate curriculum and in practice settings to develop outcome demonstration competency as an essential component of nursing performance. Organizations must establish a culture that welcomes practice questions and a process for translating evidence and examining outcomes. As evidence-based practice continues to evolve, nurses must not only ask why they do something, but whether there is an evidential foundation for practice and whether new evidence demands evaluation of potential practice change. Provision of opportunities to pose nursing practice questions, identify process flow, implement and monitor change, and then evaluate and chart related outcomes is essential.
Evolution of emphasis on outcomes has resulted in recognition of barriers to outcome demonstration relatively recently. Prior to this study, no reliable and valid barrier assessment survey tool existed for use. This study uses a convenience sample, which may not represent all perspectives related to perceptions of barriers to demonstration of nursing care outcomes. Only MPDs were interviewed at suggested best practice organizations, and interviews may not represent all organizational aspects. The cross-sectional study design limits ability to make causal inferences. Even with these limitations, this study contributes new information to the literature on barriers to nurse-sensitive outcome demonstration.
Implications for Nursing Practice and Future Research
A fundamental conceptual shift has occurred, and the impact of healthcare can no longer be measured by elements of structure or process. Outcomes are becoming the currency of healthcare exchange.13 Outcomes validate the efficiency, effectiveness, and value of nursing. Nurses must learn to speak the language of evidence to be heard and to support nursing work.13 Gill and Gill5 note Florence Nightingale’s unique and strategic ability to demonstrate the power of descriptive statistics in practice. Today’s nurses must learn from Nightingale and structure work, reporting, and accountability processes to validate effectiveness and fulfill internal and external quality reporting requirements. The MPD holds a pivotal position, and this study indicates opportunity for design of the MPD role to improve enculturation of Magnet outcome reporting requirements and ensure integration of Magnet process knowledge into hospital data collection and reporting.
Buerhaus10 stressed the importance of organizational culture in determining “better practices” and making commitments to implement them. MacDavitt et al12 demonstrated association between Magnet status and higher nurse-reported quality of care. Outcome quantification is crucial for successful Magnet application and, for organizations seeking success in the increasingly complex, outcome-oriented healthcare delivery system. Success involves utilizing, analyzing, and concretely presenting data. The study tool may assist CNOs to identify barriers to outcome demonstration specific to their organizational environment, helping to focus valuable personnel and education resources to promote successful outcome demonstration. This study also indicates optimization of resources is key for data collection and reporting methodology development. The Magnet Recognition Program may be a vehicle for promoting understanding of nursing outcomes and for developing resource-sensitive data measurement methodology facilitating outcome demonstration.
Further research is needed into methods for identification of all relevant quality outcome demonstration barriers, identification of unmet nursing resource needs, development of interventions to address barriers, and evaluation of intervention effectiveness in barrier reduction. It is critical for nurses to understand, and demonstrate, the value nursing practice adds to the business of healthcare. This study adds to the body of nursing knowledge by examining barriers to outcome demonstration, a topic not found in current literature. It is hypothesized barrier identification will lead to development and application of quality processes designed to overcome barriers at organizational and national levels. The results of this study establish a foundational knowledge base guiding evaluation of current systems. The future challenge is to design and implement processes supporting demonstration of nurse-sensitive outcomes. As the public demands safer and higher quality care, nursing and healthcare organizations must commit to demonstrating nursing impact on outcomes. The evolution to demonstration of nursing outcomes will prepare nurses to move from the bedside to the boardroom, competently articulating the value of nursing work.
The authors thank Randy Kolb and Sara Brodeur, St Cloud State University Statistical Consulting and Research Center, St Cloud, Minnesota, for their thoughtful comments and statistical methodological consultation, and the St Cloud Hospital Volunteer Auxiliary for research grant support.
3. Institute of Medicine, Board on Health Care Services. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
4. Wolf G, Triolo P, Reid Ponte P. Magnet recognition program: the next generation. J Nurs Adm. 2008; 38 (4): 200–204.
5. Gill CJ, Gill GC. Nightingale in Scutari: her legacy reexamined. Clin Infect Dis. 2005; 40 (12): 1799–1805.
6. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966; 44 (3): 166–203.
7. Loan LA, Patrician PA, McCarthy M. Participation in a national nursing outcomes database: monitoring outcomes over time. Nurs Adm Q. 2011; 35 (1): 72–81.
8. Doran D, Harrison MB, Laschinger H, et al. Relationship between nursing interventions and outcome achievement in acute care settings. Res Nurs Health. 2006; 29 (1): 61–70.
9. White P, Pringle D, Doran D, Hall LM. The nursing and health outcomes project. Can Nurse. 2005; 101 (9): 15–18.
10. Buerhaus PI. Lessons from TCAB and more: an interview with health economist and quality researcher Jack Needleman. Nurs Econ. 2010; 28 (4): 276–286.
11. Foulkes M. Nursing metrics: measuring quality in patient care. Nurs Stand. 2011; 25 (42): 40–45.
12. MacDavitt K, Chou S, Stone PW. Organizational climate and health care. Jt Comm J Qual Patient Saf. 2007; 33 (11): 45–56.
13. Hayward S, Ciliska D, DiCenso A, Thomas H, Underwood EJ, Rafael A. Evaluation research in public health: barriers to the production and dissemination of outcomes data. Can J Public Health. 1996; 87 (6): 413–417.
14. Doran D, Mildon B, Clarke S. Towards a national report card in nursing: a knowledge synthesis. Nurs Leadersh. 2011; 24 (2): 38–57.
15. Buerhaus PI, Needleman J, Mattke S, Steward M. Strengthening hospital nursing. Health Aff. 2002; 21 (5): 123–132.
16. Cheater F, Baker R, Gillies C, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009; Issue 3. Art. no.: CD005470.
17. Brown CE, Wickline MA, Ecoff L, Glaser D. Nursing practice, knowledge, attitudes and perceived barriers to evidence-based practice at an academic medical center. J Adv Nurs. 2008; 65 (2): 371–381.
18. Walczak JR, McGuire DB, Haisfield ME, Beezley A. A survey of research-related activities and perceived barriers to research utilization among professional oncology nurses. Oncol Nurs Forum. 1994; 21 (4): 710–715.
20. Lundmark V. Chapter 46: Magnet environments for professional nursing practice: In: Hughes RG, ed. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality, US Department of Health and Human Services; 2008. http://www.ncbi.nlm.nih.gov/books/NBK2667/
. Accessed February 23, 2013.
21. Draper DA, Felland LE, Liebhaber A, Melichar L. The role of nurses in hospital quality improvement. Center for Studying Health System Change. Res Brief. 2008; 3: 1–8.
22. Nembhard IM, Alexander J, Hoff T, Ramanujam R. Why does the quality of health care continue to lag? Insights from management research. Acad Manage Perspect. 2009; 23 (1): 24–42.