Journal Logo

Feature Articles

Evidence-Based Practice, Step By Step: Following the Evidence: Planning for Sustainable Change

Fineout-Overholt, Ellen PhD, RN, FNAP, FAAN; Williamson, Kathleen M. PhD, RN; Gallagher-Ford, Lynn RN, MSN, NE-BC; Melnyk, Bernadette Mazurek PhD, RN, CPNP/PMHNP, FNAP, FAAN; Stillwell, Susan B. DNP, RN, CNE

Author Information
AJN, American Journal of Nursing: January 2011 - Volume 111 - Issue 1 - p 54-60
doi: 10.1097/01.NAJ.0000393062.83761.c0
  • Free

After the evidence-based practice (EBP) team of Rebecca R., Carlos A., and Chen M. synthesized and appraised the evidence they found to answer their clinical question, they concluded that rapid response teams (RRTs) were effective in reducing both code rates outside the ICU (CRO) and non-ICU mortality (NIM), excluding patients with do not resuscitate (DNR) orders (see "Clinical Appraisal of the Evidence: Part III," November 2010). They also decided that a reduction in unplanned ICU admissions (UICUA) may be a reasonable outcome to expect. In addition, they chose the members of their RRT: an advanced practice nurse, a physician, an ICU staff nurse, a respiratory therapist, and a chaplain.

The team's next step is to develop a plan to implement an RRT in their hospital. They begin by planning how to collect baseline data on their chosen outcomes so they can evaluate the RRT's impact on those outcomes. Carlos explains to the team that measuring outcomes, typically before and after implementing an intervention, is essential to documenting the impact of the EBP implementation project on health care quality and/or patient outcomes.1 Rebecca adds that they'll also need to consider cost as an outcome and must plan for how to capture the costs of the RRT as well as evaluate the cost savings for positive changes in CRO, NIM, and UICUA.


Rebecca and Chen are excited about the plan to implement an RRT in their hospital and tell Carlos how much they appreciate his ongoing support. Carlos checks in often with the team now that the project is under way. His experience as an expert EBP mentor has taught him the importance of assessing the team's progress at frequent intervals to see how he can support them.

To help the team develop a detailed plan for implementing an RRT in their hospital, Carlos provides them with an EBP Implementation Plan template that he used in his EBP Graduate Certificate Program (Figure 1). This plan was developed using the Advancing Research and Clinical Practice Through Close Collaboration (ARCC) model, in which EBP mentors are key facilitators of sustainable change. Carlos explains that even though they now have a template to guide them in the process, EBP implementation can be unpredictable. The team cannot anticipate all of the challenges or organizational nuances they may encounter in launching an RRT in their hospital.

Figure 1:
EBP Implementation Plan Template

Preliminary checkpoint catch-up. The team reviews the template, beginning with the Preliminary Checkpoint, to determine which steps they've already taken and which they'll need to prepare for going forward. They've already completed checkpoints one through four, but two steps in the preliminary checkpoint still need to be addressed: identifying key stakeholders and acquiring approval from the internal review board (IRB; sometimes called the ethics review board, or the human subjects or ethics committee). The team members discuss their roles in the project and agree that these may evolve as the implementation plan develops.

Key stakeholders. Carlos tells Rebecca and Chen that considering who would be stakeholders in a project—in this case, those individuals or groups that may be affected by or can influence the implementation of an RRT—is a step that's often overlooked. He explains that active stakeholders are those people who have a key role in making the project happen. Passive stakeholders are those who may not be actively involved in the project but who could promote or stymie its success. Carlos advises the team to consider all potential stakeholders, as theirs is an organization-wide project and some stakeholders may not be obvious. He asks Rebecca and Chen to think about the outcomes of the project and to which stakeholders throughout the hospital they'd be important. The team discusses that, as staff nurses, they don't always think about their work from an organizational standpoint. Carlos says that thinking about the project in an organization-wide context will help them figure out who needs to be on the team. He provides examples of stakeholders who would not only be critical to the RRT process but who might also have connections that could be important to the project's success. For example, connecting with key councils (practice, quality, critical care) or work groups (education, communications) may provide access to already-established processes for introducing a policy into the organization.

The team preliminarily identifies the members of their RRT, patients, staff nurses, and administrators as active stakeholders. They identify the finance, risk management, and education departments, mid-level managers, and the chief executive and chief nursing officers as potential passive stakeholders. The team agrees that although these may not be all of the stakeholders—more may be identified as planning continues—they're likely key players who need to be included in the implementation plan for now. Carlos tells the team that it's important to keep thinking about who will impact the project and whom the project will impact, so that everyone who needs to be on board with the plan is brought on early.

IRB approval. Carlos explains that an IRB is charged with making sure that subjects involved in a research study are safe and that the research is conducted in such a way that the findings are applicable to a broader population than just those in the study, which is known as generalizability.2 The team discusses whether they need to submit their implementation plan to their hospital's IRB for approval, since they're not conducting research. Although they'll be collecting outcomes data to evaluate whether they're achieving the expected outcomes cited in the literature, their evidence-based RRT intervention is a best practice improvement project, not a research study. Still, Carlos stresses that the team has an obligation to publish how their evidence-based intervention works in their hospital. He reminds them that the seventh step in the EBP process is to disseminate results so others can learn how a project was implemented and evaluated (the process) and whether the outcomes identified in the literature were obtained (the project outcomes, or end points) (see "The Seven Steps of Evidence-Based Practice," January 2010). Carlos tells Rebecca and Chen that if they're going to publish their project, they'll need to submit their implementation plan for IRB approval. Moreover, they cannot collect their baseline data without prior IRB approval. The team discusses that when they write up their project, they can address some of the issues they had with the reporting of implementation projects in the literature, such as how differences in the formatting of these reports makes it hard to synthesize the data (see "Clinical Appraisal of the Evidence: Part III," November 2010). For these reasons, the team feels it's essential that they publish their project, so they'll pursue IRB approval.

Before the team begins writing up their implementation plan (which they will reformulate as an IRB proposal), they discuss an essential assumption they hold, which is that all patients who enter a hospital sign a "consent for treatment" expecting clinicians and others caring for them to provide the best care possible. Although patients may not refer to their care as evidence-based practice, the EBP team feels strongly that patients' expectations reflect professional practice in which daily decisions are made based on the best evidence available. With this expectation and their decision to publish the project in mind, the team discusses that the outcomes data will be used in a way that wasn't covered in the consent for treatment. Thus, the IRB review of their proposal should reveal any ways in which publishing the outcomes of the project could put recipients of the practice change at risk. In effect, the IRB would be reviewing the plan to make sure that the data from those patients who receive the intervention will be treated confidentially.

The team discusses that their RRT intervention is supported by studies of RRTs that were submitted to and approved by their respective IRBs; that the IRB approvals of these RRT projects lends confidence to their intervention. Rebecca and Chen know it's important that their plan be reviewed, but they express concern about how to engage the IRB process. Carlos tells them that the IRB has several forms available to assist clinicians and researchers in pinpointing those aspects of their study or project that may increase risk of any kind to the people involved. The team seeks out more information on their hospital's Web site and finds the appropriate form for an implementation project. They agree to complete the form together as they develop their implementation plan.

Checkpoint five and forward. As the team moves on to Checkpoint Five in the EBP Implementation Plan template, Carlos talks to them about the critical importance of defining the purpose of the project.

Purpose of the project. A clearly defined purpose sets the entire planning process in motion, Carlos says; it's the touchstone of the project that the team can return to periodically to ensure they're on course. The team agrees that the purpose of their project is to implement and evaluate the effectiveness of an RRT in their hospital.

Baseline data collection. Carlos tells the team that collecting data prior to implementation of the RRT is important because it will help determine the extent of any already existing problems as well as enable the evaluation of the project outcomes.3 He explains that various data are generated within the hospital, which he calls internal evidence. The sources for these data are in various locations and are referred to in a variety of ways, such as: quality management, risk management, finance, and human resources departments; clinical systems; operational systems; and electronic medical records/information technology (see Table 1). Carlos tells the team that internal evidence that's collected for federal and state agencies or for regulatory and specialty organizations, such as the American Nurses Credentialing Center's Magnet Recognition Program, can also be used as outcomes. As an example, he provides reports from their hospital's quality committee that include data for CRO, UICUA, and overall hospital mortality. Chen asks what it will require to get data only for NIM. Carlos replies that he'll have to find out which department in the hospital creates quality committee reports and ask if NIM data can be culled from the overall hospital mortality data. He explains that there are many data repository systems within the hospital and that each system may collect different data and may require a different way of requesting those data. Carlos helps the team understand that obtaining data may be complicated at times, but one's success greatly depends on knowing whom to ask.

Table 1:
Potential Sources and Types of Internal Evidence

To help the team capture the outcomes data they'll need to obtain at baseline and again after the project, Carlos recommends they work with the information technology and finance departments. Chen asks if putting the outcomes in a chart would help to clearly outline the "who, what, when, where, and how" of baseline data collection. The team agrees that this would help them understand the financial outcomes (sometimes referred to as the business case), the process and structure of the project,4 and the patient outcomes that will be measured at the end of the project (see Table 2).

Considerations in Measuring Outcomes for the RRT Implementation Project

The process. The team discusses how to ensure that the process of implementing an RRT in their hospital goes well. Rebecca reminds the team about their and the MERIT trial authors' observations on how the MERIT trial was conducted, particularly on how the RRT protocol was implemented.5 (The control hospitals' code teams may have functioned as RRTs, which could explain why there was no difference between the control group and the intervention group; see "Critical Appraisal of the Evidence, Part II," September 2010). She asks the group for ideas about how they can collect data on the process of implementing the RRT to demonstrate that they have done it well. Carlos says that how well they implement the intervention is called the fidelity of the intervention. He recommends keeping good notes on the work being done. They talk about the need to develop a project data collection tool that staff can use when calling the RRT. Chen volunteers to develop this form, using similar forms in the literature they reviewed as a basis. Carlos suggests that maybe Chen should see if anything new has been published, since it's been a few months since they completed their literature search.

The team talks about the importance of measuring the costs and benefits of the RRT, especially its benefits divided by the costs, which Carlos notes is called its return on investment (ROI). Carlos suggests that the team meet with the finance department to discuss their plan to measure the costs and ROI of an RRT. Rebecca volunteers to be responsible for obtaining the financial data and requests that Carlos be available for support, if needed, to which he readily agrees. Chen agrees to work with Carlos to ensure that data on CRO, UICUA, and NIM are systematically collected and to focus on the process outcomes (how well the RRT project is implemented). For example, if there was a breach in protocol implementation—in how well the RRT protocol was delivered to the active stakeholders, for instance—that breach could lead to an outcome that was different from what was expected. This unexpected outcome may not be because the RRT intervention didn't work, but because of a glitch in the process: the RRT protocol wasn't delivered as planned.

As work on the project is planned and discussed, the roles of the team naturally begin to fall into place. As part of formulating the implementation plan, they discuss what questions about data collection they'll need to ask in order to measure their outcomes of CRO, UICUA, and NIM (see Questions to Ask in Preparation for Data Collection). Carlos reflects back on the definitions and measures the team discussed in their appraisal of the evidence and how the different definitions of mortality (whether it included DNR cases, for example) led to some confusion about comparing the impact of an RRT on that variable (see "Critical Appraisal of the Evidence: Part II," September 2010). He explains the importance of how the data are measured (what mechanisms are used, for example, and why and how to know they're good methods for measuring the data). He says that in order to determine the impact of an EBP project such as the implementation of an RRT, the data must be measurable (able to be counted), accessible (the team has access to the data), and user friendly (understandable and able to be used without difficulty). Chen and Rebecca decide they want to create a data collection plan that meets all of these criteria. With the questions on data collection to guide them, they realize that multiple disciplines within the hospital (not only nursing) will be involved in helping to collect the baseline data for the project.

From the team's discussion, Rebecca and Chen put together a preliminary plan for evaluating the RRT project, keeping the following key areas in mind: the strategic case, business case, resources case, and process measures (see Table 2). They also add the following process outcomes to their plan: the number of staff educated on the RRT, the number of RRT calls, the primary reasons for calling an RRT, and family and staff satisfaction with the RRT process.

In the March column, join Rebecca, Chen, and Carlos as they move through the next several steps of the EBP implementation process, including identifying and planning for the barriers they may encounter as the EBP change is rolled out, as well as providing system-wide education on the intended use and expected outcomes of an RRT.

Questions to Ask in Preparation for Data Collection

  • How are the outcomes defined?
  • What data will be used to measure the outcomes?
  • Who "owns" the data needed for this project?
  • Who will (or already does) generate the data needed for the project?
  • What special clearances are required to access the data?
  • What are the restrictions for sharing these data?
  • Who will be responsible for collecting the data?
  • When will the data be collected?
  • Where are the data located in the hospital?
  • How will the evidence-based practice (EBP) team access the data?
  • How will the EBP team store the data?
  • What program will the EBP team use to analyze the data?
  • Who will help the EBP team with data analysis?
  • How will the EBP team manage the data (data entry, cleaning, labeling)?


1. Fineout-Overholt E, Johnston L. Teaching EBP: Implementation of evidence: moving from the evidence to action. Worldviews Evid Based Nurs 2006;3(4):194-200.
2. Department of Health and Human Services. 45 CFR 46.101 Public welfare. Protection of human subjects; 2009 .
3. Melnyk BM, Fineout-Overholt E, Stillwell SB, Williamson K. Transforming healthcare quality through innovations in evidence-based practice. In: Porter-O'Grady T, Malloch K, editors. Innovation leadership: creating the landscape of health care. 2nd ed. Sudbury, MA: Jones and Bartlett; 2010. p. 167-94.
4. Wyszewianski L. Basic concepts of healthcare quality. In: Ransom ER, et al., editors. The healthcare quality book. 2nd ed. Chicago: Health Administration Press; 2008. p. 25-42.
5. Hillman K, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet 2005;365: 2091-7.
© 2011 Lippincott Williams & Wilkins, Inc.