Secondary Logo

Evidence-Based Practice, Step by Step: Evaluating and Disseminating the Impact of an Evidence-Based Intervention: Show and Tell

Fineout-Overholt, Ellen, PhD, RN, FNAP, FAAN; Gallagher-Ford, Lynn, MSN, RN, NE-BC; Mazurek Melnyk, Bernadette, PhD, RN, CPNP/PMHNP, FNAP, FAAN; Stillwell, Susan B., DNP, RN, CNE

AJN The American Journal of Nursing: July 2011 - Volume 111 - Issue 7 - p 56-59
doi: 10.1097/01.NAJ.0000399317.21279.47
Feature Articles

This is the 11th article in a series from the Arizona State University College of Nursing and Health Innovation's Center for the Advancement of Evidence-Based Practice. Evidence-based practice (EBP) is a problem-solving approach to the delivery of health care that integrates the best evidence from studies and patient care data with clinician expertise and patient preferences and values. When delivered in a context of caring and in a supportive organizational culture, the highest quality of care and best patient outcomes can be achieved.

The purpose of this series has been to give nurses the knowledge and skills they need to implement EBP consistently, one step at a time. The final article in the series will be published in the September issue.

In this 11th article on implementing evidence-based practice, the team evaluates the impact of their intervention.

Ellen Fineout-Overholt is clinical professor and director of the Center for the Advancement of Evidence-Based Practice at Arizona State University in Phoenix, where Susan B. Stillwell is clinical professor and associate director, Lynn Gallagher-Ford is clinical assistant professor and assistant director, and Bernadette Mazurek Melnyk is dean and distinguished foundation professor of nursing at the College of Nursing and Health Innovation. Contact author: Ellen Fineout-Overholt,

In the previous article in this series, Carlos A., Rebecca R., and Chen M. completed the unit-based pilot phase of the rapid response team (RRT) rollout. They found that the RRT worked well, and they are now ready to evaluate its impact on their chosen outcomes. The hospital leadership as well as the staff had agreed upon the following outcomes: code rates outside the ICU (CRO), unplanned ICU admissions (UICUA), and hospital-wide mortality rates (excluding do-not-resuscitate situations) (HMR). Karen H., the nurse from the Clinical Informatics Department, and the pilot unit's quality council representative devised a mechanism to successfully export the RRT data from the electronic medical record (EMR) to a database that would serve as a repository until the data could be analyzed. The other departments collecting RRT outcomes data have been forwarding their information to Rebecca and Chen, who've asked Karen for help in getting this additional data onto the hospital's quality dashboard. Karen suggests that she and the EBP team meet to discuss ways to upload all of the data to one place and create a single comprehensive and regularly available summary of the RRT outcomes.

At that meeting, Karen suggests that the EBP team work out a plan with the Quality/Performance Improvement Department to analyze the data before they're posted on the dashboard, where they'll be available to everyone on the hospital intranet. The EBP team members share their excitement about taking the next step in the EBP implementation process. But when Carlos contacts the director of the department, the director informs him that it may be impossible for quality/performance improvement to take on this project at this time, as their analysts are already overloaded with work. Chen mentions that she's heard that university researchers may be interested in these kinds of projects, and that collaboration with a university might lead to further projects, which could keep the kind of excitement generated by the RRT initiative going. Carlos says that he has some connections at the local university and offers to discuss this opportunity with them.

Back to Top | Article Outline


Carlos calls the dean of research at the hospital's academic partner to inquire about interest in collaborating on the RRT project, particularly from a research perspective. The dean says there's a researcher who is very interested in the processes of codes and may want to get on board with their project. Carlos asks about data analysis and interpretation as part of that collaboration, and the dean replies that the university has resources they can use to accomplish that part of the evaluation process. Carlos lets Rebecca and Chen know of this opportunity and sends an e-mail to Debra P., the faculty researcher, outlining the RRT project and asking if she's interested in participating. Debra responds the next day, indicating her delight to be involved. The EBP team is excited that they'll have this opportunity to partner with the local university and accomplish their goal of performing data analysis.

Carlos discusses the initial RRT data with Debra, and they analyze it together. First, they look at the mean outcomes of CRO, HMR, and UICUA that were obtained from the real-time RRT reports. When they compare these outcomes over time, they see that the mean CRO was reduced, but that the mean HMR and UICUA hadn't changed from baseline. Debra asks whether there was any variation in the occupancy rate over the period of the pilot rollout; if there was, then the proportion of patients experiencing codes before and during the rollout might not be comparable. When Carlos replies that the occupancy rate remained consistent, Debra recommends that they conduct an independent t test to see if there's a statistically significant difference between CRO before and after the pilot phase. They find that the decrease in CRO is statistically significant, which means that the RRT had a positive effect on this important outcome that most likely wasn't a chance finding. The EBP team can't wait to share this great news with the unit. The team reviews with Debra the code records and RRT comments to determine if there were any RRT processes that might have had an impact on UICUA and HMR, and thereby explain the lack of a change from baseline. The team also provides Debra with questions about how the pilot went (who called the RRT and why? what challenges did the RRT face?) that they believe would be important to ask the stakeholders during the debriefing after the pilot. Debra says that these questions will be very helpful as she looks over the RRT processes. Having them in mind, she can see if the answers exist in the current data, if more data need to be gathered, or if further questions need to be asked.

After taking time to reflect on these processes, the EBP team works with Debra to revise them. Debra explains that it's important to plan the hospital-wide rollout so that all unit managers and staff are confident they understand the protocol, processes, and desired outcomes. They ask Pat M., the manager of the pilot unit, and two of her EBP champions to relate their experiences with the RRT to the executive leadership team, the unit managers' meeting, and the unit council leadership meeting. The unit managers were especially glad to hear Pat's story and her answers to their questions.

As the EBP team continues to discuss plans for a hospital-wide RRT, Debra's suggestions for how to improve the RRT processes in the larger rollout are easily integrated into the plan. For example, she proposes a simple way to examine the outcomes of HMR and UICUA: since ICU deaths were included in the HMR data, she suggests that they ask the Health Information Management Systems/Medical Records (HIMS) Department to compare the ICU deaths that occurred despite the presence of an RRT with those that occurred without an RRT present. Debra explains to the team that these data may help them to have a better picture of the impact of the RRT on HMR. She applies the same approach to UICUA, comparing the ICU admissions of those who'd been treated by the RRT with those who hadn't. She further explains how the team can continue to observe the changes in these two outcomes over time. The EBP team is glad to hear that Debra will continue to help as they collect and analyze these data.

In preparation for the hospital-wide rollout, the EBP council confirms that EBP champions on each unit will be responsible for working with the educators to conduct education sessions about the RRT. Each unit participating in the rollout has already had three in-services on all shifts, posters put up in the bathroom and staff lounge, and an algorithm posted at the unit hub explaining how to call the RRT. Finally, nurses and secretaries from all units are invited to a meeting at which Debra and the EBP team answer all questions concerning the procedure for calling an RRT.

After the hospital-wide project begins, the EBP team asks HIMS if all is well with the baseline data and how the outcomes data are being collected. HIMS informs them that indeed the staff is doing a terrific job of entering the data into the EMR. The initial RRT reports indicate that the hospital-wide rollout is going well and that the RRT protocol is being used appropriately. When the EBP team informally interviews EBP council members, they find that everyone is seeing the difference the RRT is making—and not only in the outcomes. Clinicians, for example, are experiencing a difference in how they're helping patients avoid those outcomes. This pleases the EBP team and they look forward to sharing this serendipitous finding.

Back to Top | Article Outline


As the EBP team discusses how to disseminate the results of their project, they reiterate their commitment to involve the EBP council members, who have made such a major contribution to the project's success. Debra suggests that they hold a special meeting with unit managers to answer their questions, and to give them an overview of the dissemination plan, including the impact it may have on each unit's budget. The meeting with the managers turns out to be a lively discussion about the value of dissemination and its related costs. The managers are concerned that presenting the results of the RRT intervention at conferences is not a budgeted item for this year; they're also concerned about the challenges these opportunities will present, such as being able to support the scholarship of those clinicians whose work is accepted.

The EBP team helps the unit managers to understand that each time a clinician presents an aspect of the RRT process or outcome, the unit and hospital get positive exposure. Eventually most managers agree that dissemination is a worthwhile investment and commit to be as creative and flexible with their budgets as possible as they plan for the next fiscal year. They discuss how important it is to support these new learning and development opportunities for their staff. One unit manager, however, says that there's no way she can support anyone from her unit presenting at a conference. The EBP team informs her that several manuscripts about the RRT will be submitted for publication, which creates the perfect opportunity for those who wish to contribute, but who may not have the budget this year, to support the presentations.

The EBP team decides to hold a continuing education workshop on dissemination. They invite the EBP council members to come and bring anyone from their units who has been involved in the RRT project and is interested in contributing to presentations or publications about it. In preparing to conduct this class, the team makes a list of the aspects of the RRT project that would be important to include in a presentation or publication or both. They work out an agenda for the workshop (see Dissemination Workshop Agenda). Rebecca, Chen, and Carlos are excited about sharing the outcomes of first the pilot and then the rollout to the whole hospital. They are thrilled that they've made such a difference in their hospital's culture, as well as in patient outcomes.



Back to Top | Article Outline


The EBP council, the educators, the RRT, and the EBP team, along with Debra, meet to discuss how to plan for dissemination of the project and its results. They discuss first putting the results of the pilot and then of the hospital-wide RRT rollout on the hospital's intranet. Carlos invites Karen from clinical informatics to join them to discuss the possibility of having an "EBP Corner" on the intranet, where updates can be provided for the latest EBP events. Karen says this is very doable and that she'll get back to them in a couple of days on how to set this up and how they'll be able to contribute to it. Carlos agrees to take the lead for this aspect of the dissemination project.

The EBP council, with mentorship from Rebecca and Chen, expresses the desire to present the RRT project at a professional meeting. The group decides that one of the annual EBP conferences across the country would be the best place to share this project. Debra offers to help council members review the variety of EBP conferences and discuss which would be the best match. She asks them to consider which audience would like to hear about their project and where it could have a meaningful impact. She offers to join them when they start to write and then submit an abstract, and, if it's accepted, to help them put together the presentation. She also shares tips she's used that have served her well (see Presentation Tips).



To the EBP team's great delight, the chief nursing officer pops into the council meeting and tells everyone that she wants to submit this project to the American Organization of Nurse Executives (AONE) annual meeting. She's so excited about the synergy between leadership and staff that she believes this is just what participants at AONE need to hear. Carlos asks the members of the RRT if they'd like to discuss the possibility of presenting their experience at the annual Institute for Healthcare Improvement (IHI) meeting, which he tells the group may be a good venue for this project. They readily discuss sharing how their transdisciplinary team worked together to improve outcomes and other issues from the project that would interest IHI participants. They all agree to engage in this discussion further as the project continues.

Amid all this activity, Rebecca and Chen remind Carlos that there are clinicians who would rather publish than present. Carlos and Debra meet with those who are interested in publishing to provide an overview of the publishing process (see Publishing Tips). They assure those individuals who feel they don't write well enough to publish in a journal that they'll do fine as part of a team.



With plans in hand, the teams of clinicians begin to prepare their abstracts or manuscripts. The presenting teams submit their abstracts to their respective conferences. The writing teams take a little longer to prepare their manuscripts, while their team leaders call or write the journals they've selected to see if there's any interest in articles on various aspects of the RRT. The EBP team reflects on their initial PICOT question and on what a difference just asking the right question and answering it appropriately has made in their hospital.

Join the EBP team next time as they complete the hospital-wide rollout and make the RRT a hospital policy. In so doing, they will learn how to create system-wide sustainable change.

© 2011 Lippincott Williams & Wilkins, Inc.