Secondary Logo

Journal Logo


Evidence-Based Practice for Advanced Practice Emergency Nurses, Part III

Planning, Implementing, and Evaluating an Evidence-Based Small Test of Change

Shapiro, Susan E. RN, PhD; Donaldson, Nancy A. RN, DNS, FAAN

Author Information
Advanced Emergency Nursing Journal: July 2008 - Volume 30 - Issue 3 - p 222-232
doi: 10.1097/01.TME.0000334374.36905.69
  • Free


IN PREVIOUS ARTICLES in this series, advanced practice (AP) emergency nurses were introduced to the steps involved in an evidence-based practice (EBP) small test of change (Shapiro, 2007) and the critical appraisal process (Shapiro & Donaldson, 2008). In this article, we describe how to successfully implement a best practice that was identified through a critical appraisal of the literature. Successful implementation does not just happen; it requires careful planning, including an assessment of the gap between current practice and the “best practice,” and an intimate knowledge of the inner workings and people that comprise the microsystem of the emergency department (ED). Along with these elements, there needs to be systematic engagement with opinion leaders and stakeholders in a small test of the proposed change—critically important to ultimate adoption if the change proves effective. Not every small test of change is a success, so planning for change includes planning and conducting a meaningful evaluation of the impact of the change on clinical and system outcomes, then sharing the results with stakeholders to build agreement on next steps, which may be to (a) sustain and spread the change, (b) modify it and continue to evaluate it, or (c) integrate it with other change bundles to advance its adoption. In rare cases, the small test may demonstrate that the proposed change fails to live up to expectations that it improve patient or system outcomes and cannot be reasonably adapted to the current context of care.

We begin this presentation with an overview of two important theoretical constructs useful in understanding EBP change: change theory and microsystem analysis, as well as brief review of one rapid cycle change process. We then provide guidance for planning, implementing, and evaluating a small test of change in a “typical” ED. Finally, we place this information in the important context of the rapidly changing healthcare environment and the role of the AP emergency nurse.


Change, and the behaviors associated with change, has fascinated scholars and researchers for decades. In fact, a Google Scholar (2007) search on February 4, 2008, using the term “change theory,” yielded over 3 million hits in about 0.06 seconds! Melnyk and Fineout-Overholt (2005) present an overview of three widely accepted theories in their text and augment them with their own framework that stresses the need for a clear vision and goals; a belief in the ability to achieve those goals; a strategic plan that builds on the strengths of the system and deals with the potential barriers or challenges; and reliance on the three virtues of action, persistence, and patience. Some common elements that cross the models are (a) a recognition that something needs to be changed; (b) careful planning about how to go about making the change; (c) the need to enlist opinion leaders and colleagues directly and indirectly involved in the change to support the change; and (d) a way to evaluate whether the change has accomplished the desired ends.

Another common theme running through this large body of theoretical and tactical literature related to change is that the process of change is difficult to accomplish, whether it is on a personal level such as changing habits or an organizational level such as changing clinical policies, protocols, and behaviors. It is critical that the AP emergency nurse understand and appreciate that even in the best circumstances—when the evidence is compelling, management is supportive, resources are available, and staff is receptive—successful change only comes about with careful planning and constant attention to the change process and intended goals. After achieving the initial small test of change, with a commitment to continuing the change as common practice, equal effort must be focused on sustaining the change, as human beings are inclined to return to previously fixed behaviors. Hardwiring the desired change with cues, acceptable forced choices, and removing the option of previous practices are all important to sustaining enduring change.


Many healthcare organizations and EDs have adopted rapid cycle change as their model for expediting data-driven performance improvement, which includes unit-based EBP types of projects (Varkey, Reller, & Resar, 2007). Although it would most likely take months of planning and weeks of implementation to complete the change from a three-tier triage system to a five-tier system, rapid-cycle change is a great way to pilot-test various components of that change such as a new triage classification tool, or new computer screens or paper charts that are needed to support the change, and these would lend themselves easily to a rapid-cycle model such as Plan-Do-Study-Act (PDSA; Figure 1) (Institute for Healthcare Improvement, 2004).

Figure 1
Figure 1:
Plan-do-study-act model with EBP components

The PDSA model is a model complementary to that of an EBP small test of change. The “Plan” component of the PDSA model includes the problem identification; the literature review, appraisal, and synthesis; and the gap analysis, microsystem analysis, communications plan, and training plan components of the EBP change process (see below). Implementing the practice change is the “Do” part of the PDSA model. The “Study” part of the PDSA model corresponds to the evaluation phase of an EBP small test of change, and the “Act” portion corresponds to the decision of whether to adopt, revise, and restudy; or abandon the change. The biggest difference between the two models is that whereas the PDSA model is designed to quickly evaluate a succession of narrowly circumscribed changes in key workflow processes, the EBP change is one step in supporting the translation of scientifically sound evidence into consistent nursing practice that yields predictable patient care and system outcomes.


When planning for an EBP change, it is helpful to think of the ED as a clinical microsystem. Dartmouth Health Systems (Trustees, 2008) defines a microsystem as follows:

… a small group of people who work together on a regular basis to provide care to discrete subpopulations including the patients…. Clinical microsystems are the front-line units that provide most health care to most people. They are the places where patients, families and care teams meet.

Microsystems also include support staff, processes, information technology, patterns of information, behavior, and results; they are the building blocks that form the hospital (Trustees, 2008). The clinical microsystem model is useful in helping appreciate the complexity of the hospital and unit-level work environment, and enumerating the people and processes that give life to the daily functioning of the ED.

Although the hospital may mandate certain changes, and certain stakeholders enthusiastically embrace them, the hard work of implementing and adopting them occurs at the microsystem level. In an editorial devoted to challenges associated with the adoption of evidence into clinical practice, Kitson noted, “we ignore at our peril the interactions, social networks, power, politics and cultural variations that we experience in our own organizations and how the contexts enhance or detract from successful research use in practice” (2007, p. S2). More recently, Kitson et al. (2008) point out that actual EBP implementation is a highly complex, nonlinear process that is a function of the nature and type of evidence, the context in which it is being applied, and the way the change process is facilitated. Within a large ED located within a larger hospital system, where the functional microsystem may be at the shift level rather than at the unit level, the rate or success of adopting the new practice may vary from shift to shift. It is important to recognize and acknowledge the complexity of these changes in order to to respond appropriately to the challenges that will inevitably arise.


The Gap Analysis

Once the AP emergency nurse has synthesized the evidence and identified a best practice that is appropriate to her or his patients and work environment, the next step is to perform a gap analysis. The gap analysis is a systematic way to evaluate the differences between the ideal practice (represented by the evidence-based “best practice”) and current practice. This, in turn, helps the AP emergency nurse focus her or his efforts on the key elements of change necessary to better align current practice with the identified best practice. It is important to take the time to complete a formal analysis whether using a tool such as depicted in Table 1 or another approach. A systematic tool guides the nurse to deconstruct the proposed practice change into components, reconsider the evidence supporting the recommended changes, and then identify the current practices, policies, and procedures associated with each component. Most EBP changes are far more complex than would appear at first glance, and using this strategy increases the likelihood that the implementation plan anticipates and accounts for that complexity. Table 1 contains a glimpse of part of a hypothetical gap analysis for a change from a three-tier to five-tier triage system (we will continue to refer to this hypothetical practice change throughout the remaining portions of this article).

Table 1
Table 1:
Example of a gap analysis worksheet

The Microsystem Analysis

Planning for change includes identifying the key people and processes that will be affected by that change, and it is critical to the success of the proposed change to consider all those who will be impacted by the change. The greater the difference between the current practice and the proposed practice, the greater the potential impact on key clinicians and processes. We have found workbook materials from the Clinical Microsystems Web site (identified on the site as “green books”) especially helpful in this regard, and they provide a workbook specifically for EDs (Trustees, 2001). Completing the ED Profile (page 6 of the workbook) gives a good overview of key processes and stakeholders and may be all that is necessary for a small change in practice. For more comprehensive changes such as changing from a three-tier to a five-tier triage system, the AP emergency nurse may find it helpful to complete additional sections of the workbook. Be sure to also consider opinion leaders and informal and formal sources of support beyond the microsystem that are important to engage in shaping the change and in acknowledging its impacts and the implications for next steps.

Designing the Change

Using the information contained in the gap analysis and the microsystem analysis, the AP emergency nurse is now ready to construct the actual work plan for conducting and managing the small test of change. Once again we recommend using a formal planning process and provide an example of a tool that can help organize the project plan (Table 2). There are many project planning tools available both electronically and on paper, and the choice of which tool(s) to use is less important than taking the time to systematically consider and physically plan the change process. The plan should include, at a minimum, a thorough list of all the components of the proposed change and a list of key actions. It also needs to clearly address an evaluation plan that includes both pre- and postchange measures if possible. Finally, there needs to be a communication plan, a training plan, and a timeline to accomplish all the necessary components of the change.

Table 2
Table 2:
Sample project planning worksheet for change from three-tier to five-tier triage system

The Components of the Change

Referring back to both the gap analysis and the microsystem analysis, determine the actual changes in practice that will take place. Continuing with the hypothetical proposal to change from a three-tier to five-tier triage process, some of the components of the practice change may include changing the tracking board procedures, changing documentation systems, changing chart color codes, and the like. Another big component of the change is all the necessary changes in the policies and procedures related to triage. If the gap analysis has been completed faithfully, these components will already be clearly enumerated, just waiting to be slotted into the project plan. All of the components will need to be incorporated into the communications plan and many of them into the training plan, both of which are described in more detail below.

The key processes and stakeholders identified in the microsystem analysis are used to inform the actual project plan. In fact, the gap analysis, microsystem analysis, and project plan (with all its components) are iterative; one informs the other and back again, with the goal that the project plan is reflective of the true scope of the change. The discussion of change theory, above, includes the need to enlist colleagues and associates to support the change. These colleagues and associates are essential to assist the AP emergency nurse in these critical analytic and planning processes, and it would be wise to enlist their expertise and input as early as possible.

The Communications Plan

To develop a workable communications plan, the AP emergency nurse needs to determine what each group of stakeholders needs to know about the change, when they need to know it, and the most effective way to disseminate the information. Spend only as much time developing the plan as it takes to match the components of the change with the key stakeholders; most of the efforts need to be focused on the actual communications. Whether verbal, electronic, or paper-based, these efforts should present information that is readily accessible to the stakeholders. Do not overwhelm the recipients with more information than they need; in an age of information overload, the more concise and targeted the message, the more receptive the recipient. We have heard it said that to communicate new information successfully, it must be delivered to the intended audience in eight different ways, at eight different times. This will require real creativity on the part of the project team but is worth the investment.

As with everything else, the timing of communications is critical. A good communications plan begins introducing the change several weeks in advance with “stay tuned” types of messages. More information is delivered more frequently as the date of the change approaches. Communicating the change from a three-tier to five-tier triage system could employ many creative communication strategies, for example, playing on the numbers 3 and 5. Consider branding the change with either a color or a mascot. For example, all messages communicating the change in triage systems could be delivered on a designated color of stationery or slides, or printed messages could all contain a background image of a hand with all five fingers spread wide. These kinds of devices are incredibly effective in getting messages across to diverse stakeholders.

The Training Plan

Stakeholders whose practice is changing as part of the EBP change will need to undergo formal training related to the change. Here again, the format and content will vary, depending on the stakeholder or group of stakeholders. In our example, nurses will need extensive training on the new triage system, complete with clinical scenarios and possibly some precepted shifts at triage. Admission staff may need minimal training related to new forms or screens that will be used, and physicians/nursing assistants may have no formal training needs (although they clearly need to be considered in the communications plan). Plans should include the content of the training; who will conduct the training; evaluating the effectiveness of the training, including any competency checklists, return demonstrations, etc; and timing of the training relative to the change.

When large numbers of stakeholders require extensive training, the timing of training becomes especially challenging. Training should take place as near to the practice change as possible. Although just-in-time training may not be possible, or even ideal, training that occurs too far in advance is likely to fade before the change takes place, and stakeholders will swear they were never taught the necessary material, despite seeing their signatures on attendance sheets for classes at which the material was covered. This is a delicate balance made more challenging by competing training needs for staff to stay competent in the rapidly changing field of emergency nursing.

Change Champions

One of the common threads in many of the currently popular change theories is the need to enlist a core group of colleagues who will support both the concept and processes involved in the change. We call these invaluable advocates change champions. Change champions are vital to the success of any EBP change, and they serve several vital functions.

  1. They facilitate two-way communications between the leaders of the practice change and the staff, and their role in bottom-up communications is almost more important than top-down. They will let the AP emergency nurse know what is working and what is not and provide help in fixing the parts that are not working.
  2. They reinforce training at the bedside, or wherever the change is happening. They provide informal just-in-time reminders of what has changed and what the new workflow and processes are.
  3. They reinforce the need for the changes, reminding staff of the evidence base for the change and what improvements in outcomes can be anticipated by adhering to the new workflow and processes.

Depending on the nature of the change being implemented, the AP emergency nurse may either work with the champions on an informal basis, recruiting them from among those staff that indicate an interest or enthusiasm for the need for the change, or the champions might be organized into various work teams to address different components of the overall change. For example, there might be one team looking at changes in documentation, another focusing on training, and yet another on communications. The number and composition of the teams will depend on the practice change and the personnel involved, but involving staff at this level is a great way to build and sustain interest and investment in the proposed change.


There are two equally important components to implementing an EBP change. The first is the actual implementation date, and that involves both setting the date and ensuring the transition happens. Assuming that the change results in desired beneficial outcomes, the second critical phase of implementation is sustaining the change until its adoption makes it a new common practice, rather than the innovation it once was. Scholars of research utilization and EBP—now known as “implementation science”—stress the need for artful facilitation of change as crucial to the success of any efforts to integrate evidence into practice (Kitson et al., 2008), and this is the key role of the AP emergency nurse during the facilitation.

Setting the date for the change is an important first step, and we recommend setting that date only after having constructed a realistic work plan so there is adequate time for training and communication before implementing the change. However, once the date is set, every effort should be made to stick with that date; staff have been trained and are anticipating the change, and any enthusiasm and much of the training will be lost if implementation does not occur as planned. Other things to think about related to implementation are as follows.

  • Start on a Tuesday morning. For most EDs, this is a relatively quiet time and staffing is usually good. Weekends tend to be too busy for staff, and Mondays tend to be too busy for managers as well as staff. And avoid those days known for intensive operational activities such as approving payroll or posting new schedules.
  • Be present for the change. It is critically important that the AP emergency nurse who is leading the change be present when staff members are first asked to change their workflow or practices. Again, this may be more important with big changes such as the change from the three-tier to five-tier triage system, but even for smaller changes, if the AP emergency nurse is not invested in the change enough to be there when it happens, why should he or she expect the staff to be?
  • Be prepared to acknowledge and reward staff on the first shift that experiences the change. Provide food or drink if possible, or gift cards, or some other small token of appreciation for the challenges they overcome as they make the transition. They are partners in the change whether they like it or not, and recognizing their added contributions will go a long way in building a critical mass of committed staff.
  • Do not expect everything to go perfectly, and let staff know this. Tell them that early feedback will be important to identify things that may not work as planned. However, the problems must be real problems, not just “I don't like it.” Let staff know that opting out of trying the change is not an option; rather, they can participate in evaluating both the new process and identified outcomes.

Initiating the change is, in many ways, the easier of the two tasks of implementation. If preliminary evaluation results confirm the expected improved outcomes or other benefits of the change, then stakeholders must be reengaged in supporting the recommendation that the change be sustained and the hard work of sustaining the change over time begins. Often neglected in small tests of change, sustaining the change may be the more challenging component, and for several reasons. The first is the power of habits; staff members revert to old habits without even knowing they are doing so, so they need constant reminders of the importance of adopting the EBP change. The second challenge is that the infrastructure supporting old practices often remains visible and accessible for a time after the new practice is implemented, sometimes even more than the new practice. For example, if triage documentation is paper-based, and the new forms for the five-tier system are backordered, that is a temptation to revert to the three-tier system for which the old forms can still be found in the many little hidey-holes staff have for keeping critical items available. This is where the change champions provide critical assistance and support in helping sustain the change when the AP emergency nurse is not there, when it gets really busy, or when staff loses the internal motivation to continue with the new practice.

The AP emergency nurse will need to monitor the change closely for the first several weeks following implementation; the precise time is contingent upon the extent of the change and how quickly all staff can be exposed to it. In our example, just about all nurses will have worked at least one shift in the 2 weeks immediately after implementation, making it likely that unanticipated needs will already have been met. If, however, the practice change involves procedures that are used less frequently than triage, then it may take a longer period of time for all the rough spots and kinks to be identified and corrected. The phrase “inspect what you expect” captures the focus of this monitoring, whereby the AP emergency nurse attends most closely to the most critical components of the change. As staff change their practice to accommodate the change (and actually, they are probably initially changing their practice just to make the AP emergency nurse happy; to some extent the success of the project will probably ride in the AP emergency nurse's credibility with the staff), the monitoring decreases, and this is a sign that the change has been incorporated successfully into practice.


The AP emergency nurse will want to evaluate the change in at least two ways. The first is a process evaluation; that is, have the key processes changed as planned. In addition to the process evaluation, it is vitally important to evaluate the patient or system outcomes that were initially targeted when the need for change was recognized. In many cases, if the AP emergency nurse has followed the steps outlined in the EBP process (Shapiro, 2007), the evaluation components will already be in place, as they formed the basis for the EBP change to begin with. Our example is a little different in this regard, in that the indication for the practice change may not have been identified from performance data, but rather from the professional literature or attendance at professional meetings. Thus the focus is on process change, not so much on outcomes, although it would be important to evaluate the impact of these processes changes on patient and system outcomes such as early recognition and rapid treatment of impending shock and throughput times. Blank et al. (2007), for example, evaluated staff satisfaction and wait times as outcomes associated with their implementation of the five-tier system.

The evaluation component needs to be planned and communicated as carefully as the other components of the change, and resources need to be applied accordingly. Using existing data sources for these evaluations is always easier and often more reliable than designing new data collection tools and relying on others to collect the data in addition to their usual workload. We caution against using staff or patient surveys to evaluate changes other than those administered by the facility as part of their ongoing human resources or patient relations performance improvement efforts. There is a science to developing and testing valid surveys, in addition to which, it takes a lot of efforts to ensure that the return rates on the surveys are adequate to represent the target population. Surveys that have not been adequately designed or tested will yield data that may be unreliable, and that is not a good framework for evaluating an EBP change.

The timing of the evaluation is also important. An evaluation done within weeks of an EBP change will, in all probability, provide different results than one done 3–6 months postimplementation. Both the immediate and long-term impacts of the change need to be assessed and although there is usually plenty of energy to do the immediate evaluation, the greater challenge is to evaluate the change months after implementation, especially when the AP emergency nurse faces so many competing demands from patients, colleagues, and administrators. Here again, if the evaluation is based on data that are already being collected and reported, then it is a relatively simple task for the AP emergency nurse to show how those changes are a result of the change the ED recently implemented.

The overarching reason for evaluating an EBP small test of change is to decide whether to keep the change as it was initially implemented; modify and reevaluate the change on the basis of evaluation data; or abandon the change in the face of evidence that it either failed to accomplish the goal or, worse yet, resulted in poorer rather than better outcomes. Note that like change itself, the decision to accept, modify, or abandon the change is evidence-based, with the evidence in this case consisting of the results of the evaluation. In the hypothetical case of the change from a three-tier triage system to a five-tier system, it is highly unlikely that the ED would revert to its previous system. However, the evaluation may indicate that components of the change need to be adjusted, and each of those adjustments could be conducted as a rapid-cycle change as described above.


As has been stated previously, there is no longer any debate about the relevance or importance of EBP as the basis for nursing care (Shapiro, 2007). Advanced practice emergency nurses are expected to be knowledgeable in this area and to take a leadership role in implementing and evaluating EBPs. This series of articles has introduced AP emergency nurses to some of the principles of EBP and provided some instructions on core skills needed to conduct an EBP small test of change. The material contained in this series is by no means exhaustive of the literature on the process of EBP or its application to emergency care and emergency nursing practice. Instead, it has provided the reader with a foundation of knowledge about the process. It is expected that AP emergency nurses will continue their professional growth in this area so their colleagues and administrators can rely on them to

  • help identify areas of emergency nursing practice appropriate for EBP review and update;
  • critically appraise and synthesize the evidence pertaining to various aspects of emergency nursing practice, and identify best practices appropriate for their patients and setting;
  • serve as both content and process experts, advising staff and management on EBP small tests of change; and
  • mentor staff nurses and junior colleagues in aspects of EBP.

Practices based on current and validated evidence offer two important advantages over practices that persist on the basis of tradition or obsolete evidence. First, practice based on current, valid evidence will be consistent from nurse to nurse—allowing, of course, for minor variations on the basis of the nurse's personal style, particular assessment findings, or patient preferences. Second, and closely related to the first advantage, is that the results of the intervention will be predictable. This predictability applies to both what is expected in relation to therapeutic results and what is expected in the way of risks and adverse effects. Because nursing is a human science, with human factors affecting both the nurse (and colleagues) and the patient (and family), these anticipated results cannot be predicted precisely, but they can be predicted within some expected parameters, those parameters having been established through clinical or systems research. This offers great advantages to practices that vary widely with results that cannot reasonably be predicted. Our patients will be better off, and our systems will be more efficient when as we continue to adopt clinical and operational practices on the basis of the best available evidence.


Atack, L., Rankin, J. A., & Then, K. L. (2005). Effectiveness of a 6-week online course in the canadian triage and acuity scale for emergency nurses. Journal of Emergency Nursing, 31, 436–441.
    Blank, F. S. J., Santoro, J., Maynard, A. M., Provost, D., & Keyes, M. (2007). Improving patient safety in the ED waiting room. Journal of Emergency Nursing, 33, 331–335.
    Emergency Nurses Association. (2008). Standardized ED triage scale and acuity categorization: Joint ENA/ACEP statement. Retrieved March 24, 2008, from
      Google Scholar. (2007). Retrieved June 29, 2007, from
      Institute for Healthcare Improvement. (2004). Retrieved March 24, 2008, from
      Kitson, A. L. (2007). What influences the use of research in clinical practice? Nursing Research, 56, S1–S2.
        Kitson, A. L., Rycroft-Malone, J., Harvey, G., McCormack, B., Seers, K., & Titchen, A. (2008). Evaluating the successful implementation of evidence into practice using the PARiHS framework: Theoretical and practical challenges, Implementation Science, 3(1). Biomed Central.
        Melnyk, B., & Fineout-Overholt, E. (2005). Evidence-based practice in nursing & health. Philadelphia: Lippincott Williams & Wilkins.
        Shapiro, S. E. (2007). Evidence-based practice for advanced practice emergency nurses. Advanced Emergency Nursing Journal, 29, 331–338.
        Shapiro, S. E., & Donaldson, N. E. (2008). Evidence-based practice for advanced practice emergency nurses, Part II: Critically appraising the literature. Advanced Emergency Nursing Journal, 30, 136–147.
        Trustees, Dartmouth College. (2001). Assessing, diagnosing, and treating your emergency department. In Clinical Microsystems: The place where patients, families, and clinical teams meet. Retrieved February 5, 2008, from
        Trustees, Dartmouth College. (2008). Clinical microsystems: Background. Clinical Microsystems. Retrieved March 3, 2008, from
          Varkey, P., Reller, K., & Resar, R. K. (2007). Basics of quality improvement in health care. Mayo Clinic Proceedings, 82, 735–739.
          Worster, A., Fernandes, C. M., Eva, K., & Upadhye, S. (2007). Predictive validity comparison of two five-level triage acutiy scales. European Journal of Emergency Medicine, 14, 188–192.
            Worster, A., Sardo, A., Fernandes, C. M., Eva, K., & Upadhye, S. (2007). Triage tool inter-rater reliability: A comparison of live versus paper case scenarios. Journal of Emergency Nursing, 33, 319–323.

              EBP; implementing EBP; EBP and rapid-cycle change

              © 2008 Lippincott Williams & Wilkins, Inc.