Secondary Logo

Journal Logo

Innovation Reports

Developing the Agile Implementation Playbook for Integrating Evidence-Based Health Care Services Into Clinical Practice

Boustani, Malaz A. MD, MPH; van der Marck, Marjolein A. PhD; Adams, Nadia MHA; Azar, Jose M. MD; Holden, Richard J. PhD; Vollmar, Horst C. MD, MPH; Wang, Sophia MD; Williams, Christopher MHA, MBA, PhD; Alder, Catherine JD, MSW; Suarez, Shelley RN; Khan, Babar MD, MSc; Zarzaur, Ben MD, MPH; Fowler, Nicole R. PhD, MHSA; Overley, Ashley MD; Solid, Craig A. PhD; Gatmaitan, Alfonso DSc

Author Information
doi: 10.1097/ACM.0000000000002497
  • Open



Despite the more than $32 billion the National Institutes of Health has invested annually, incorporating evidence-based health care services into clinical practice remains an inefficient process.1 Furthermore, once evidence-based services are implemented, they may not be sustained, adapted, or distributed across health care delivery organizations.1 As a consequence, a majority of patients receive care that is not based on existing evidence from the literature.2

To provide consistently high-value, evidence-based health care services, organizations must use methods that reflect the complex and interconnected nature of today’s health care delivery systems. In this article, we describe the Agile Implementation (AI) Playbook, a model developed by the Center for Health Innovation and Implementation Science at Indiana University School of Medicine. This model provides a reproducible and scalable process to rapidly localize, implement, and sustain evidence-based health care services.


In September 2007, one of us (M.A.B.) assembled an interdisciplinary team of clinicians, implementation scientists, and health care administrators to develop a process to select and implement evidence-based health care services, evaluate the fidelity and impact of those services, and ensure their sustainability and scalability across health care delivery organizations. Over several years and more than 40 implementation projects at the Indiana University Health System and Eskenazi Health, an urban safety-net health care system, the team iteratively developed and refined what would ultimately become the AI Playbook, which was finalized in 2015. The team used multiple theories and frameworks, described below, to develop the AI Playbook.

Theories and frameworks informing the development of the AI Playbook

Complexity theory.

A complex adaptive system is an open, dynamic network of semiautonomous individuals who are interdependent and connected in multiple nonlinear ways (see Figure 1). Such a network has the ability to adapt to new states in response to its evolving environment by learning from prior experiences.3,4 From this perspective, member diversity and culture, member interactions, the surrounding environment, previous history, and changing and learning processes all make health care delivery organizations unique.5 As such, the capability of these organizations to adapt to constant internal and external changes depends on the characteristics of the individual members (e.g., skills, adaptability, and attitudes), as well as on the local organizational structures and environment.3,5

Figure 1:
Diagram of a complex adaptive system. The health care delivery system is a complex adaptive system, where individuals are interdependent and connected in multiple nonlinear ways to each other and the unique characteristics of the system itself. Members of such a system experience constant changes that can be internal (e.g., patients, administration, equipment, etc.) or external (e.g., payers, the economy, independent accreditation agencies, etc.). The capability of the system to adapt to these changes depends on the characteristics of the individuals as well as on the system’s organizational structure and environment.

Behavioral economics.

Behavioral economics recognizes that individuals’ behavior is not only driven by their interactions with internal and external stressors but also by their personal attributes, such as emotion, attention, and skills, and by their relational attributes, such as empathy, trust, and history. There are opportunities to leverage these human tendencies in information processing and decision making by modifying the social and physical environment.6

Sources of variation theory.

Mapping the delivery of a typical health care service reveals three sources of variation in the clinical care provided: (1) the clinical decision, which often involves multiple providers and is a function of the decision makers’ clinical knowledge and experience as well as currently accepted or emerging practices; (2) the process of translating the clinical decision into patient care; and (3) the patient’s response to the health care service provided (see Figure 2).7 Understanding these sources of variation is crucial to implementing effective and sustainable evidence-based health care services.

Figure 2:
Diagram of the sources of variation theory. There are three sources of variation in clinical care. At each level of an organization, there is variation in the clinical decision, the process of translating the clinical decision into patient care (the production line), and the patient’s response to the health care services provided.

Five factors framework.

Chaudoir and colleagues’ framework depicts five macro-to-micro nested factors that influence the implementation of innovations.8 At the most macro level is an external, sociocultural, structure-level factor representing the broader context or community in which an organization is nested. Next is an organizational-level factor, followed by a provider-level factor, a patient-level factor, and finally a health-related innovation-level factor that characterizes the innovation itself.

Implications of these theories and frameworks.

Across these theories, the common message is that health care delivery organizations are complex, adaptive, and sociotechnical. Implementing changes requires an approach that attends to: (1) variation that is both temporal (across process steps) and hierarchical (across levels of analysis); (2) the human element and human-to-human or human-to-technology interfaces; and (3) the way organizations function in and adapt to the broader sociocultural, legal-political, and organizational-regulatory environments.

The AI Playbook

The AI Playbook is a model for carrying out an interconnected, eight-step, AI cycle (see Figure 3), facilitated by a trained AI agent. This agent may be someone either internal (e.g., clinician or administrator) or external to the organization, but she or he should know how to identify an appropriate evidence-based solution and be able to facilitate changes at both the organizational level (zooming out) and the individual level (zooming in). The steps of the AI process are rooted in the theories and frameworks described above, in that they take into consideration the uniqueness of each health system (complexity theory) and recognize that variation in clinical decisions, translation into patient care, and patient responses (sources of variation theory) will influence the outcomes of a selected solution. The AI Playbook is designed to leverage aspects of behavioral economics and the sociocultural and multilevel factors described in the five factors framework to guide interactions and evaluations to encourage individuals to act in ways that enable the success and sustainability of the selected solution. Below we describe the eight steps in the AI Playbook.

Figure 3:
Diagram of the Agile Implementation Playbook. This model has eight interconnected steps for identifying and implementing an evidence-based health care service to address a designated opportunity. These steps attend to the complex nature of health care delivery organizations and acknowledge the uniqueness of the individuals and organizations involved.

Step 1: Identify opportunities.

The AI agent proactively works with leadership and clinical providers to identify opportunities for locally implementing new evidence-based health care services. High demand for addressing an identified opportunity—gauged by the amount of time, personnel, and financial resources executive leadership and frontline clinical providers are willing to invest—is required to proceed.

Step 2: Identify evidence-based health care services.

The AI agent conducts a systematic search to identify evidence-based health care services that address the selected opportunity and promote the quadruple aim (high-quality, accessible, cost-efficient, and patient-centered care). When identifying potential solutions, we recommend employing a critical appraisal, such as the grading process used by the U.S. Preventive Services Task Force, to determine the quality and strength of the evidence supporting a service. If there are no evidence-based services that address the selected opportunity and promote the quadruple aim, the AI agent either confirms with leadership their willingness to develop new evidence-based services or returns to Step 1.

Step 3: Develop evaluation and termination plans.

The AI agent works with organizational leadership to develop an evaluation protocol and selects the appropriate measures for the organization, the care delivery service type, and the implementation goals. They also set milestones and indicators of success. The evaluation plan must identify the criteria for de-implementing the planned service as early as possible if it is deemed a failure as well as who will lead the de-implementation.

Step 4: Assemble a team to develop a minimally viable service.

The AI agent works with leadership to build a diverse, local, interdisciplinary implementation team to convert the selected evidence-based service(s) into a minimally viable service by adapting the content and delivery process to the local setting. A minimally viable service reflects the critical aspects of the solution that must be retained to stay true to the original evidence-based process or method. This step takes into consideration the unique characteristics of the local health care delivery system and its individuals in their surrounding environment. The minimally viable service is iteratively revised in subsequent steps.

Step 5: Perform implementation sprints.

The AI agent facilitates self-contained sprint cycles, or units of focused work, to assess the proper process for adapting the selected service to the local setting and to evaluate the service’s outcomes via the plan developed in Step 3. Sprints result in lessons learned that can be applied to subsequent redesign-and-sprint iterations.

Step 6: Monitor implementation performance.

The AI agent and the implementation team develop feedback loops to monitor the fidelity and performance of the selected service. They reflect on what they are learning, gauge impact while acknowledging any conflict and tension, detect emerging problems, identify and prioritize solutions to those problems, and adjust the implementation process and sprints accordingly.

Step 7: Monitor whole system performance.

The AI agent and implementation team monitor the impact of the selected service on the overall quality and financial performance of the entire organization to detect any unintended or adverse consequences as well as any emergent opportunities that can be leveraged for additional benefit.

Step 8: Develop a minimally standardized operating procedure.

If the implementation of the selected service is determined to be meeting internal demands and goals, the AI agent and implementation team develop a minimally standardized operating procedure manual. This manual describes the basic attributes of the solution that are required to maintain fidelity to the final service and that should be incorporated when implemented in other settings. It is updated on a regular basis and helps promote the successful service to other departments within the same organization and across organizations.


In 2010, Eskenazi Health enlisted the Center for Health Innovation and Implementation Science to reduce the vulnerability of intensive care unit (ICU) survivors through the development of a critical care recovery center. Stakeholders indicated that there was demand for this improvement (Step 1) by citing the growing trend of hospitals that care for critically ill patients being responsible for their care for a period of time after their initial recovery and/or hospital discharge. Additionally, administrators saw high-quality post-ICU care as a marketable attribute for the facility. After selecting a collaborative care model as the viable evidence-based service (Step 2), the Center for Health Innovation and Implementation Science and selected leadership identified appropriate indicators for success (Step 3), including improved physical and cognitive symptoms in patients, reduced inappropriate post-ICU health encounters, and lower cost.

Adapting the solution to the local setting (Step 4) involved specifying minimum care components, such as early assessment of functionality, patient and caregiver education, and longitudinal monitoring of patient outcomes. Multiple sprints (Step 5) were used to translate the evidence-based protocols of the collaborative care model into protocols to meet the cognitive, functional, and psychological needs of ICU survivors and family caregivers. During one sprint, high no-show rates to the first post-ICU follow-up appointment at the critical care recovery center signaled a need to add a direct referral from the ICU for 90 days after discharge and to set up a pre-clinic phone call with patients and family caregivers to promote the value of the recovery center.

To monitor the critical care recovery center’s performance (Step 6), the team distributed quarterly dashboards with data on several measures (e.g., percent receiving antidepressants, number of primary care or specialty visits, etc.) and evaluated changes in patients’ cognitive, functional, and behavioral-psychological symptoms at multiple time points. To assess the impact on the health system (Step 7), the team tracked overall readmissions, emergency department use, and a variety of costs related to care utilization.

Early results demonstrated improvements in cognitive, functional, and behavioral measures and a reduction in acute health care utilization.9 The critical care recovery center is currently in its seventh year of operation. The development of a minimally standardized operating procedure (Step 8) allowed the service to be adapted for home-based ICU survivors and trauma survivors involved in two National Institutes of Health-funded clinical trials currently underway.

In addition to improving care for ICU survivors, the AI Playbook has been used to implement solutions to improve dementia care and reduce health care-associated infections. In addition, as part of the Great Lakes Practice Transformation Network (, a multi-state effort to improve the quality and reliability of the ambulatory care provided within a set area funded by the Centers for Medicare and Medicaid Services’ Transforming Clinical Practice Initiative, the Center for Health Innovation and Implementation Science has trained more than 50 quality improvement advisors in AI to guide the transformation of 2,100 practices and implement resources adapted to the unique characteristics of each. A formal evaluation of the impact of this work by an independent organization has been contracted and is expected by the end of 2019.

The AI Playbook can address issues arising from all types of variation within a health care system, and it acknowledges both internal and external forces and accommodates the unique characteristics of each system and its environment. Rooting the AI Playbook in grounded theories and frameworks is supported by evidence that interventions with a theoretical basis are more effective than those without, especially when combinations of theories are used, like in the AI approach.10 However, the final version of the AI Playbook is also a function of practical experience; it incorporates lessons learned during its development and use, including the importance of confirming demand for a service (Step 1) and relying on previously proven, evidence-based solutions (Step 2) instead of attempting to develop a solution from scratch. Finally, the AI Playbook emphasizes instituting a termination plan (Step 3) for unsuccessful solutions to solicit buy-in from administrators and ensure that resource waste will be minimal.

Next Steps

To improve the implementation of evidence-based health care services, we intend to continue disseminating the details of the AI Playbook and illustrating how health care delivery organizations can successfully leverage it. Doing so will require educating staff and administrators about both the AI process to facilitate mastery of the underlying theories and frameworks and the ability to identify and adapt appropriate evidence-based solutions to local settings.

Acknowledgments: The authors wish to acknowledge the support they received from their health care partners: Indiana University Health, Eskenazi Health, and the practice members of the Great Lakes Practice Transformation Network.


1. Balas EA, Boren SA. Managing clinical knowledge for health care improvement. Yearb Med Inform. 2000(1):65–70.
2. Ebell MH, Sokol R, Lee A, Simons C, Early J. How good is the evidence to support primary care practice? Evid Based Med. 2017;22:88–92.
3. Anderson RA, Crabtree BF, Steele DJ, McDaniel RR Jr. Case study research: the view from complexity science. Qual Health Res. 2005;15:669–685.
4. Hagedorn H, Hogan M, Smith JL, Bowman C, Curran GM, Espadas D, et al. Lessons learned about implementing research evidence into clinical practice. Experiences from VA QUERI. J Gen Intern Med. 2006;21 Suppl 2:S21–S24.
5. Boustani MA, Munger S, Gulati R, Vogel M, Beck RA, Callahan CM. Selecting a change and evaluating its impact on the performance of a complex adaptive health care delivery system. Clin Interv Aging. 2010;5:141–148.
6. Patel MS, Volpp KG. Leveraging insights from behavioral economics to increase the value of health-care service provision. J Gen Intern Med. 2012;27:1544–1547.
7. Azar J, Adams N, Boustani M. The Indiana University Center for Healthcare Innovation and Implementation Science: Bridging healthcare research and delivery to build a learning healthcare system. Z Evid Fortbild Qual Gesundhwes. 2015;109:138–143.
8. Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8:22.
9. Khan BA, Lasiter S, Boustani MA. CE: critical care recovery center: an innovative collaborative care model for ICU survivors. Am J Nurs. 2015;115:24–31; quiz 34, 46.
10. Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health. 2010;31:399–418.
Copyright © 2018 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the Association of American Medical Colleges.