Secondary Logo

Journal Logo

An EPA Agenda for the Next Decade

Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade

ten Cate, Olle PhD1; Balmer, Dorene F. PhD2; Caretta-Weyer, Holly MD, MHPE3; Hatala, Rose MD, MSc4; Hennus, Marije P. MD, PhD, MClinEd5; West, Daniel C. MD6

Author Information
doi: 10.1097/ACM.0000000000004106
  • Free

Abstract

Since the concept of Entrustable Professional Activities (EPAs) was introduced, the number and range of EPA-related publications in the literature has increased tremendously. This literature may be categorized as: (1) endorsements of EPAs as novel in medical education and other health professions domains; (2) commentaries, criticisms, elaborations, clarifications, and guidelines; (3) proposals for EPA frameworks, locally, nationally and internationally; (4) early implementation studies with a focus on feasibility; and (5) studies seeking validity support for EPA-based assessment or entrustment-based decisions. In contrast, reports exploring theory, large-scale implementation, outcomes, and impacts are limited. 1–3

The growth of EPA literature has paralleled a similar growth in competency-based medical education (CBME). The general concept of CBME requires further empirical and conceptual support, 4 and EPAs can potentially serve as an effective framework around which to scaffold CBME programs or implementation efforts.

Clarity about the theory and concept of EPAs and their potential implications for health professions education make it necessary to propose a research agenda. An EPA is defined as a “unit of professional practice (task or bundle of tasks) that can be fully entrusted to a trainee, once he or she has demonstrated the necessary competence to execute this activity unsupervised.” 5 Entrustment decision making, that is, deciding how far to trust trainees to carry out patient care on their own, 6 is inextricably connected to the use of EPAs. EPAs and entrustment decision making reflect more than a different framing of educational objectives, or a different tool for assessment. EPA-based programs explicitly frame a culture in which learners gradually receive increased autonomy and responsibility in patient care as they work toward becoming competent health professionals. 7–10 EPAs provide a shared framework that is intuitive to learners and teachers because it mirrors the learning and training processes that naturally occur in the clinical learning environment. Yet the interpretations of entrustment in the clinical setting vary and would benefit from conceptual analysis. 11 In addition, future research should be designed to investigate the contribution of EPAs to the goals of CBME programs to graduate health professionals who can practice: at a defined level of proficiency, in accord with local conditions, to meet local needs, in a system of fixed outcomes and time-variable training. 12,13

Approach

To prepare for this perspective piece, M.P.H. reviewed all 2020 EPA-related publications (January to October) and extracted recommendations for future research from the discussion sections. This yielded 134 papers; surprisingly, only 28 included statements that could be qualified as a recommendation for future work. Some of these statements were rather general, for example, to establish effectiveness of EPAs, to conduct more outcome-based or more rigorous studies, or to confirm validity of their own findings. Other statements have informed the recommendations below.

In addition to literature resources, the authors drew from personal experience with and knowledge of EPAs, conference presentations, and yet unpublished research with which they were acquainted. They held 2 team brainstorming sessions in the fall of 2020 to generate and vet ideas. They initially organized potential recommendations according to 2 distinctions: (1) an agenda for research versus an agenda for educational development and (2) micro (individual learner or teacher) versus meso (institutional) versus macro (regional, national or international) levels of agenda items. After a merge of development and research, which appeared difficult to disentangle, the categories outlined in Box 1 and the manuscript emerged: an agenda for research and development at micro, meso, and macro levels, respectively.

Box 1 Recommended Areas of Development and Research in the Decade Ahead

Agenda for development and research at the micro level of learning and teaching

  • Unpacking teacher and learner understanding of entrustment-focused assessment
  • Understanding the conditions for rich entrustment decisions
  • Addressing the tensions between formative and summative assessments
  • Exploring the impact of EPAs and entrustment decisions on cognition, motivation, and agency

Agenda for development and research at the meso level of institutions, programs, and specialty domains

  • Exploring the purpose, methods, and consequences of EPAs for professions
  • Adapting curricula to prepare learners for EPAs
  • Elaborating programs of assessment and the use of technology
  • Articulating the validity argument for EPA-based assessment: emphasizing educational and clinical outcomes
  • Optimizing summative entrustment decision making

Agenda for development and research at the macro level of regional, national, and international stakeholders

  • Using EPAs across the educational continuum and into practice
  • Acknowledging EPAs as units of practice across systems
  • Acceptance of readiness for unsupervised practice by all relevant stakeholders
  • Sharing EPAs across disciplines and professions
  • Implementing EPAs for continuing professional development and maintaining certification

We did not include recommendations for additional implementation projects that we anticipate will dominate the literature in the near term, such as proposals for EPAs for specialties or professions. We expect these to appear anyway and can only recommend that they be rigorously executed. 14

Agenda for Development and Research at the Micro Level of Learning and Teaching

The micro level, that of the clinical education microsystem, involves individual teachers, learners, and patients, and is highly relevant for programs that work with EPAs. 15 Clinical education microsystems require trust, honesty, and self-respect among their members, 16 features that align well with entrustment decision making. Entrustment decision making is not a novel phenomenon in these systems, but it has not been formalized as a core focus of most educational programs. In programs that use EPAs, an ad hoc entrustment decision is terminology to signify when a supervising clinician decides to delegate a particular patient care task in a particular situation to a learner, not necessarily with a commitment for future instances. 17 A summative entrustment decision is when a promotion or advancement decision is granted that implies permission to act with decreased supervision and could lead to advancement in training, completion of training, licensure, or certification to practice. Using EPAs as a competency framework for education and assessment deliberately focuses on a gradual growth in autonomy that happens naturally in the clinical workplace and breaks down summative entrustment into relevant units of professional practice. Entrustment decisions, as originally described, were not meant to be simply another judgment of proficiency, without consequences for learner growth in autonomy. 18 Perhaps because some programs have simply implemented entrustment-based ratings within existing training program structures, as opposed to explicitly redesigning and scaffolding training programs based on the principle of progressive entrustment, not all programs that currently employ EPAs create the intended progression in learner autonomy before and after “entrustment decisions.” The gap between the original intent of entrustment decision making and the real-world implementation leads to intended and unintended consequences for both learners and programs; this is a fertile ground for future research. How can clinical supervisors best shift to entrustment-based assessment? How can EPA assessments be merged with longitudinal, interprofessional, i.e., multisource, evaluations of general behavior to create a picture of trustworthiness? How do learners experience entrustment decision making?

Unpacking teacher and learner understanding of entrustment focused assessment

Entrustment decision making differs from observational assessment in the clinical workplace. While the latter regards observed performance retrospectively: (What happened and how did the learner perform?), the former requires making an inference about readiness for less supervision prospectively: (What responsibility can a learner be expected and trusted to bear in future instances?). 18,19 Entrustment decision making asks the supervisor to use their expertise to make a judgment about readiness for future performance in a way that is not done in traditional assessments of competence, which focus only on the current performance. 19 Thinking ahead is difficult because it requires contextualization of what has been observed and a more holistic inference to support a decision. 20 Adequate entrustment-based assessment forms and faculty development may increase the understanding of entrustment as assessment; however, the conditions for trust to develop, such as a supervisor’s sufficient exposure to a learner, 21 may require structural changes in clinical training. There are other challenges: judgment about learners cannot always be expressed well in words, 22 scoring formats inherently reduce the richness of impressions into preformulated rubrics that do not always match with what clinical teachers think, 23,24 and clinical teachers struggle to articulate reasons for underperformance. 25,26 This critical direction for future research and development will deepen understanding of how teachers and learners make sense of entrustment decisions, and should include procedures and tools to support teachers and learners. Studies of learner views on entrustment processes are only slowly emerging 27,28; this area deserves more attention.

Understanding the conditions for rich entrustment decisions

ten Cate and Chen recently reported that when making entrustment decisions, clinicians value 5 characteristics in learners: agency, reliability, integrity, capability, and humility. 29 These findings, based on observational interview and focus group studies with clinical teachers, should be further validated using more direct methods, such as analyzing written feedback narratives linked to judgment of readiness for less supervision. One significant condition for a rich entrustment decision is the learner–supervisor relationship. Supervisors well acquainted with learners are better positioned to make decisions to trust them with critical health care activities. 30 Reciprocal trust of learners in their supervisors may facilitate self-directed learning. 27,31 While longer relationships presumably enable more valid entrustment decisions, 21,32 and short rotations may hamper their validity, 33 the dynamics in supervisory relationships and their effects on entrustment decisions have not been studied extensively. Qualitative research methods, possibly supported by artificial intelligence techniques such as using natural language processing to analyze feedback statements, 34 may shed light on how trusting relationships build and how these affect the quality of entrustment decisions, both over time with less intensity and in short but intensive collaborations.

Addressing the tensions between formative and summative assessments

Formative assessment (for the purpose of feedback for development, sometimes called assessment for learning) and summative assessment (for the purpose of making decisions that formally affect student progress, sometimes called assessment of learning) are difficult to disentangle. Programmatic assessment, in which routine information about a learner’s competence is continually collected and analyzed, 35 is an emerging movement in health professions education and requires frequent assessments that intend to be primarily formative. 36,37 However, students often experience these assessments as summative, 38 as all of their successes and failures are documented. 39 Using EPAs to make summative entrustment decisions also requires extensive documentation of a variety of assessments to guide decision making by clinical competency committees. 40 The tension between the need to provide formative feedback and to use the same information to inform summative decisions creates a significant dilemma and should be an important focus of future research.

Exploring the impact of EPAs and entrustment decisions on cognition, motivation, and agency

Whether EPAs provide clarity of learning objectives as many have hypothesized 41 and consequently lead to improved learning has yet to be confirmed. Similarly, whether or not the mere granting of responsibility affects motivation is unknown. A decision to trust a learner with an EPA may enhance intrinsic motivation, as learners may feel acknowledged in their competence, feel increased autonomy to act without direct supervision, and feel increased responsibility to their teams and patients as a valued partner in a health care team. 42,43 The effects of EPA-based programs on self-regulated learning, the development of adaptive expertise, and the relevance of self-determination theory on motivation are a few directions for this line of inquiry. 19,44 Learner agency is a condition for success in an EPA-based clinical learning environment. 27,29 To date, much attention has been directed toward creating an EPA assessment framework with a shared language, roles for teachers and learners, and a learning environment that supports CBME. But learners are not only shaped by EPA assessment; they can also shape EPA assessment. 41 Agency, defined as exercising the capacity to influence understanding of the consequences of one’s own actions and using the power of participating in the production of social systems and processes, 45 is critical to workplace learning. 41 While there is promising work on how trainees are influenced and shaped by the EPA assessment structure, future research should explore how learners’ agentic behavior influences and shapes EPA assessment.

Agenda for Development and Research at the Meso Level of Institutions, Programs, and Specialty Domains

Beyond the micro system of teachers, learners, and patients, we must consider the learning environment at large. Our fixed beliefs and entrenched accreditation systems are interdependent with the context in which EPAs are implemented, entrustment occurs, and professionals develop even after they are deemed ready for unsupervised practice. 46,47 The increasingly complex and variable learning environment where the work of CBME and EPA implementation occurs 48 constitutes an intriguing yet daunting area for further exploration. These studies will have far-reaching implications for those who implement EPAs across specialties, schools, accreditation bodies, countries, and other broad contexts.

Two issues stand out. First, EPAs are a potential curriculum blueprint to define a specialty and guide educators and learners in determining the desired outcomes of training. Second, EPAs and entrustment decision making have opened a new avenue for assessment that requires thoughtful elaboration regarding its validity. The 2 paths of EPAs as blueprint and EPAs for assessment have a natural intersection and feedback loop when we consider our graduating physicians. While more in line with program evaluation than primary research, it behooves us to feed data about EPA achievement in graduating physicians back to the designers to adapt EPA standards if needed. What are the implications for certification and continuing professional development? Do we have appropriate expectations for graduating physicians? Are all EPAs achievable? Will supervision or coaching be needed after training? Are there curricular changes required at an institutional level that would facilitate attainment of certain EPAs? Can patient outcomes, collected at the institutional level, inform EPA development?

Exploring the purpose, methods, and consequences of EPAs for professions

EPAs signpost the essential tasks of a specialty for society, institutions, teachers, and learners. While originated with an educational purpose, thoughtful EPAs may even define or reshape professions. 49–53 Future research could examine whether EPAs provide a common language within which to teach, learn, assess, and even structure a profession. Establishing EPAs involves assumptions of what defines a profession in its tasks and what may not be covered by EPAs. 3 While a comprehensive set of EPAs, i.e., all relevant units of professional practice, may define a specialty or profession in their scope of practice, 8 a sole focus on clinical tasks induces the risk of reductionism. 54–57 University schooling for any profession is broader than vocational training, and defining everything that is fundamental in terms of EPAs should be open to criticism. 58,59

For postgraduate education, EPA frameworks have used different logics, i.e., with procedures as activities, diseases as a focus, functionalities of clinical practices, or a mix, 5 and some have suggested the inclusion of diverse stakeholders such as allied health professionals and patients 60 in their development.

Some EPAs have led to criticism as they did not seem to meet the criteria of being an EPA 57,61 and guidelines for EPA development, 5,14 as well as tools to evaluate their quality 62,63 have been developed. This issue will require continued attention.

Adapting curricula to prepare learners for EPAs

The role of preclinical and basic science education in EPA-based curricula is largely uncharted territory. EPAs constitute professional practice, and entrustment decisions for EPAs in the preclinical environment do not make sense, but basic science knowledge and specific skills practiced with simulation may be a prerequisite for entrusting learners with critical clinical tasks. For example, curricula may adapt to incorporate EPAs in clerkships, interspersed with classroom block periods to enable basic sciences teaching or boot camps to be geared to the upcoming clerkship and their EPAs. 64 Or a curriculum may start with an EPA (e.g., certification for emergency health professions technician) informing basic sciences and thus creating vertical integration from the very start. 65

Elaborating programs of assessment and the use of technology

To make entrustment decisions, institutions need to implement robust programs of assessment; efficiently collect data in clinical learning environments; and process, analyze, and display those data. 66 Validity evidence is emerging to support individual workplace-based and other assessment tools informing summative entrustment decisions. 67 However, research is needed to better understand how to integrate different sources of information, including the perspectives of patients and interprofessional team members. Taken together, these sources can provide a comprehensive picture of a learner’s progression over time and inform entrustment decisions. While the primary aim is to support learner development and progression, summative entrustment decisions should have consequences for clinical training, such as changes to rotational schedules or the organization of clinical responsibilities, 68 and may inform program and specialty quality measures.

We also need to better understand how to use technology and learning analytics to integrate workplace-based assessment in complex clinical learning environments. While tools, such as e-portfolios and mobile apps, are emerging, 69–73 much work needs to be done. One fundamental approach to filling these gaps is to use implementation science, design science, and even artificial intelligence to facilitate adoption of these tools within complex learning environments. These approaches provide a lens focused on understanding stakeholder needs, overcoming barriers to implementation, maximizing the use of facilitators, processing information available in the workplace better for feedback and assessment, and building models that, with local flexibility and adaptation, can be used to guide large-scale implementation across many different clinical environments within and across institutions. 34,74

Meanwhile, teacher portfolios have also gained popularity. 75,76 When multiple clinical teachers evaluate multiple clinical trainees, theoretically the same database may serve both populations. Teachers may benefit from benchmarking their ratings against colleagues, which could serve as source for faculty development and build their own education portfolio.

Articulating the validity argument for EPA-based assessments: Emphasizing educational and clinical outcomes

Arguably, one of the most pressing questions for educators is whether the shift toward EPA-based assessment will help us to make more informed decisions regarding our learners’ progression related to educational outcomes and whether these translate into better patient outcomes. To answer this question, we must articulate the validity argument for EPA-based assessments in the context of the decisions made about learners and examine the evidence regarding the intended and unintended consequences of these assessments on institutions, as well as learners, faculty, and society. 77 By implementing an EPA-based assessment system, are we actually making more informed decisions about our learners’ progression toward unsupervised practice? Are learners receiving enhanced feedback on their performance that better prepares them for practice? Do we observe fewer adverse events for learners who have been entrusted with critical tasks after a careful entrustment decision-making process? Not only do we need to consider the validity argument as a whole for EPA-based assessments, but we must also more deeply examine the argument within and across specialties as we know that validity is not a property of the EPA assessment tool itself, but is context bound. For example, several authors have recently studied how CBME outcome frameworks with EPAs in the Canadian context were translated into entrustment decision-making procedures locally, and signaled threats to validity because of contextual and specialty limitations and needs. 78,79 Given that learner progression, entrustment, and promotion rest on EPA-based assessments, this critical work will require collective, multi-institutional effort.

In addition to considering the validity argument pertaining to the use of an EPA-based assessment system on our learners, we will also need to consider the impact on patients. Are patient outcomes changing as a result of our EPA-based assessment systems? There is a chain of relationships between education, learner assessment, and quality of care, fraught with potential threats to validity. Only a few studies have found credible correlations between education quality or learner competence and patient outcomes. 80–82 The assumption that CBME, in this case with EPAs, leads to better health care remains to be confirmed. 83,84 For instance, more research needs to be done to understand how resident-sensitive quality measures 85 or measures of adverse events in patients relate to assessments based on EPAs and other competency models.

Optimizing summative entrustment decision making

While formative, ad hoc entrustment decisions happen naturally in the course of clinical care, summative entrustment decision making is unfamiliar to many. Developing and examining the validity argument underpinning the integration of various individual assessments can contribute to a more programmatic approach to summative decisions on learner progress. These types of summative entrustment decisions are ideally made by teams. 6,84,86 Clinical competency committees or entrustment committees in undergraduate and postgraduate programs 87–90 need clear, if possible evidence-based, guidelines on how to optimize team decision making in different contexts. Validity evidence must be gathered to support and defend these crucial decisions. Early work has begun; much lies ahead. 86,91

Agenda for Development and Research at the Macro Level of Regional, National, and International Stakeholders

At the macro level, regional, national, and international stakeholders have an interest in ensuring that health professions education training programs produce graduates that are prepared to provide high-quality, safe, and effective patient care to meet the needs of society. EPAs have the potential to provide a useful competency model at the macro level with certifications for EPAs, as a reflection of trusted scope of practice, indicating that competency standards have been met at transition points in the educational continuum. Transitions occur vertically as an individual moves through preclinical, clinical, postgraduate training, and continuing professional development. Transitions also occur horizontally as individuals move across regional or national regulatory jurisdictions. For example, medical licenses are usually accepted within jurisdictions, but often not between them, because distinct states, provinces, and countries do not always mutually accept the status and inherent capabilities of the holders of these licenses. EPAs have the potential to inform these horizontal transitions. In fact, the World Health Organization has proposed a general framework of competences in health care, to prepare for specified “practice activities,” an approach inspired by EPAs. 52 To understand how EPAs could be used in this context, there is a need for further development of EPAs as a competency model for CBME, especially related to the implementation, utility, and translatability of EPAs at the national and international levels. These issues constitute complex and challenging problems that, from a research perspective, may require different approaches: conceptual studies that generate and prioritize intelligent and creative solutions, implementation studies, and effect studies, first focused on feasibility and acceptability and subsequently on quality of outcomes. What significance can EPAs have across the continuum of education and practice? And across disciplines and professions? How do EPAs affect legal and financial privileging?

Using EPAs across the educational continuum and into practice

Achievement of a true continuum of health professions education requires open and honest transmission of developmentally focused data for each trainee from undergraduate to graduate programs and eventually into continuing professional development. An EPA framework could provide the infrastructure for learner handovers during these transition periods. 90,92,93 Transition periods, however, touch upon the interplay of EPAs, regulatory bodies, and funding models. They have the potential to reveal misalignment between stakeholder perspectives and priorities, and variations in implementation and interpretation of EPAs due to context-specific applications within unique learning environments. These transition periods deserve additional focus and attention as we set priorities for the agenda.

In addition, EPAs provide promising standards for specialties to use in initial certification and licensure of graduates, especially to support competency-based, time-variable progression through transitions in training. 94–96 However, current training program structures and funding models are not designed for time-variable progression. Furthermore, we will need to understand how EPA-based assessment systems and traditional high-stakes assessments will fit together for issues of certification and licensure. Future research will need to focus on addressing barriers to time-variable progression, including their funding models, licensing criteria, and certification procedures. In addition, practical issues related to the management of clinical service schedules and hospital credentialing will also need to be better understood. This type of research will require investiture from national accrediting bodies and funding organizations including government and regulatory agencies. The possibility of scaling up small but successful projects such as the Education in Pediatrics Across the Continuum (EPAC) pilot program, 94,95 to create a true competence-based (not time-based) continuum focused on growth, warrants further study. In addition, the use of both EPAs and Milestones for competency development, as is the case in North America, poses important questions to investigate about how these competency models relate as well as how they can, and perhaps cannot, coexist. 97,98

Acknowledging EPAs as units of practice across systems

EPAs have been identified, defined, and described for specific programs in single institutions. Over time, national specialty associations have created frameworks of EPAs for all programs within a given specialty. 99 The specific skill sets of graduates from such programs can be more easily and precisely defined, potentially facilitating an individual’s ability to transfer both within and across different specialty programs within all institutions of that nation. To make such transfers a reality will require nationally coordinated efforts to build EPA frameworks and agree that summative entrustment decisions or STARs (Statements of Awarded Responsibility 100) would be accepted across institutions.

One further step is international alignment; however, this will be more complicated because countries hold different criteria for unsupervised practice. While in the Western world, EPAs for medical graduates focus on “entering residency” and working under direct or indirect supervision, in many other countries, the expectation is that medical graduates are ready to deliver unsupervised primary care in rural areas. While most EPAs may be the same, the level of expected autonomy differs and additional research and development in this area will be required.

Acceptance of readiness for unsupervised practice by all relevant stakeholders

Licensure and certification for acknowledged health professionals are tied to legal privileges, responsibilities, and liabilities. Valid summative entrustment decisions for EPAs, made by well-informed clinical competency committees, might provide not only a solid foundation for licensure and certification but also a basis for granting autonomy for a limited but growing scope of practice. Restricted forms of autonomy, responsibility, liability, and privileging for billing of patients before completion of training should be explored, with attention to legal, ethical, economic, and patient safety considerations. Building on a Continuous Professional Development model, EPAs could be collected even after training to build a portfolio that supports the practitioner’s scope of practice. Clearly, the infrastructure to enable these modifications must be developed.

Sharing EPAs across disciplines and professions

While medical specialties typically develop their own frameworks of EPAs, some EPAs may transcend the boundaries of disciplines or professions as transdisciplinary EPAs (e.g., ultrasound imaging, 101 colonoscopy 102). A national project in the Netherlands defines EPAs for a variety of nonuniversity health care workers, to create a flexible workforce that allows for switches between professions serving both flexible career paths and a better response to societal health care needs. 103 Such initiatives require national efforts, and international efforts are emerging. 52

Implementing EPAs for continuing professional development and recertification

As EPAs are units of professional practice that reflect what practitioners do, they define scopes of practice. For practitioners who have completed formal training, EPAs could serve as the framework to support continuing learning and skill development to achieve higher levels of proficiency and mastery. This concept could apply to recent graduates whose readiness for unsupervised practice is called into question and would benefit from supervision in the early stage of practice. 104 EPAs provide a more granular conceptualization of the areas in which a new practitioner may yet have gaps in their clinical knowledge and skills to continue to follow their continued progression under supervision once in practice. This concept could also apply to mid- and late-career practitioners who need to show maintenance of existing skills and learn new skills. These applications of EPAs deserve further study, especially from the perspectives of entrustment, supervision, and patient safety.

For EPAs that involve serious risks, and perhaps for all EPAs, including an expiration after a specified period of nonpractice may be advisable. 5,8,105 Adding the privilege of new EPAs may compensate for loss of EPAs not practiced. This approach can result in a dynamic portfolio that defines a professional’s competence at any given time. 106 However, a system that includes EPAs for maintenance of certification could have significant implications, as it would require an administrative system that in our current context may seem impractical or unacceptable. While this idea presents a challenge, the benefits of such an approach to macro-level stakeholders, including patients, of ensuring high-quality patient care could enhance the public trust in health care providers. Thus, developing a better understanding of what is needed to maintain permission to practice in a scope defined by EPAs in a feasible way is important.

Discussion

Since the development of the concept of EPAs, there has been an impressive effort at local, national, and international levels to understand how EPA-based frameworks could be used to guide health professions education. At this point, EPAs have outgrown their initial characterization as a novelty and now are steadily entering the routine language and practice of health professions education. At first, many editorials and commentaries were devoted to introducing the concept for specific domains and disciplines; now curricula are being reconstructed to align with EPAs. Implementation work and outcome studies are just beginning, as very few programs have, as of mid-2021, graduates who have completed an EPA-based program. 107 As impressive as this work is, the effort so far raises more questions than it answers for both the future of EPAs and the future of CBME more broadly. In this paper, we summarized many rich and important avenues for ongoing development and investigation. To reach their full potential, health professions educators must make a convincing case for institutions, faculty, and learners to gain support for EPA-based CBME. To do so will require, in economic terms, understanding the return on investment (ROI). The ROI for learners and faculty is linked to whether the EPA framework and CBME system lead to better educational outcomes for our learner and, subsequently, better patient outcomes. More research to understand how to design and implement EPA-based CBME so that it is intuitive and has value to help inform teaching and learning are key strategies to enhance meaning. Furthermore, implementing the programs of assessment needed to support EPA-based CBME will require both institutional and macro-level investment of financial and other resources. Such investments owe a research and development effort that provides answers to fundamental questions of educational outcomes, quality of care, patient safety, system efficiencies, financial parameters (e.g., cost savings, revenue generation) and other important patient outcomes.

There is much work ahead, and like many educators, the authors too look forward to being surprised and delighted by upcoming studies and literature.

References

1. Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: A scoping review. Med Educ. 2019; 53:766–777.
2. O’Dowd E, Lydon S, O’Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. Med Educ. 2019; 53:234–249.
3. Brydges R, Boyd VA, Tavares W, et al. Assumptions about competency-based medical education and the state of the underlying evidence: A critical narrative review. Acad Med. 2021; 96:296–306.
4. Gruppen L, Frank JR, Lockyer J, et al.; ICBME Collaborators. Toward a research agenda for competency-based medical education. Med Teach. 2017; 39:623–630.
5. ten Cate O, Taylor D. The recommended description of an entrustable professional activity (AMEE guide no. 140) [published online ahead of print November 9, 2020]. Med Teach. doi:10.1080/0142159X.2020.1838465.
6. ten Cate O, Hart D, Ankel F, et al.; International Competency-Based Medical Education Collaborators. Entrustment decision making in clinical training. Acad Med. 2016; 91:191–198.
7. Dijksterhuis MG, Voorhuis M, Teunissen PW, et al. Assessment of competence and progressive independence in postgraduate clinical training. Med Educ. 2009; 43:1156–1165.
8. ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE guide no. 99. Med Teach. 2015; 37:983–1002.
9. ten Cate O, Snell L, Carraccio C. Medical competence: The interplay between individual ability and the health care environment. Med Teach. 2010; 32:669–675.
10. ten Cate O. AM last page: What entrustable professional activities add to a competency-based curriculum. Acad Med. 2014; 89:691.
11. Gingerich A, Daniels V, Farrell L, Olsen SR, Kennedy T, Hatala R. Beyond hands-on and hands-off: Supervisory approaches and entrustment on the inpatient ward. Med Educ. 2018; 52:1028–1040.
12. McGaghie W, Miller G, Sajid A, Telder T. Competency-Based Curriculum Development in Medical Education—An Introduction. Geneva, Switzerland: World Health Organization; 1978.
13. Powell DE, Carraccio C. Toward competency-based medical education. N Engl J Med. 2018; 378:3–5.
14. Taylor D, Park YS, Smith C, ten Cate O, Tekian A. Constructing approaches to entrustable professional activity development that deliver valid descriptions of professional practice. Teach Learn Med. 2021; 33:89–97.
15. ten Cate O. Entrustment decisions: Bringing the patient into the assessment equation. Acad Med. 2017; 92:736–738.
16. Mohr J, Batalden P, Barach P. Integrating patient safety into the clinical microsystem. Qual Saf Health Care. 2004; 13(suppl 2):ii34–ii38.
17. ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013; 5:157–158.
18. ten Cate O, Schwartz A, Chen HC. Assessing trainees and making entrustment decisions: On the nature and use of entrustment-supervision scales. Acad Med. 2020; 95:1662–1669.
19. ten Cate O, Carraccio C, Damodaran A, et al. Entrustment decision making: Extending Miller’s Pyramid. Acad Med. 2021; 96:199–204.
20. Postmes L, Tammer F, Posthumus I, Wijnen-Meijer M, van der Schaaf M, ten Cate O. EPA-based assessment: Clinical teachers’ challenges when transitioning to a prospective entrustment-supervision scale [published online ahead of print December 11, 2020]. Med Teach. doi:10.1080/0142159X.2020.1853688.
21. Hirsh DA, Holmboe ES, ten Cate O. Time to trust: Longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014; 89:201–204.
22. van Enk A, ten Cate O. “Languaging” tacit judgment in formal postgraduate assessment: The documentation of ad hoc and summative entrustment decisions. Perspect Med Educ. 2020; 9:373–378.
23. Crossley J, Jolly B. Making sense of work-based assessment: Ask the right questions, in the right way, about the right things, of the right people. Med Educ. 2012; 46:28–37.
24. ten Cate O, Regehr G. The power of subjectivity in the assessment of medical trainees. Acad Med. 2019; 94:333–337.
25. Gingerich A, Sebok-Syer SS, Larstone R, Watling CJ, Lingard L. Seeing but not believing: Insights into the intractability of failure to fail. Med Educ. 2020; 54:1148–1158.
26. Ginsburg S, van der Vleuten C, Eva KW, Lingard L. Hedging to save face: A linguistic analysis of written comments on in-training evaluation reports. Adv Health Sci Educ Theory Pract. 2016; 21:175–188.
27. Caro Monroig AM, Chen HC, Carraccio C, Richards BF, ten Cate O, Balmer DF. Medical students’ perspectives on entrustment decision-making in an EPA assessment framework [published online ahead of print November 24, 2020]. Acad Med. doi:10.1097/acm.0000000000003858.
28. Pinilla S, Kyrou A, Maissen N, et al. Entrustment decisions and the clinical team: A case study of early clinical students. Med Educ. 2021; 55:365–375.
29. ten Cate O, Chen HC. The ingredients of a rich entrustment decision. Med Teach. 2020; 42:1413–1420.
30. Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014; 19:435–456.
31. Gupta A, Watkins AC, Fahey TJ, Barie PS, Narayan M. Entrustable professional activities: Do general surgery residents trust them? J Surg Educ. 2020:77;520–526.
32. Englander R, ten Cate O. LICs and entrustable professional activities: A perfect match. Poncelet AN, Hirsh DA, eds. In: Longitudinal Integrated Clekrkships. Principles, Outcomes, Practical Tools, and Future Directions. North Syracuse, NY: Alliance for Clinical Educators; 2016:261–269.
33. Holmboe E, Ginsburg S, Bernabeo E. The rotational approach to medical education: Time to confront our assumptions? Med Educ. 2011; 45:69–80.
34. Stahl CC, Jung SA, Rosser AA, et al. Natural language processing and entrustable professional activity text feedback in surgery: A machine learning model of resident autonomy. Am J Surg. 2021; 221:369–375.
35. Schuwirth L, van der Vleuten C, Durning SJ. What programmatic assessment in medical education can learn from healthcare. Perspect Med Educ. 2017; 6:211–215.
36. Torre DM, Schuwirth LWT, Van der Vleuten CPM. Theoretical considerations on programmatic assessment. Med Teach. 2020; 42:213–220.
37. Schuwirth LWT, Van der Vleuten CPM. Programmatic assessment: From assessment of learning to assessment for learning. Med Teach. 2011; 33:478–485.
38. Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency-based workplace learning: When theory meets practice. BMC Med Educ. 2013; 13:123.
39. ten Cate O, Dahdal S, Lambert T, et al. Ten caveats of learning analytics in health professions education: A consumer’s perspective. Med Teach. 2020; 42:673–678.
40. Martin L, Sibbald M, Brandt Vegas D, Russell D, Govaerts M. The impact of entrustment assessments on feedback and learning: Trainee perspectives. Med Educ. 2020; 54:328–336.
41. Geraghty JR, Ocampo RG, Liang S, et al.; Core Entrustable Professional Activities for Entering Residency Pilot Program. Medical students’ views on implementing the core EPAs: Recommendations from student leaders at the core EPAs pilot institutions. Acad Med. 2021; 96:193–198.
42. ten Cate TJ, Kusurkar RA, Williams GC. How self-determination theory can assist our understanding of the teaching and learning processes in medical education. AMEE guide No. 59. Med Teach. 2011; 33:961–973.
43. Sasek C. Entrustable Professional Activities and Physicians Assistant Student Motivation. Dissertation. Lincoln., Nebraska: University of Nebraska; 2020.
44. Wijnen-Meijer M, Van der Schaaf M, Booij E, et al. An argument-based approach to the validation of UHTRUST: Can we measure how recent graduates can be trusted with unfamiliar tasks? Adv Health Sci Educ Theory Pract. 2013; 18:1009–1027.
45. McPhee RD, Canary HE. Structuration theory. Bruhn Jensen K, Craig RT, eds. In: The International Encyclopedia of Communication Theory and Philosophy. Hoboken, NJ: John Wiley & Sons, Inc; 2016:1–15.
46. Sebok-Syer SS, Chahine S, Watling CJ, Goldszmidt M, Cristancho S, Lingard L. Considering the interdependence of clinical performance: Implications for assessment and entrustment. Med Educ. 2018; 52:970–980.
47. Sebok-Syer SS, Pack R, Shepherd L, et al. Elucidating system-level interdependence in electronic health record data: What are the ramifications for trainee assessment? Med Educ. 2020; 54:738–747.
48. Nordquist J, Chan MK, Maniate J, Cook D, Kelly C, McDougall A. Examining the clinical learning environment through the architectural avenue. Med Teach. 2019; 41:403–407.
49. Leipzig RM, Sauvigne K, Granville LJ, et al. What is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs end-of-training entrustable professional activities for geriatric medicine. J Am Geriatr Soc. 2014; 62:924–929.
50. Deitte LA, Gordon LL, Zimmerman RD, et al. Entrustable professional activities: Ten things radiologists do. Acad Radiol. 2016; 23:374–381.
51. Chesbro SB, Jensen GM, Boissonnault WG. Entrustable professional activities as a framework for continued professional competence: Is now the time? Phys Ther. 2018; 98:3–7.
52. Fitzpatrick S. Global Competency Framework for Universal Health Coverage. Geneva, Switzerland: World Health Organization; 2021.
53. de Raad T, Kuilman L, Wiersma F, ten Cate O. The fate of EPAs after graduation. A survey study among graduated physician assistants. Unpublished.
54. Lombarts MJMH. Competence-based education misses the essence of the medical profession. Perspect Med Educ. 2015; 4:326–328.
55. Holmboe ES, Sherbino J, Englander R, Snell L, Frank JR; ICBME Collaborators. A call to action: The controversy of and rationale for competency-based medical education. Med Teach. 2017; 39:574–581.
56. Krupat E. Critical thoughts about the core entrustable professional activities in undergraduate medical education. Acad Med. 2018; 93:371–376.
57. Tekian A. Are all EPAs really EPAs? Med Teach. 2017; 39:232–233.
58. Hodges BD, Lingard L. The Question of Competence. Ithaca, NY: ILR Press/Cornell University Press; 2012.
59. Grant J. The incapacitating effects of competence: A critique. Adv Health Sci Educ Theory Pract. 1999; 4:271–277.
60. El-Haddad C, Damodaran A, McNeil HP, Hu W. A patient-centered approach to developing entrustable professional activities. Acad Med. 2017; 92:800–808.
61. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005; 39:1176–1177.
62. Post JA, Wittich CM, Thomas KG, et al. Rating the quality of entrustable professional activities: Content validation and associations with the clinical context. J Gen Intern Med. 2016; 31:518–523.
63. Taylor DR, Park YS, Egan R, et al. EQual, a novel rubric to evaluate entrustable professional activities for quality and structure. Acad Med. 2017; 92(11 suppl):S110–S117.
64. ten Cate O, Graafmans L, Posthumus I, Welink L, van Dijk M. The EPA-based Utrecht undergraduate clinical curriculum: Development and implementation. Med Teach. 2018; 40:506–513.
65. Brenner J, Bird J, Ginzburg SB, et al. Trusting early learners with critical professional activities through emergency medical technician certification. Med Teach. 2018; 40:561–568.
66. Hauer KE, O’Sullivan PS, Fitzhenry K, Boscardin C. Translating theory into practice: Implementing a program of assessment. Acad Med. 2018; 93:444–450.
67. Weller JM, Castanelli DJ, Chen Y, Jolly B. Making robust assessments of specialist trainees’ workplace performance. Br J Anaesth. 2017; 118:207–214.
68. ten Cate O. Entrustment as assessment: Recognizing the ability, the right, and the duty to act. J Grad Med Educ. 2016; 8:261–262.
69. George BC, Bohnen JD, Schuller MC, Fryer JP. Using smartphones for trainee performance assessment: A SIMPL case study. Surg (United States). 2020; 167:903–906.
70. Warm EJ, Kinnear B, Kelleher M, Sall D, Schauer DP, Friedstrom S. Variation in entrustment when sharing a single assessment system between university- and community-based residency programs: A comparison. Acad Med. 2020; 95:590–598.
71. Kelleher M, Kinnear B, Sall D, et al. A reliability analysis of entrustment-derived workplace-based assessments. Acad Med. 2020; 95:616–622.
72. Young JQ, Sugarman R, Schwartz J, McClure M, O’Sullivan PS. A mobile app to capture EPA assessment data: Utilizing the consolidated framework for implementation research to identify enablers and barriers to engagement. Perspect Med Educ. 2020; 9:210–219.
73. Young JQ, McClure M. Fast, easy, and good: Assessing entrustable professional activities in psychiatry residents with a mobile app. Acad Med. 2020; 95:1546–1549.
74. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015; 10:53.
75. Dalton CL, Wilson A, Agius S. Twelve tips on how to compile a medical educator’s portfolio. Med Teach. 2018; 40:140–145.
76. Kuhn GJ. Faculty development: The educator’s portfolio: Its preparation, uses, and value in academic medicine. Acad Emerg Med. 2004; 11:307–311.
77. Kane MT. Validating the interpretations and uses of test scores. J Educ Meas. 2013; 50:1–73.
78. Tavares W, Rowland P, Dagnone D, McEwen L, Billett S, Sibbald M. Translating outcome frameworks to assessment programs: Implications for validity. Med Educ. 2020; 54:932–942.
79. Melvin L, Rassos J, Stroud L, Ginsburg S. Tensions in assessment: The realities of entrustment in internal medicine. Acad Med. 2020; 95:609–615.
80. Bansal N, Simmons KD, Epstein AJ, Morris JB, Kelz RR. Using patient outcomes to evaluate general surgery residency program performance. JAMA Surg. 2016; 151:111–119.
81. Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA. 2009; 302:1277–1283.
82. Epstein A, Nicholson S, Asch D. The production of and market for new physicians’ skill. Am J Health Econ. 2016; 2:41–65.
83. Gruppen LD, ten Cate O, Lingard LA, Teunissen PW, Kogan JR. Enhanced requirements for assessment in a competency-based, time-variable medical education system. Acad Med. 2018; 93(3 suppl):S17–S21.
84. Touchie C, Kinnear B, Schumacher D, et al. On the validity of summative entrustment decisions. Unpublished.
85. Schumacher DJ, Holmboe E, Carraccio C, et al. Resident-sensitive quality measures in the pediatric emergency department: Exploring relationships with supervisor entrustment and patient acuity and complexity. Acad Med. 2020; 95:1256–1264.
86. Smit MP, de Hoog M, Brackel HJL, ten Cate O, Gemke RJBJ. A national process to enhance the validity of entrustment decisions for Dutch pediatric residents. J Grad Med Educ. 2019; 11(suppl 4):158–164.
87. Hauer KE, Cate OT, Boscardin CK, et al. Ensuring resident competence: A narrative review of the literature on group decision making to inform the work of clinical competency committees. J Grad Med Educ. 2016; 8:156–164.
88. Garber AM, Ryan MS, Santen SA, Goldberg SR. Redefining the acting internship in the era of entrustment: One institution’s approach to reforming the acting internship. Med Sci Educ. 2019; 29:583–591.
89. Moeller JJ, Warren JB, Crowe RM, et al. Developing an entrustment process: Insights from the AAMC CoreEPA pilot. Med Sci Educ. 2020; 30:395–401.
90. Brown DR, Warren JB, Hyderi A, et al.; AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group. Finding a path to entrustment in undergraduate medical education: A progress report from the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group. Acad Med. 2017; 92:774–779.
91. Kinnear B, Warm EJ, Hauer KE. Twelve tips to maximize the value of a clinical competency committee in postgraduate medical education. Med Teach. 2018; 40:1110–1115.
92. Warm EJ, Englander R, Pereira A, Barach P. Improving learner handovers in medical education. Acad Med. 2017; 92:927–931.
93. Morgan HK, Mejicano GC, Skochelak S, et al. A responsible educational handover: Improving communication to improve learning. Acad Med. 2020; 95:194–199.
94. Andrews JS, Bale JF Jr, Soep JB, et al.; EPAC Study Group. Education in Pediatrics Across the Continuum (EPAC): First steps toward realizing the dream of competency-based education. Acad Med. 2018; 93:414–420.
95. Murray KE, Lane JL, Carraccio C, et al.; Education in Pediatrics Across the Continuum (EPAC) Study Group. Crossing the gap: Using competency-based assessment to determine whether learners are ready for the undergraduate-to-graduate transition. Acad Med. 2019; 94:338–345.
96. Schumacher DJ, West DC, Schwartz A, et al.; Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network General Pediatrics Entrustable Professional Activities Study Group. Longitudinal assessment of resident performance using entrustable professional activities. JAMA Netw Open. 2020; 3:e1919316.
97. Franzen D, Kost A, Knight C. Mind the gap: The bumpy transition from medical school to residency. J Grad Med Educ. 2015; 7:678–680.
98. Thibodeau L, Beeson M, Bhat R, et al. Integration of entrustable professional activities with the milestones for emergency medicine residents. West J Emerg Med. 2019; 20:35–42.
99. de Graaf J, Bolk M, Dijkstra A, van der Horst M, Hoff RG, ten Cate O. The implementation of entrustable professional activities in postgraduate medical education in the Netherlands: Rationale, process and current status. Acad Med. 2021; 96(7 suppl):S29–S35.
100. ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82:542–547.
101. Leidi A, Rouyer F, Marti C, Reny JL, Grosgurin O. Point of care ultrasonography from the emergency department to the internal medicine ward: Current trends and perspectives. Intern Emerg Med. 2020; 15:395–408.
102. Wells K, Fleshman J. Screening colonoscopy should be available to all. JAMA Surg. 2019; 154:636.
103. NN. College for health care and hospital training—Flexlevel project to advance care education trajectories [in Dutch]. CZO Flexlevel https://www.czoflexlevel.nl. Accessed March 23, 2021.
104. Duijn C, Bok H, ten Cate O, Kremer W. Qualified but not yet fully competent: Perceptions of recent veterinary graduates on their day-one skills. Vet Rec. 2020; 186:216.
105. Mehta NB, Hull AL, Young JB, Stoller JK. Just imagine: New paradigms for medical education. Acad Med. 2013; 88:1418–1423.
106. ten Cate O, Carraccio C. Envisioning a true continuum of competency-based medical education, training and practice. Acad Med. 2019; 94:1283–1288.
107. Wiersma F, Berkvens J, ten Cate O. Flexibility in individualized, competency-based workplace curricula with EPAs: Analyzing four cohorts of physician assistants in training. Med Teach. 2017; 39:535–539.
Copyright © 2021 by the Association of American Medical Colleges