Over the past 10 years, the momentum to implement competency-based curricula in medical education has reached an important turning point—theory is rapidly translating to practice.1 This transformation has been catalyzed by the development of entrustable professional activities (EPAs). Originally described by Olle ten Cate2 in the context of the transition from graduate medical education (GME) to independent practice, EPAs represent “tasks or responsibilities to be entrusted to the unsupervised execution by a trainee once he or she has attained sufficient specific competence.” Applying this construct to undergraduate medical education (UME) offers a framework to describe the tasks that graduating medical students might be expected to carry out without direct supervision on entering residency.3
The EPA framework complements other constructs in competency-based medical education, specifically competencies and milestones.4 Competencies describe performance as the abilities of a learner, and milestones mark a learner’s progress toward demonstrating competency. In contrast, an EPA describes a unit of work.4,5 These constructs are inextricable—an individual must develop the necessary competencies relevant to a particular EPA, and she must apply those competencies in an integrated manner to the activity at hand in order to be entrusted. A supervisor may infer that a learner possesses the requisite competencies if she performs a given task in a manner sufficient to be entrusted to complete that task without direct supervision. If a learner is not deemed to be entrustable, however, the supervisor can refer to the relevant competencies and developmental milestones to prioritize her goals for additional learning and/or practice.6 Competencies and milestones are also helpful in describing the development of a learner’s broad capabilities; entrustment to perform an EPA indicates a learner’s readiness to perform that task unsupervised.
Residency program directors increasingly have expressed concern that some medical school graduates are not prepared for some of the patient care responsibilities expected of them in the first weeks of residency.7–9 Historically, the UME community has not agreed on a core set of tasks that program directors might expect all graduates to be able to perform without direct supervision on the first day of residency. Formal entrustment from any UME program should signify that a learner can perform the requisite patient care activities with a certain level of independence. EPAs can serve as an outline for what a UME program must, at minimum, teach for a learner to be successful on day one of residency and what clinical supervisors in medical school must entrust their learners to be able to do without direct supervision in the next phase of training.
The Core EPAs for Entering Residency
Given both the increased recognition of the gaps in entering residents’ performance and the recent implementation of competency-based approaches in GME,10 the Association of American Medical Colleges (AAMC) commissioned a project to focus on improving the transition from UME to GME.11 In January 2013, the AAMC convened an experienced drafting panel with representatives from across the continuum from UME through practice, which included distinguished clinical medical educators, a student, a resident, and a basic scientist. This group was charged with identifying a short list of integrated activities that medical school graduates might be expected to perform “without direct supervision” on day one of residency. A reactor panel provided feedback on the drafting panel’s efforts. List 1 presents the resulting 13 Core EPAs for Entering Residency (Core EPAs) that were published in May 2014.5 Their publication served as a call to action to the medical education community.
List 1The 13 Core Entrustable Professional Activities for Entering Residency, 20145
- Gather a history and perform a physical examination
- Prioritize a differential diagnosis following a clinical encounter
- Recommend and interpret common diagnostic and screening tests
- Enter and discuss orders/prescriptions
- Document a clinical encounter in the patient record
- Provide an oral presentation of a clinical encounter
- Form clinical questions and retrieve evidence to advance patient care
- Give or receive a patient handover to transition care responsibility
- Collaborate as a member of an interprofessional team
- Recognize a patient requiring urgent or emergent care and initiate evaluation and management
- Obtain informed consent for tests and/or procedures
- Perform general procedures of a physician
- Identify system failures and contribute to a culture of safety and improvement
About the Core EPA Pilot
Since the Core EPAs were released, educators have been grappling with the practical and theoretical implications of a competency-based medical education system that hinges on entrustment. In an effort to galvanize the medical education community and foster the broad implementation of the Core EPA framework in UME, in August 2014, the AAMC initiated a pilot consisting of teams from 10 medical schools in the United States, representing a diversity of institution types (e.g., new and established, research intensive and community based, private and public, single campus and multicampus). Table 1 lists the 10 schools working collaboratively on this pilot.
In this article, we provide a progress report on behalf of the pilot schools. We present our organizational structure and the early results of our collaborative efforts to provide guidance to other institutions planning to implement the Core EPA framework.
The goal of the pilot is to optimize safe and effective patient care by ensuring that each graduate from our medical schools is prepared for the core initial duties of an entering resident. To accomplish this goal, we are designing and implementing educational systems that use the Core EPA framework to achieve two aims. First, we want to develop objective and reliable summative determinations of each student’s readiness to perform the 13 Core EPAs that are based on multimodal evidence of EPA-specific knowledge, skills, and attitudes as well as work habits that demonstrate fundamental trustworthiness. Second, we want to share lessons learned regarding the Core EPA framework to optimize this approach and foster its propagation to other UME programs.
A team from each of the 10 medical schools participating in the pilot first assembled in Washington, DC, in October 2014. Early efforts focused on establishing a vision, team building, clarifying terminology, and fostering a shared understanding of the EPA construct. This work included an in-depth review of existing documents previously prepared by the drafting group and reactor panel. We then assigned three to four schools to work together on each Core EPA.
The pilot schools implemented the Core EPA framework for use with the entering class of 2015. The sequencing of the Core EPA curricula, assessments, and entrustment decisions varies depending on each school’s curriculum design. However, efforts to align approaches are under way. Some schools have implemented programming for more senior students as well. An iterative process for continuous improvement will inform refinements as each successive class begins the curriculum.
Importantly, we are not rushing to render formal entrustment decisions. When the entering class of 2015 approaches graduation in 2019, each school will pilot making summative entrustment decisions for each of its students in accordance with the guiding principles described below. These decisions are meant to inform the feasibility of implementing the Core EPAs construct in UME programs. As such, they will be theoretical in nature and will therefore not be included in the medical student performance evaluations or shared with residency programs. Deidentified entrustment results will be reported in aggregate to the AAMC and shared with the medical education community through presentations and publications. In this pilot phase, we do not expect that all students will be deemed entrustable in all the Core EPAs.
Reporting of entrustment decisions will include an open discussion of the limitations of this approach as well as outcomes related to individual students’ performance. Pilot schools will be asked to report the percentages of graduates who have been entrusted in each Core EPA; and for those students who did not attain entrustment, schools will report whether those individuals were not included in the pilot, whether insufficient evidence was gathered to inform a summative decision (a system implementation issue), or whether the individual’s performance was deemed inadequate to support entrustment (individual performance concerns). This deidentified information will also be shared with the medical education community to determine whether the Core EPAs need modification, whether additional curriculum changes are needed to support students’ attainment of entrustment, and which assessment approaches are most accurate and sustainable.
Each pilot school assembled a core team to include a member from the dean’s office, a residency program director, a clerkship director, and a fourth member with expertise in curriculum design, assessment, or faculty development. Additionally, each school engaged other key faculty and learners in local pilot activities. This team composition was deliberately designed to include representation from clinical educators in UME and GME and to enhance engagement and implementation at each school.
Interinstitutional teams focused on each of the Core EPAs and on four main concepts:
- Formal entrustment: As we consider the desired educational outcome of entrustment, are novel curriculum and assessment practices needed? Who will make entrustment decisions? When and upon what evidence? Are entrustment decisions time-limited? How will residency program directors view the credibility of entrustment decisions?
- Assessment: Can all Core EPAs be assessed in a reliable, valid, and cost-effective manner? How is formative feedback used in the process of entrustment? Who provides the assessment? How might students, residents, patients, and other health care professionals be engaged in the assessment process?
- Curriculum development: What are the learning experiences required to inform entrustment decisions? When in the curriculum should they occur? How should the Core EPAs be sequenced?
- Faculty development: What are effective practices to prepare faculty to perform competency assessments, to provide formative feedback linked to the Core EPAs, and to participate in decisions to entrust learners?
The ultimate question, beyond the time frame and scope of the pilot, is whether the implementation of the Core EPA framework in UME will result in improved quality of care for patients.
Pilot Findings to Date
We, as representatives of the pilot schools (clerkship directors, residency program directors, administrators, and faculty), have delved deeply into the Core EPA framework and the descriptions in the AAMC Core EPA guides. These guides consist of two manuals. The first is for curriculum developers and contains details on how the drafting panel mapped the Core EPAs to domains of competence. The second is for frontline faculty and learners and describes each Core EPA with associated narrative descriptions and clinical vignettes.5
Now, in our second year of work, we have begun to develop novel approaches and thoughtful frameworks to help other schools implement this program. Importantly, we are advocating for a systems-based approach to implementing the Core EPAs in UME. Each school must determine how teaching and assessing the Core EPAs can be integrated throughout its unique curriculum and must consider what resources will be necessary to support meaningful entrustment decision making. Below, we describe our current, shared understanding in each of the four concept areas listed above.
Early discussions have focused on how the Core EPA framework differs from other assessment methods. Fundamentally, entrustment is a workplace-based construct. The Core EPAs are an intuitive approach to determining competency,11 since clinical supervisors routinely make implicit, ad hoc entrustment decisions throughout the process of care delivery. However, such ad hoc entrustment decisions are based on multiple factors, only some of which are under the control of the learner.12,13 Amplifying this implicit entrustment process with a formal structure may foster a more systematic, informative, reliable, and ultimately transferable method of credentialing learners for unsupervised practice.
In addition to performing the key functions and achieving the requisite competencies of a given Core EPA, to be entrusted demands that each learner be trustworthy.5,12 Trustworthiness includes knowing one’s limits (discernment), communicating honestly (truthfulness), and fulfilling one’s commitments (conscientiousness).14 Because of concerns about declaring a developing learner to be untrustworthy, specific feedback regarding these elements should be provided using alternative language. Also, it is important to acknowledge the impact of stressors in the working and learning environment on an individual’s behavior and to foster appropriate responses among learners. The residency program directors participating in the pilot repeatedly endorsed the primacy of this aspect of the entrustment process. Explicit measures of the components of trustworthiness, and opportunities for coaching, should be a central aspect of this framework.
Because frontline faculty are in a position to observe students in clinical environments, they must be commissioned to record their observations in a systematic fashion.12 In GME, discipline-specific expectations have been developed by those with similar clinical backgrounds (i.e., specialty societies, residency review committees, certifying boards). UME faces the challenge of establishing shared expectations for the Core EPAs across clinical disciplines. The AAMC Core EPA guides5 have started that process, but much remains to be described. We must systematically determine the typical developmental pathways for each Core EPA as well as the appropriate level of supervision required as learners make their way toward entrustment. We must explicitly define the body of evidence needed to support formal entrustment decisions, including the levels of case complexity and acuity, and build systems to generate that evidence. Workplace-based evidence will be essential, but we need to develop assessment tools that can be used easily by frontline faculty and residents. As part of the pilot, we are currently exploring the use of the supervision scale adopted for UME by Chen and colleagues15 (see List 2), with the inclusion of contextual information, as a tool to capture workplace-based assessments. We envision a process by which multiple forms of assessment evidence, including ad hoc entrustment decisions in the workplace, will be used to render summative entrustment decisions.15
List 2Proposed Undergraduate Medical Education Entrustment and Supervision Scalea
- 1. Not allowed to practice EPA
- a. Inadequate knowledge/skill (e.g., does not know how to preserve sterile field); not allowed to observe
- b. Adequate knowledge, some skill; allowed to observe
- 2. Allowed to practice EPA only under proactive, full supervision
- a. As coactivity with supervisor
- b. With supervisor in room ready to step in as needed
- 3. Allowed to practice EPA only under reactive/on-demand supervision
- a. With supervisor immediately available, all findings double checked
- b. With supervisor immediately available, key findings double checked
- c. With supervisor distantly available (e.g., by phone), findings reviewed
- 4. Allowed to practice EPA unsupervised
- 5. Allowed to supervise others in practice of EPA
aBased on the Graduate Medical Education and Proposed Undergraduate Medical Education Entrustment and Supervision Scale developed by Chen et al.15 Reproduced with permission. EPA indicates entrustable professional activity.
A systematic approach is needed to embed the teaching and assessment of the Core EPAs throughout the UME curriculum. Although full entrustment may not be achieved until students approach graduation, learners need the opportunity to develop their skills over time. Through early experiences, students should build the relevant competencies, which they later can apply in the clinical workplace to successfully care for patients.
In many health systems, students have been relegated to a more peripheral role in patient care activities, so they may not have sufficient opportunities to participate in some of the Core EPAs.16 UME programs in collaboration with their associated health systems must ensure sufficient clinical experiences for students to develop competence. Clinical programs in UME are typically constructed to provide breadth of experience, with short periods on a variety of clinical teams. Yet, many faculty members are reticent to formally assess students with whom they have had only brief interactions. In addition to developing facile tools to capture episodic observations, we may find that a reorganization of clinical experiences to create longitudinal relationships will be needed to inform summative entrustment decisions.17
Building faculty knowledge and skills related to the Core EPA framework in UME will be imperative, including developing content that is essential for each Core EPA; methods to teach this material; techniques for direct observation and provision of feedback; assessment expertise to provide data that is accurate, timely, and standardized; appropriate documentation of performance; and expertise in the judicious review of evidence to render entrustment decisions. Various faculty roles will require differing levels of training, and a systematic program is needed to address those diverse needs throughout the UME curriculum.
We determined that implementation of the structures needed to promote the mastery of these core clinical skills for practice18 will require a significant collaborative effort across institutions. For this framework to have meaning, entrustment decisions must be recognized as learners transfer between institutions; a graduate who was entrusted at one school should meet the expectations of the residency program director at another. Consistent standards for performance across all medical schools will be required.
To support a cohesive approach across all schools, we recommend that institutions intending to use the Core EPA framework follow the guiding principles provided in List 3. Specific interventions and tools may vary somewhat by institution, but universal alignment with these guiding principles should generate more trust in each other’s entrustment decisions.
List 3Guiding Principles for Institutions Implementing the Core Entrustable Professional Activities for Entering Residency (Core EPAs) Framework
- Employ a systematic approach to map educational opportunities and assessments for each EPA
- Explicitly measure the attributes of trustworthiness in addition to the specific knowledge, skills, and attitudes required for each EPA
- Create a longitudinal view of each learner’s performance via, at minimum, aggregated performance evidence, and consider the added value of longitudinal relationships and formal coaching structures in informing entrustment decisions
- Gather multimodal performance evidence from multiple assessors about each learner for each EPA
- Include global professional judgments about the entrustment of each learner in the body of evidence that supports summative entrustment decisions
- Ensure a process for formative feedback along the trajectory to entrustment to provide opportunities for both remediation and potential acceleration of responsibilities
- Create a process to render and maintain formal entrustment decisions by a trained group (entrustment committee) that reviews performance evidence for each learner
- Ensure that each learner is an active participant in the entrustment process—aware of expectations, engaged in gathering and reviewing performance evidence, and generating individualized learning plans to attain entrustment
- Align formal entrustment decisions regarding individual learners with nationally established performance expectations, as currently described in the Core EPAs Curriculum Developer’s Guide5
Impact, Limitations, and Next Steps
The scope of the Core EPA pilot is quite broad, and many schools across the country are actively engaged in designing and implementing related programs.13,19–22 During recent meetings, we have sought to define how our collective efforts will advance the feasibility and generalizability of the pilot to extend beyond the work of individual schools.
The Core EPA pilot could have two potential levels of impact. First, as individual schools apply the Core EPAs as an organizing framework, they will create opportunities for students to learn and practice the EPAs and will implement low-stakes formative assessments.23 We expect this work to better prepare medical school graduates for the core responsibilities of entering residents, resulting in a significant improvement in the UME to GME transition. This level of impact can be accomplished through schools’ independent responses to the published Core EPA guides,5 supplemented by ongoing education from the AAMC about the pilot and the sharing of best practices.
Second, to escalate the entrustment construct to a higher level of impact, advancement into GME would become contingent on ensuring individuals’ readiness in a summative fashion (and perhaps in the future, in a time-independent manner). This process will require making formal entrustment decisions that are consistent and honored across institutions. Attaining this outcome for every student demands adequate learning opportunities, a structured process for assessment, a definition of the evidence necessary to inform such decisions, implementation of entrustment committees, processes for coaching and remediation, and a structure to enable an educational handover from UME to GME. We aspire to propel the pilot to this second level of impact.
National conversations have raised concerns about whether all 13 Core EPAs represent appropriate goals for UME. There is some debate about whether they all, as currently written, meet the definition of an EPA.24 Additionally, authentic student participation in some of the Core EPAs may be a challenge in many health systems. We are cognizant that our students may not be in a position to reach the same level of entrustment for every EPA. We suspect that final expectations for independent practice (e.g., Chen and colleagues’15 supervisory level 3a versus level 3b) need to be tailored to specific EPAs. Finally, the resources required to implement the Core EPA framework have not been described. The AAMC has provided significant resources to support our collaborative efforts. However individual schools have not received any direct funding from the AAMC, and individual school funding of efforts related to the Core EPA pilot has varied markedly. Reporting deidentified, school-specific results in light of variations in resource allocation will be an important aspect of our study of the feasibility and efficiency of implementing this framework.
Despite acknowledgment of these potential limitations, the pilot schools have agreed to adhere to the AAMC Core EPA guides5 as much as possible, and we urge other institutions to do so as well. We may be in the best position to collectively report significant feasibility challenges, which should prompt further conversation between UME and GME educators regarding reasonable expectations and result in evidence-driven improvements to the AAMC Core EPA guides over time. Premature departure from the current, published expectations of the pilot will reduce our ability to establish consistent national standards for entrustment that would be respected across institutions. Until our collective understanding of the Core EPA framework is better aligned, we caution against referencing a student’s Core EPA status in her residency applications.
Going forward, we plan to focus on the processes that enable formal entrustment decision making. We will continue to develop content and tools for specific Core EPAs, using approaches that are deliberately aligned with this overarching collective goal. We are in the process of establishing our plan for program evaluation. As we progress, we will define promising elements that we agree to implement across multiple institutions, and we will share our outcomes and tools.
Engaging the Broader Community
Very soon, a learning community—a group of educators from other institutions who provide additional expertise in the concepts of entrustment, assessment, curriculum development, and faculty development—will be established to enhance our original pilot team. Additionally, the AAMC supports a listserv of nearly 1,000 medical educators that has become a locus for the exchange of literature, questions, and ideas regarding entrustment and the Core EPAs. (For more information about this listserv, visit www.aamc.org/initiatives/coreepas/).
Throughout the five-year pilot, we will regularly report our progress to the larger medical education community with ongoing presentations at regional, national, and international education conferences, through publications, and via the listserv.
The Core EPAs offer a valuable framework to clarify the core clinical expectations of medical school graduates. This effort will enhance the transition from UME to GME with the ultimate goal of improving patient care. We are striving to capitalize on the expertise across institutions to tackle the challenges related to the implementation of this novel approach and to promote a unified national response. Thoughtful collaboration among medical educators will provide supervisors, students, and the public with more confidence that our graduates are prepared for their duties upon entering residency.
Acknowledgments: The authors thank the Association of American Medical Colleges, Robert Englander, Carol Aschenbrener, Maryellen Gusic, and all of the members of the Core Entrustable Professional Activities for Entering Residency pilot for their support, inspiration, and contributions to this article. The pilot team leaders include Jonathan M. Amiel, MD, Patrick Cocks, MD, Karin Esposito, MD, PhD, Michael Green, MD, ScM, Kimberly Lomis, MD, George C. Mejicano, MD, MS, Michael S. Ryan, MD, MEHP, Alex Stagnaro-Green, MD, MHPE, Margaret Uthman, MD, and Dianne Wagner, MD.
1. Irby DM, Cooke M, O’Brien BCCalls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med. 2010;85:220–227.
2. Ten Cate ONuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157–158.
3. Wijnen-Meijer M, ten Cate OT, van der Schaaf M, Borleffs JCVertical integration in medical school: Effect on the transition to postgraduate training. Med Educ. 2010;44:272–279.
4. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf MCurriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach. 2015;37:983–1002.
6. ten Cate O, Hart D, Ankel F, et alEntrustment decision making in clinical training. Acad Med. 2016;91:191–198.
7. Okusanya OT, Kornfield ZN, Reinke CE, et alThe effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies. J Surg Educ. 2012;69:536–543.
8. Naylor RA, Hollett LA, Castellvi A, Valentine RJ, Scott DJPreparing medical students to enter surgery residencies. Am J Surg. 2010;199:105–109.
9. Lyss-Lerman P, Teherani A, Aagaard E, Loeser H, Cooke M, Harper GMWhat training is needed in the fourth year of medical school? Views of residency program directors. Acad Med. 2009;84:823–829.
11. Englander R, Flynn T, Call S, et alToward defining the foundation of the MD degree: Core entrustable professional activities for entering residency. Acad Med. 2016;91:1352–1358.
12. Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate OWhen do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010;85:1408–1417.
13. Hauer KE, Kohlwes J, Cornett P, et alIdentifying entrustable professional activities in internal medicine training. J Grad Med Educ. 2013;5:54–59.
14. Kennedy TJ, Regehr G, Baker GR, Lingard LPoint-of-care assessment of medical trainee competence for independent clinical work. Acad Med. 2008;83(10 suppl):S89–S92.
15. Chen HC, van den Broek WE, ten Cate OThe case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015;90:431–436.
16. Ryan MS, Lockeman KS, Feldman M, Dow AThe gap between current and ideal approaches to the core EPAs: A mixed methods study of recent medical school graduates. Med Sci Educ. 2016;26:463–473.
17. Hirsh DA, Holmboe ES, ten Cate OTime to trust: Longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014;89:201–204.
18. McGaghie WC, Barsuk JH, Wayne DBAM last page: Mastery learning with deliberate practice in medical education. Acad Med. 2015;90:1575.
19. Mulder H, Ten Cate O, Daalder R, Berkvens JBuilding a competency-based workplace curriculum around entrustable professional activities: The case of physician assistant training. Med Teach. 2010;32:e453–e459.
20. Boyce P, Spratt C, Davies M, McEvoy PUsing entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011;11:96.
21. Fessler HE, Addrizzo-Harris D, Beck JM, et alEntrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: Report of a multisociety working group. Chest. 2014;146:813–834.
22. Aylward M, Nixon J, Gladding SAn entrustable professional activity (EPA) for handoffs as a model for EPA assessment development. Acad Med. 2014;89:1335–1340.
23. van der Vleuten CP, Schuwirth LW, Driessen EW, Govaerts MJ, Heeneman STwelve tips for programmatic assessment. Med Teach. 2015;37:641–646.
24. Ten Cate OTrusting graduates to enter residency: What does it take? J Grad Med Educ. 2014;6:7–10.