Toward Defining the Foundation of the MD Degree: Core Entrustable Professional Activities for Entering Residency

Abstract

Currently, no standard defines the clinical skills that medical students must demonstrate upon graduation. The Liaison Committee on Medical Education bases its standards on required subject matter and student experiences rather than on observable educational outcomes. The absence of such established outcomes for MD graduates contributes to the gap between program directors’ expectations and new residents’ performance.

In response, in 2013, the Association of American Medical Colleges convened a panel of experts from undergraduate and graduate medical education to define the professional activities that every resident should be able to do without direct supervision on day one of residency, regardless of specialty. Using a conceptual framework of entrustable professional activities (EPAs), this Drafting Panel reviewed the literature and sought input from the health professions education community. The result of this process was the publication of 13 core EPAs for entering residency in 2014. Each EPA includes a description, a list of key functions, links to critical competencies and milestones, and narrative descriptions of expected behaviors and clinical vignettes for both novice learners and learners ready for entrustment.

The medical education community has already begun to develop the curricula, assessment tools, faculty development resources, and pathways to entrustment for each of the 13 EPAs. Adoption of these core EPAs could significantly narrow the gap between program directors’ expectations and new residents’ performance, enhancing patient safety and increasing residents’, educators’, and patients’ confidence in the care these learners provide in the first months of their residency training.

Since the publication of the Flexner report in 1910,¹ medical schools have provided a four-year curriculum that emphasizes the basic sciences in the first phase and the clinical sciences in the second phase. Standards set by the Liaison Committee on Medical Education (LCME) focus more on required subject matter and student experiences than on educational outcomes. Although all undergraduate courses must have outcomes-based learning objectives, the LCME standards do not provide guidance as to what those “outcomes” should be.² As a result, there is disagreement about the specific clinical abilities and acumen that should be expected of a medical degree (MD) graduate. This absence of clear, standard expectations has led to a gap between residency program directors’ expectations and new residents’ performance and to significant heterogeneity in entering residents’ performance.^3–6

The literature is replete with evidence that residents are trusted to perform clinical tasks early in residency often without supervision. For example, Raymond and colleagues⁷ studied more than 2,500 residents from 10 core specialties who reported performing activities from the routine (discussing a patient’s care with an insurance company) to the emotionally laden (managing an angry patient) to the technically challenging (performing a thoracentesis) during their first days. The creation of common expectations for entering postgraduate year (PGY) 1 residents is critical to the delivery of safe patient care in the early months of residency.

Over the past decade, medical educators in the United States and Canada have begun to focus on the transitions in education and training, including the transition from medical school to residency.^8,9 The 2010 Carnegie Foundation report entitled Educating Physicians: A Call for Reform of Medical School and Residency strongly recommended the standardization of educational outcomes and the individualization of learning pathways in the continuum of physician education.¹⁰ The combination of increasing concerns about entering residents’ performance and the national and international focus on needed changes in medical education have created a strong energy for reform.

In response, in January 2013, two of us (R.E., C.A.A.), while at the Association of American Medical Colleges (AAMC), convened a Drafting Panel to define a concise list of professional activities that every MD graduate should be able to do without direct supervision (defined as the supervisor not physically present) on day one of residency. The members of the Drafting Panel were chosen to ensure representation across the educational continuum. We hypothesized that engagement from those on the graduate medical education (GME) side of the transition from medical school to residency would be critical to the uptake of the Drafting Panel’s product. To represent this perspective, we asked the past chair of the Accreditation Council for Graduate Medical Education (ACGME) and an expert in GME (T.F.) to chair the panel. Panel members included a medical student (J.T.), surgical resident (B.L.), student affairs dean (M.J.G.), basic scientist (T.B.F.), undergraduate medical education (UME) leader from Canada (J.R.), two UME leaders from the United States (L.C., S.A.L.), two residency program leaders (S.C., R.M.M.), two institutional GME leaders (M.L.L., M.C.W.), and an educational leader from the American Board of Pediatrics (C.C.). The two AAMC staff also brought experience in educational leadership across the continuum.

In this article, we (1) reaffirm the initial guiding principles and conceptual framework for the development of core entrustable professional activities (EPAs) for entering residency, (2) outline the process for the development of the Core Entrustable Professional Activities for Entering Residency document¹¹ so that others might replicate it in developing other EPAs, and (3) suggest next steps toward validity testing and implementation of the core EPAs to achieve the ultimate goal of graduating medical students who have been entrusted to perform these activities without direct supervision by the time they enter residency.

Guiding Principles for Developing the Core EPAs for Entering Residency

The Drafting Panel established several guiding principles, the first being that patient safety would be the primary motivator for this work. Whether or not the “July effect” exists,^12,13 our rationale for engaging in this work was primarily based on two patient safety issues: (1) The clinical skills of new residents are highly variable and do not always meet the expectations of program directors,^5,14,15 and (2) residents perform many activities for which they may not have had adequate instruction or assessment without direct supervision.¹⁶

The second guiding principle was to develop a “core” of common skills that apply to all physicians, regardless of specialty choice. A comprehensive medical school education includes instruction in additional skills linked to the school’s mission beyond patient care as well as in skills that are specific to a specialty rather than applicable to all medical school graduates (e.g., suturing for the procedural specialties). We did not attempt to address these additional skills.

Our third guiding principle was that learner assessment is critical to the successful implementation of any core requirements in medical education. We envisioned an assessment system based on frequent formative assessments, ultimately culminating in entrustment decisions based on the aggregate evidence. Finally, our fourth guiding principle was that implementation of such core requirements must be coupled with robust resources for faculty development. Faculty will need to learn new approaches to teaching, assessment, and feedback and develop the skills to make entrustment decisions regarding a student’s readiness to perform specific tasks without direct supervision.

Conceptual Framework: Choosing EPAs and Linking Them to Competencies and Milestones

We reviewed two conceptual frameworks used in competency-based medical education—EPAs^17,18 and competencies (further defined by their milestones or performance levels). The concept of EPAs was introduced in 2005¹⁷ as a practical approach to overcoming the limitations created by the abstract, context-independent, and granular nature of competencies. EPAs are observable and measurable units of work that focus on care delivery and, as such, align the assessment process with what learners actually do and what faculty observe in the workplace. EPAs also introduce the notion of trust and its implications for variable levels of supervision to the assessment equation.¹⁹ Because faculty make entrustment decisions all the time in clinical supervision situations, we expected that the explicit addition of entrustment to the EPA framework would appeal to faculty and learners.

To provide a more holistic determination of a learner’s clinical abilities, we developed an integrated framework that defines EPAs and links them to their critical competencies and milestones. EPAs allow educators to assess a learner’s integration of the competencies needed for actual care delivery into her or his performance. The competencies in turn facilitate the description of the granular components of the learner’s abilities that underpin the successful performance of clinical activities. Competencies and their milestones thus can serve to “diagnose” a learner who is unable to progress to entrustment on a given EPA.

EPAs were initially designed for the residency-to-practice transition with entrustment linked to the learner’s ability to practice the EPA unsupervised. We adapted this entrustment concept for the UME-to-GME transition.^17,18 New residents always have either direct supervision (supervisor directly in the room) or indirect supervision (senior residents or faculty immediately available to assist). Therefore, we defined the core EPAs for entering residency as “activities that all entering residents should be expected to perform on day one of residency without direct supervision.”¹¹ Of note, as learners change settings (e.g., change institutions in going from a student to a resident), direct supervision may be initially required to verify that they possess the skills that are dependent on system functionality (such as “document a clinical encounter in the patient record”).

Getting to the Final 13: Why These EPAs?

We began by performing a literature review using (1) the Thomson Reuters Web of Science database²⁰ that includes the Web of Science Core Collection and MEDLINE to search for medical school graduation requirements and residency program director expectations, and (2) MedEdPORTAL (www.mededportal.org) to search for capstone courses. The results were reviewed by one of the authors (R.E.) and an AAMC staff member, and those resources that delineated clear graduation expectations in the form of expected skills, tasks, or competencies for all medical graduates were compiled as background information for the Drafting Panel to read prior to their first meeting.

After reviewing these resources about published graduation requirements,^4,5,8,21,22 program directors’ expectations for entering residents,^4,5,22 and tasks that new interns performed without direct supervision,⁷ we identified activities that thematically clustered into 21 distinct EPAs. Resident members of the AAMC’s Organization of Resident Representatives provided their own list of activities that a resident should be able to perform on the first day of residency, which clustered thematically into 22 EPAs. Twenty-one themes from the Organization of Resident Representatives’ list were identical to our results.

With these 21 overlapping EPAs as our base, we used a modified Delphi process to narrow this list. In the first round, we ranked the 21 EPAs as high, medium, or low priority. In the second round, all EPAs with a majority of high priority rankings and no low priority rankings were placed at the top of the list; those with a majority of medium priority rankings were placed in the middle, and those with a majority of low priority rankings were placed at the bottom. We then considered which, if any, of the high-priority EPAs should be dropped because they were redundant or were too advanced for a graduating medical student and which of the medium- or low-priority EPAs should be reconsidered. After the second round, 9 high-priority EPAs remained on our list and 4 medium-priority items were added, for a total of 13. In the final round, we reaffirmed that all 13 EPAs should be shared with a reactor panel for feedback.

Next, we convened the reactor panel, which was composed of medical educators from across the continuum, students, residents, and other health professionals who volunteered to read the background literature and provide feedback on the list of 13 EPAs. Conference calls engaged more than 100 unique participants, who agreed on the inclusion of 12 of the EPAs and on the elimination of 1 EPA (manage common medical conditions), which most felt should require direct supervision early in residency. The members of the reactor panel voiced a strong need for an EPA related to quality improvement and patient safety. Thus, we added the EPA entitled “Identify system failures and contribute to a culture of safety and improvement.” Finally, after considerable discussion about the specific “general procedures of a physician” (EPA 12), the reactor panel members concluded that all students graduating medical school must be able to perform four core procedures—bag and mask ventilation, basic cardiopulmonary resuscitation, venipuncture, and intravenous line insertion. See List 1 for the final 13 EPAs.¹¹

List 1

Final 13 Core Entrustable Professional Activities for Entering Residency¹¹

1. Gather a history and perform a physical examination
2. Prioritize a differential diagnosis following a clinical encounter
3. Recommend and interpret common diagnostic and screening tests
4. Enter and discuss orders/prescriptions
5. Document a clinical encounter in the patient record
6. Provide an oral presentation of a clinical encounter
7. Form clinical questions and retrieve evidence to advance patient care
8. Give or receive a patient handover to transition care responsibility
9. Collaborate as a member of an interprofessional team
10. Recognize a patient requiring urgent or emergent care and initiate evaluation and management
11. Obtain informed consent for tests and/or procedures
12. Perform general procedures of a physician
13. Identify system failures and contribute to a culture of safety and improvement

For each EPA, we then created a description that includes the critical functions required to perform that activity and any inclusion or exclusion criteria, such as the common tests learners are expected to order and interpret. These descriptions point to the competencies that are critical to each EPA, as well as the key features of the curriculum needed to teach learners to perform at a level of entrustment.²³

Mapping the EPAs to Their Critical Competencies

We identified the critical competencies that learners must integrate into their practice to be entrusted to perform each EPA without direct supervision. The taxonomy created by two of us (R.E., C.A.A.) and others²⁴ was used to ground discussions in a single competency list. This taxonomy of general physician competencies was based on the 2012 ACGME competencies.²⁵ We then compared the more than 150 competency lists that exist across the continuum of education and training, across four countries, and across health professions vs. the ACGME competencies. The result was a list of 58 competencies within eight domains, with the six ACGME competency domains and their corresponding competencies at its core. This final list incorporated all of the primary concepts relevant to physicians that we pulled from the 150-plus lists we reviewed. See List 2 for these domains of competence and List 3 for examples of the competencies within the domain of patient care.

List 2

The 8 Domains of General Physician Competence, From the Taxonomy²³ Used to Map the Core Entrustable Professional Activities for Entering Residency to Their Critical Competencies

1. Patient care
2. Knowledge for practice (Medical knowledge for physicians)
3. Practice-based learning and improvement
4. Professionalism
5. Interpersonal and communication skills
6. Systems-based practice
7. Interprofessional collaboration
8. Personal and professional development

List 3

The 11 Competencies Within the Domain of Patient Care (PC), From the Reference List of General Physician Competencies²³

• PC 1: Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice
• PC 2: Gather essential and accurate information about patients and their condition through history taking, physical examination, and the use of laboratory data, imaging, and other tests
• PC 3: Organize and prioritize responsibilities to provide care that is safe, effective, and efficient
• PC 4: Interpret laboratory data, imaging studies, and other tests required for the area of practice
• PC 5: Make informed decisions about diagnostic and therapeutic interventions based on patient information and preferences, up-to-date scientific evidence, and clinical judgment
• PC 6: Develop and carry out patient management plans
• PC 7: Counsel and educate patients and their families to empower them to participate in their care and enable shared decision making
• PC 8: Provide appropriate referral of patients including ensuring continuity of care throughout transitions between providers or settings and following up on patient progress and outcomes
• PC 9: Provide health care services to patients, families, and communities aimed at preventing health problems or maintaining health
• PC 10: Provide appropriate role modeling
• PC 11: Perform supervisory responsibilities commensurate with one’s roles, abilities, and qualifications

Next, we used a Q-sorting methodology²⁶ to reach consensus on the five to eight most critical competencies for each EPA. Theoretically, each EPA could be mapped to many competencies, but only a few of those are required to reach entrustment. We defined critical competencies as those that must be demonstrated before a learner may perform the activity without direct supervision. For example, driving a car includes competencies in ignition, steering, braking, and managing a GPS. In this case, the first three competencies would be critical to an entrustment decision for a new driver, while the last is not. Similarly, a new resident should be able to obtain informed consent without direct supervision for tasks such as blood transfusions, central lines, or immunizations. Doing so requires patient care competencies such as developing and carrying out management plans and counseling and educating patients and their families. Although performing the procedure (Patient care-1) might seem critical to obtaining informed consent, residents often obtain consent for procedures they do not do (e.g., blood transfusion), so we did not link procedural competency to the informed consent EPA.

Developing Milestones for the Critical Competencies

We created milestones for each of the competencies linked to 1 or more of the 13 EPAs through the process described above. Milestones are behavioral descriptors of learners at advancing levels of performance along a developmental continuum. Our milestones were derived predominantly from the five sets of ACGME specialty milestones that were published at the time—internal medicine,²⁷ surgery,²⁸ pediatrics,²⁹ psychiatry,³⁰ and emergency medicine.³¹ We developed two milestones for each competency—one representing a novice or preentrustable learner and the other representing an entrustable learner, defined as one who is competent to perform the activity without direct supervision. We relied heavily on the pediatrics milestones for our novice learner descriptors, as this was the only set of milestones designed to span the continuum from entry to medical school to practice. We used the entry-level milestones for the other four specialties and the second- or third-level pediatrics milestones to represent the entrustable PGY1 learner.

Guiding Assessment and Faculty Development

The process of entrustment requires faculty to observe learners directly and make professional judgments about the level of supervision they need. We sought to add clarity to these judgments by identifying specific behaviors that should be expected of learners at each of the two levels—a novice learner requiring direct supervision and a competent learner ready to perform the EPA without direct supervision. Towards that end, we created descriptive behavioral narratives and case vignettes for each EPA based on the milestones for preentrustable and entrustable learners. Figure 1 depicts the relationship between EPAs, competencies, and milestones and the resultant use of the milestones to build these narrative descriptions.

Figure 1: The relationship between entrustable professional activities (EPAs), domains of competence (DoC), competencies, and milestones and the resultant use of the milestones to build narrative descriptions of preentrustable and entrustable learners (adapted from the Core Entrustable Professional Activities for Entering Residency¹¹).

We then used the expected behaviors for each EPA to create vignettes that both depict and contrast the preentrustable learner and the entrustable learner. Through this process, we aimed to develop a shared mental model, rooted in the competencies and milestones, that supervisors could use to decide when a learner warrants advancement to indirect supervision (supervisor not present in the room) for a specific EPA. We recognize that an intermediate level—the advanced beginner (e.g., a learner able to perform the EPA with a supervisor present but with minimal intervention from the supervisor)—may emerge as we implement the core EPAs and use the expected behaviors and vignettes for faculty development and assessment of students. If feedback from the field indicates such a need, a third, intermediate level could be added to our framework. We did not include the two highest levels of performance—able to perform the EPA unsupervised (proficient) or able to supervise others in performing the EPA (expert)—because we did not expect these behaviors of graduating medical students.

Once these narratives and vignettes were created, we posted the draft document on the AAMC Web site and distributed it to relevant AAMC constituencies for several months of open comment. During this open comment period, Olle ten Cate,³² the originator of the EPA concept, suggested that three of the EPAs might not fit his original definition of independent, stand-alone entrustable activities (EPAs 7, 9, and 13). We believe that input from the ongoing pilot and beta testing will help to address this concern.

The final version includes a description of each EPA, a list of key functions, links to critical competencies and milestones, and narrative descriptions of expected behaviors and clinical vignettes for both preentrustable and entrustable learners, and was shared in June 2014.¹¹

Next Steps: Testing the EPAs in the Field

The process of defining the core EPAs for entering residency is just the first step toward improving alignment between the expectations of residency program directors and the actual abilities of incoming residents. Through dialogue and experimentation, the medical education community (both UME and GME) must determine the curriculum, assessment, and faculty development needs for each EPA. In developing curricula, educators will need to pay attention to structural implications. What is the educational structure that best advances students toward entrustment? For example, are there implications for moving from a traditional block structure to one that emphasizes longitudinal experiences? Further, the community needs to explore and develop robust approaches for medical schools to make entrustment decisions. They will need to address such issues as the following: (1) Who makes final entrustment decisions? (i.e., are they made by committee or individuals?); (2) what is the core evidence required for these decisions? and (3) when in the curriculum would entrustment be expected for each of the 13 EPAs?

Although publication of the final list of core EPAs happened less than two years ago, two avenues of inquiry are under way already to implement and test this framework. First, the AAMC is sponsoring a multiyear pilot with 10 of its member medical schools to define the curriculum, assessment, and faculty development program needed to teach each of the EPAs and to delineate pathways to entrustment for medical students. As an indication of both the remarkable national interest in developing common expectations for graduating students across medical schools and the degree to which these EPAs have resonated with medical educators and faculty, more than half of the 141 LCME-accredited schools applied to join this five-year pilot. Many schools are already engaged in piloting implementation of some or all of the EPAs. To foster collaboration among the medical education community, the AAMC launched the Core EPAs for Entering Residency Web site (www.aamc.org/coreepas) and a dedicated listserv in early 2015.

In addition, the 2015 AAMC medical education meeting in Baltimore, Maryland included a two-hour poster session devoted to how institutions are implementing the core EPAs. The session featured more than 50 posters from 35 medical schools. Finally, the core EPAs have set the foundation for international work in this area. For example, implementation of the Future of Medical Education in Canada project⁹ includes a group developing Canadian EPAs, which will become the standard for UME in Canada and which use the Core EPAs for Entering Residency document as a foundation. Once developed, these EPAs will be tested through a pilot similar to ours. The Netherlands and Germany also are engaged in developing core EPAs for their medical school graduates.

Implementation of the core EPAs also will be tested through the AAMC’s Education in Pediatrics Across the Continuum project (www.aamc.org/initiatives/epac/), a pilot program involving a small number of learners at four institutions. This project is designed to test the feasibility of implementing competency-based, rather than time-based, transitions from both UME to GME and GME to fellowship or practice. Four schools have agreed to recruit at least four cohorts of four students per year who are interested in a career in pediatrics and are willing to stay at their home institution for residency. These students will advance through the “preclinical” phase of their education and their clinical immersion experience, into residency, through residency, and into practice or fellowship in a time-variable fashion. The schools will use the core EPAs as the primary framework for determining when the students transition from UME to GME, in addition to their school’s existing graduation requirements. The core EPAs will be linked to the pediatrics EPAs, which will serve as the primary framework for determining when the students transition from GME to fellowship or practice. Each school now has one to two cohorts of students, and the project leaders are highly optimistic on the basis of their performance thus far that at least some of these students will be able to progress more quickly to residency than the students following the traditional four-year model, although they also recognize that some ultimately may take longer to do so.

Linking the core EPAs with specialty and subspecialty EPAs provides a continuum of learning and assessment that could have major implications for professional development trajectories. First, if learners enter residency with a known skill set, program directors and faculty can immediately build on these baseline skills without having to assess the learners’ abilities. This opens an opportunity for more rapid progress along the developmental continuum during residency. Second, learners’ level of performance at the end of residency training could guide their learning for the first cycle of maintenance of certification, as they transition into practice.

Scholarship is needed to guide and assess the impact of this shift from focusing on time and process to focusing on outcomes-based EPAs. We must continue to investigate the curricula, assessment strategies, faculty development practices, and educational structures that best support this outcomes-based educational design. We also need to assess the process of implementing the EPAs to understand what works, under what conditions, and for whom. Ultimately, we must evaluate the program’s impact on patient care and safety in real time.

Conclusions

By focusing on the performance outcomes that are expected of all students receiving an MD degree, the core EPAs for entering residency offer a promising framework to guide the shift from focusing on time and predefined curricula as in the Flexner era to focusing on competency-based medical education. In their current form, these EPAs are guidelines, grounded in the literature and vetted through a process that broadly engaged the medical education community.³³ Only through implementation can we test the hypothesis that entrustment on these 13 clinical tasks will result in new physicians being better prepared to assume responsibilities at the outset of their residency years and provide safer care to their patients from day one.

Deciding how best to operationalize these core EPAs will require creativity, innovation, and perseverance over the coming years. This journey was sparked by a desire to improve the quality and safety of the care that new residents provide to patients. Continued commitment to this goal will drive us to determine whether the core EPAs meet the mark for entering residents.