Secondary Logo

Journal Logo

Innovation Reports

Finding a Path to Entrustment in Undergraduate Medical Education: A Progress Report From the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group

Brown, David R. MD; Warren, Jamie B. MD, MPH; Hyderi, Abbas MD, MPH; Drusin, Ronald E. MD; Moeller, Jeremy MD; Rosenfeld, Melvin PhD; Orlander, Philip R. MD; Yingling, Sandra PhD; Call, Stephanie MD; Terhune, Kyla MD; Bull, Janet MA; Englander, Robert MD; Wagner, Dianne P. MD; for the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group

Author Information
doi: 10.1097/ACM.0000000000001544
  • Free

Abstract

Problem

Competency-based medical education is emerging as the predominant paradigm across the education continuum. Graduate medical education (GME) clinical competency committees periodically assess and report milestones achievements for their residents, but comparable systematic competency assessment and reporting structures do not exist in undergraduate medical education (UME), where the advancement process tends to focus primarily on identifying struggling students.1–3 The transition to a competency- and outcomes-based educational model requires UME to move beyond the traditional time-based curriculum. Increased focus on competencies in GME has exposed a “gap between residency program directors’ expectations and new residents’ performance.”1 To address this gap and ensure that all medical school graduates have a basic level of preparedness for the responsibilities of residency, a drafting panel convened by the Association of American Medical Colleges (AAMC) defined 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs) that all graduating medical students might be expected to perform on day one of residency without direct supervision.1,4 (See Englander et al4 for the complete list of the 13 Core EPAs.)

In addition to addressing the UME-to-GME transition, the Core EPAs framework also offers a practical process for assessing competencies.2,3 It allows educators to take a holistic approach to the assessment of competencies and their corresponding milestones because they represent the activities of

the day-to-day work of the professional; situate competencies and milestones in the clinical context in which we live; make assessment more practical by clustering milestones into meaningful activities; [and] explicitly add the notions of trust and supervision into the assessment equation.4

By creating a shared understanding of specific professional workplace activities, clustering competencies and milestones to fit those activities, and crafting developmental models of the knowledge, skills, and attitudes associated with those activities, the Core EPAs framework guides the “gestalt” of supervisors so they are able to provide effective assessment and feedback about the ability of learners to perform specific professional activities in the workplace.3

Foundational to the Core EPAs framework are the concepts of trust5,6 and supervision,2 which include complex relational issues involving the learner, the supervisor, and the context of the situation.2,5,6 Day-to-day decisions to entrust learners traditionally have been a foundation of clinical education,6 but explicit, summative entrustment decisions using the Core EPAs framework are new to medical education.1–4 Entrustment raises the issue of how to assess learners’ trustworthiness for specific activities, as well as how trust develops in supervisors, in learners, and in curricular and assessment systems. Ten medical schools are working with the AAMC on a pilot project to test the implementation and evaluation of the Core EPAs framework in UME, including how to operationalize the concept of entrustment. In this report, we describe the initial discussions and findings regarding entrustment from this pilot.

Approach

The Core EPAs pilot began in October 2014 with a goal of developing and implementing a process for making summative entrustment judgments for the graduating class of 2019. The pilot was organized into workgroups related to each of the 13 Core EPAs. Additional workgroups were formed to address concepts that affected all of the Core EPAs: Curriculum Development, Assessment, Faculty Development, and Entrustment. A steering committee was made up of the team leaders from each participating medical school. The Entrustment Concept Group, with representation from each school, was charged with guiding the development of the structures and processes for entrustment decision making. Most of the authors of this report are members of this concept group. We reviewed the literature on entrustment and iteratively discussed guiding principles, entrustment processes, trustworthiness frameworks, and findings from the other Core EPA workgroups.

Outcomes

From this literature review, we identified three initial lenses through which to consider the entrustment process—the perceived trustworthiness of the learner, the workplace-based gestalt judgment of a supervisor working with a learner, and the summative formal entrustment decision for each Core EPA. We defined entrustment across each of the Core EPAs as the point at which learners both (1) possess the requisite knowledge, skills, and attitudes needed to perform the EPA and (2) demonstrate specific elements of trustworthiness both foundationally and within the context of entrustment decisions indicating that they are able to perform the EPA without direct supervision.1,6

Creating transparency in formal summative entrustment decision-making processes

We developed guiding principles for making formal summative entrustment decisions that are transparent to faculty and learners. We recommend the following principles to operationalize a formal process for entrustment:

  • Create a process to describe and maintain formal entrustment decisions by a trained group of administrators and faculty,
  • Base entrustment decisions on a longitudinal view of each learner’s performance,
  • Include day-to-day ad hoc workplace entrustment judgments by clinical supervisors in the body of evidence supporting formal entrustment decisions,
  • Explicitly measure attributes of learners’ trustworthiness as foundational to all the Core EPAs (in addition to EPA-specific knowledge and skills),
  • Gather multimodal performance evidence from multiple assessors,
  • Ensure a process for formative feedback, and
  • Ensure that each learner is an active participant in entrustment decisions.

Deconstructing trustworthiness

Kennedy and colleagues6 studied supervisors’ assessments of learners’ ability to work independently, and they identified four dimensions that guided supervisors’ decisions about learners’ trustworthiness for independence: knowledge/skill, discernment of limitations, truthfulness, and conscientiousness. We considered three of these dimensions—discernment, truthfulness, and conscientiousness—essential elements of entrustment that are foundational to all of the Core EPAs. We then further explored these three dimensions of trustworthiness. Like professionalism, trustworthiness includes cross-cutting behaviors and attitudes that appear in multiple contexts. Nonetheless, the individual dimensions of discernment, truthfulness, and conscientiousness unpack trustworthiness into specific observable behaviors. The fourth dimension described by Kennedy and colleagues—knowledge/skill—while foundational to each Core EPA is also context specific. Therefore, the individual Core EPA workgroups, rather than our concept group, were tasked with drafting knowledge/skill-specific developmental paths to entrustment for each of the 13 Core EPAs.3,7 Since trustworthiness crosses all of the Core EPAs, through an iterative group process, we constructed a developmental framework of trustworthiness that could be studied by the Core EPA pilot schools (see Chart 1).3,6,7

Chart 1
Chart 1:
Developmental Framework of Trustworthiness in Medical Students, Created by the Entrustment Concept Group to Be Used by the Core Entrustable Professional Activities for Entering Residency Pilot Medical Schools, 2016a

An important concept in the Core EPA pilot is continuity with GME. To align with efforts in GME, we conceptualized trustworthiness in the clinical context as a skill with the aspirational “proficient” level of the Dreyfus8 model anchored to the skill level that highly reliable residents possess (see Chart 2). This framework of trustworthiness as aspirational can be used by learners, faculty, and committees charged with making entrustment decisions, through the compilation of the results of multisource assessments mapped to the Core EPAs. Elements of this trustworthiness framework are being integrated into classroom and small-group professionalism assessments and embedded in assessments for specific workplace activities. During the next stage of the pilot, we will explore the consequences of different approaches to implementing this framework across the 10 pilot schools and evaluate the validity of the rating scale across sites.

Chart 2
Chart 2:
Trustworthiness Assessment Tool, Created at the University of Illinois College of Medicine as Part of the Core Entrustable Professional Activities for Entering Residency Pilot, 2016a

Implementing an entrustment decision-making process

Each pilot school is exploring how the guiding principles for entrustment that we developed (listed above) may impact its structures and functions and what adaptations may be required. Adoption of these principles affects the prioritization of funding and faculty time and may challenge existing medical school culture. In Table 1, we summarize the plans for applying the guiding principles at the pilot schools as well as important remaining challenges.

Table 1
Table 1:
Activities Relating to the Guiding Principles for Entrustment at 10 Medical Schools Participating in the Core Entrustable Professional Activities for Entering Residency (Core EPAs) Pilot, 2016

The existence of the following attributes at some of the pilot schools has facilitated operationalizing the guiding principles:

  • Structures for longitudinal relationships between faculty and learners,
  • Portfolios that allow tracking of competency assessment data,
  • Analytic systems that allow the aggregation of competency assessment data into dashboards, and
  • Learner handovers across educational settings within UME.

We also identified common barriers to this work, the most notable being funding and faculty development. While longitudinal faculty–learner relationships facilitate the development of trust, schools with fewer resources find it challenging to create these programs. In addition, some schools have rigorous informatics systems in place to compile multisource assessment data into dashboards, while others are only beginning to explore this possibility and investigate the necessary funding. Finally, while some schools have conceptualized a process to render formal entrustment decisions, most are still in the exploratory phase.

Specific Core EPA–related curricula, point-of-care assessments, and faculty development are essential to adopting systems of formal entrustment decision making; these programs are being developed by the different concept groups and individual Core EPA workgroups.9,10 By providing a transparent model of entrustment for the Core EPAs, we hope to systematize ad hoc workplace entrustment judgments and provide learners with reliable feedback as they progress toward entrustment.

Next Steps

In most medical schools, a single advancement committee focuses on identifying struggling students, never discussing the majority of students who are not struggling. Therefore, monitoring every student’s progress and ultimately using aggregate evidence to render an entrustment decision for each Core EPA is a major shift. The Core EPA pilot schools are implementing a variety of EPA-specific assessments, as well as assessments of trustworthiness, in a variety of contexts across curricula. With this shift, the pilot schools are working to engage students and faculty, compile evidence, and pilot the work of entrustment committees. Our work, described above, raised key questions in three categories (discussed below) that require further exploration. Although each pilot school is expected to discover different solutions to these challenges, their collective efforts and the resulting lessons and conversations will help us to further understand these core concepts.

Evaluating the entrustment process

The pilot schools are in the process of defining developmental paths to entrustment that can be monitored as students progress through UME. We will examine the role of longitudinal supervisory or coaching relationships, student-driven assessments, trustworthiness-based assessments, supervisory scales, EPA-specific assessments, and enhanced transparency about students’ progression towards entrustment. The pilot schools will compare the amount and type of performance evidence that they are able to compile on individual students and discuss their confidence in that evidence. Since medical school graduates commonly transition to new institutions for residency, we must rely on one another’s entrustment decisions. Open discussions among the pilot participants regarding the limitations of our assessment evidence will help us to create a shared understanding of the curricula, workplace activities, assessment systems, and faculty development necessary to enable institutions to entrust students to perform the Core EPAs. The role of a formal educational handover from UME to GME is also being explored.

Compiling evidence of trustworthiness

We also plan to study explicit assessments of the elements of trustworthiness (discernment, conscientiousness, and truthfulness) as professional attributes that can be developed through coaching. We will seek evidence of trustworthiness in non-patient-care settings, in clinical settings independent of Core EPA performance, and within the act of performing a given Core EPA. Lapses in trustworthiness may occur in situations of high stress or extreme fatigue, so we will consider how students can anticipate such lapses, which lapses are remediable, and which are not. Providing students with this kind of feedback may profoundly impact their subsequent strategies and behaviors. Multisource assessment of students in the context of the Core EPA framework may show that evidence of trustworthiness is demonstrable throughout the curriculum. Longitudinal use of the trustworthiness framework (see Chart 1) could enable students to identify training situations in which consistent discernment, conscientiousness, or truthfulness is challenging to maintain. Furthermore, while we believe that our trustworthiness framework will be useful to inform entrustment decisions, the pilot schools have not yet begun testing or validating it as an assessment tool. Finally, in addition to studying the relevant components of the trustworthiness model created by Kennedy and colleagues, we plan to explore how factors such as communication skills, motivation, emotional intelligence, and systems orientation may impact supervisors’ assessment of students’ trustworthiness.4–7

Ensuring trust in entrustment

As trust is reciprocal, our efforts to promote a safe transition from UME to GME for our students and the safety of their future patients seek to engender trust from all stakeholders. The role of the student in the entrustment process is crucial; we must win the trust of our students by establishing the validity, reliability, and appropriateness of our new assessments and by instituting mechanisms to help students meet expectations for entrustment. The limited opportunities for students to participate directly in patient care and perform the Core EPAs in current educational environments may pose a significant systematic barrier to EPA measurement. The identification of developmental paths toward entrustment can assist in identifying appropriate activities for different levels of students for each of the Core EPAs. By sharing findings among institutions, we plan to explore which interventions in curriculum, assessment, faculty development, feedback, and remediation best support the entrustment process. These factors all contribute to determining whether it is possible under the current rules and structures of UME to entrust graduating medical students to perform all 13 Core EPAs.

The Core EPAs framework requires us to learn new terminology, make changes in faculty and student approaches to feedback and assessment, and modify the structures and processes of UME assessment systems. The work of the Entrustment Concept Group specifically, and the Core EPAs pilot as a whole, seeks to inform meaningful culture change in UME through a shared understanding of the assessment of trust and a shared trust in that assessment.

Acknowledgments: The authors thank the Association of American Medical Colleges, Carol Aschenbrener, Maryellen Gusic, and all of the members of the Core Entrustable Professional Activities for Entering Residency pilot for their support, inspiration, and contributions to this report. The members of the Entrustment Concept Group include Judith L. Bowen, MD, David Brown, MD, Stephanie Call, MD, MSPH, Ronald E. Drusin, MD, Abbas Hyderi, MD, MPH, Jeremy Moeller, MD, Philip R. Orlander, MD, Melvin Rosenfeld, PhD, Kyla Terhune, MD, Dianne P. Wagner, MD, Jamie B. Warren, MD, MPH, and Sandra Yingling, PhD.

References

1. Englander R, Flynn T, Call S, et al.Toward defining the foundation of the MD degree: Core entrustable professional activities for entering residency. Acad Med. 2016;91:1352–1358.
2. Chen HC, van den Broek WE, ten Cate OThe case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015;90:431–436.
3. Aylward M, Nixon J, Gladding SAn entrustable professional activity (EPA) for handoffs as a model for EPA assessment development. Acad Med. 2014;89:1335–1340.
4. Englander R, Aschenbrener CA, Call SA, et al.Core entrustable professional activities for entering residency. MedEdPORTAL. 2014. https://www.mededportal.org/icollaborative/resource/887. Accessed November 3, 2016.
5. ten Cate O, Hart D, Ankel F, et al.Entrustment decision making in clinical training. Acad Med. 2016;91:191–198.
6. Kennedy TJ, Regehr G, Baker GR, Lingard LPoint-of-care assessment of medical trainee competence for independent clinical work. Acad Med. 2008;83(10 suppl):S89–S92.
7. Brown DR, Gillespie CC, Warren JBEPA 9—Collaborate as a member of an interprofessional team: A short communication from the AAMC Core EPAs for Entering Residency pilot schools. Med Sci Educ. 2016;26:457–461.
8. Dreyfus SEThe five-stage model of adult skill acquisition. Bull Sci Technol Soc. 2004;24:177–181.
9. Lomis K, Amiel JM, Ryan MS, et al.Implementing an entrustable professional activities framework in undergraduate medical education: Early lessons from the AAMC Core Entrustable Professional Activities for Entering Residency Pilot. Acad Med. 2017;92:765–770.
10. Favreau MA, Tewksbury L, Lupi C, et al.Constructing a shared mental model for faculty development for the Core Entrustable Professional Activities for Entering Residency. Acad Med. 2017;92:759–764.
Copyright © 2017 by the Association of American Medical Colleges