Secondary Logo

Journal Logo

Finding a Path to Entrustment in Undergraduate Medical Education: A Progress Report From the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group

Brown, David R., MD; Warren, Jamie B., MD, MPH; Hyderi, Abbas, MD, MPH; Drusin, Ronald E., MD; Moeller, Jeremy, MD; Rosenfeld, Melvin, PhD; Orlander, Philip R., MD; Yingling, Sandra, PhD; Call, Stephanie, MD; Terhune, Kyla, MD; Bull, Janet, MA; Englander, Robert, MD; Wagner, Dianne P., MD for the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group

doi: 10.1097/ACM.0000000000001544
Innovation Reports
Free
SDC

Problem To better prepare graduating medical students to transition to the professional responsibilities of residency, 10 medical schools are participating in an Association of American Medical Colleges pilot to evaluate the feasibility of explicitly teaching and assessing 13 Core Entrustable Professional Activities for Entering Residency. The authors focused on operationalizing the concept of entrustment as part of this process.

Approach Starting in 2014, the Entrustment Concept Group, with representatives from each of the pilot schools, guided the development of the structures and processes necessary for formal entrustment decisions associated with students’ increased responsibilities at the start of residency.

Outcomes Guiding principles developed by the group recommend that formal, summative entrustment decisions in undergraduate medical education be made by a trained group, be based on longitudinal performance assessments from multiple assessors, and incorporate day-to-day entrustment judgments by workplace supervisors. Key to entrustment decisions is evidence that students know their limits (discernment), can be relied on to follow through (conscientiousness), and are forthcoming despite potential personal costs (truthfulness), in addition to having the requisite knowledge and skills. The group constructed a developmental framework for discernment, conscientiousness, and truthfulness to pilot a model for transparent entrustment decision making.

Next Steps The pilot schools are studying a number of questions regarding the pathways to and decisions about entrustment. This work seeks to inform meaningful culture change in undergraduate medical education through a shared understanding of the assessment of trust and a shared trust in that assessment.

D.R. Brown is vice chair, chief of family and community medicine, and associate professor of humanities, health, and society, Florida International University Herbert Wertheim College of Medicine, Miami, Florida.

J.B. Warren is assistant professor of pediatrics, Oregon Health & Science University School of Medicine, Portland, Oregon.

A. Hyderi is associate dean for curriculum and associate professor of family medicine, University of Illinois College of Medicine, Chicago, Illinois.

R.E. Drusin is vice dean for education and professor of medicine, Columbia University College of Physicians and Surgeons, New York, New York.

J. Moeller is residency program director and assistant professor of neurology, Yale School of Medicine, New Haven, Connecticut.

M. Rosenfeld is senior associate dean for medical education and associate professor of cell biology, New York University School of Medicine, New York, New York.

P.R. Orlander is associate dean for curricular affairs, vice chair for education, and professor of internal medicine, McGovern Medical School at the University of Texas Health Science Center at Houston, Houston, Texas.

S. Yingling is associate dean for educational planning and quality improvement and assistant professor of medical education, University of Illinois College of Medicine, Chicago, Illinois. She was manager, Division of Education Quality, Institute for Innovations in Medical Education, New York University School of Medicine, New York, New York, at the time this work began.

S. Call is associate chair for education, residency program director, and professor of medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia.

K. Terhune is residency program director and associate professor of surgery, Vanderbilt University School of Medicine, Nashville, Tennessee.

J. Bull is lead specialist in competency-based learning and assessment, Association of American Medical Colleges, Washington, DC.

R. Englander is associate dean for undergraduate medical education, University of Minnesota Medical School, Minneapolis, Minnesota.

D.P. Wagner is associate dean for undergraduate medical education and professor of medicine, Michigan State University College of Human Medicine, East Lansing, Michigan.

Editor’s Note: An Invited Commentary by Th.J.(Olle) ten Cate appears on pages 736–738.

Funding/Support: This work was supported by the Association of American Medical Colleges and by the medical schools participating in the Core Entrustable Professional Activities for Entering Residency pilot.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Previous presentations: The work of the Entrustment Concept Group was presented as a poster at Learn Serve Lead: The AAMC Annual Meeting in November 2015 in Baltimore, Maryland, and in November 2016 in Seattle, Washington.

Correspondence should be addressed to David R. Brown, Herbert Wertheim College of Medicine, 11200 SW 8th St., AHC2 593, Miami, FL 33199; telephone: (305) 348-8388; e-mail: drbrown@fiu.edu; Twitter: @FIUMedicine.

Back to Top | Article Outline

Problem

Competency-based medical education is emerging as the predominant paradigm across the education continuum. Graduate medical education (GME) clinical competency committees periodically assess and report milestones achievements for their residents, but comparable systematic competency assessment and reporting structures do not exist in undergraduate medical education (UME), where the advancement process tends to focus primarily on identifying struggling students.1–3 The transition to a competency- and outcomes-based educational model requires UME to move beyond the traditional time-based curriculum. Increased focus on competencies in GME has exposed a “gap between residency program directors’ expectations and new residents’ performance.”1 To address this gap and ensure that all medical school graduates have a basic level of preparedness for the responsibilities of residency, a drafting panel convened by the Association of American Medical Colleges (AAMC) defined 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs) that all graduating medical students might be expected to perform on day one of residency without direct supervision.1,4 (See Englander et al4 for the complete list of the 13 Core EPAs.)

In addition to addressing the UME-to-GME transition, the Core EPAs framework also offers a practical process for assessing competencies.2,3 It allows educators to take a holistic approach to the assessment of competencies and their corresponding milestones because they represent the activities of

the day-to-day work of the professional; situate competencies and milestones in the clinical context in which we live; make assessment more practical by clustering milestones into meaningful activities; [and] explicitly add the notions of trust and supervision into the assessment equation.4

By creating a shared understanding of specific professional workplace activities, clustering competencies and milestones to fit those activities, and crafting developmental models of the knowledge, skills, and attitudes associated with those activities, the Core EPAs framework guides the “gestalt” of supervisors so they are able to provide effective assessment and feedback about the ability of learners to perform specific professional activities in the workplace.3

Foundational to the Core EPAs framework are the concepts of trust5,6 and supervision,2 which include complex relational issues involving the learner, the supervisor, and the context of the situation.2,5,6 Day-to-day decisions to entrust learners traditionally have been a foundation of clinical education,6 but explicit, summative entrustment decisions using the Core EPAs framework are new to medical education.1–4 Entrustment raises the issue of how to assess learners’ trustworthiness for specific activities, as well as how trust develops in supervisors, in learners, and in curricular and assessment systems. Ten medical schools are working with the AAMC on a pilot project to test the implementation and evaluation of the Core EPAs framework in UME, including how to operationalize the concept of entrustment. In this report, we describe the initial discussions and findings regarding entrustment from this pilot.

Back to Top | Article Outline

Approach

The Core EPAs pilot began in October 2014 with a goal of developing and implementing a process for making summative entrustment judgments for the graduating class of 2019. The pilot was organized into workgroups related to each of the 13 Core EPAs. Additional workgroups were formed to address concepts that affected all of the Core EPAs: Curriculum Development, Assessment, Faculty Development, and Entrustment. A steering committee was made up of the team leaders from each participating medical school. The Entrustment Concept Group, with representation from each school, was charged with guiding the development of the structures and processes for entrustment decision making. Most of the authors of this report are members of this concept group. We reviewed the literature on entrustment and iteratively discussed guiding principles, entrustment processes, trustworthiness frameworks, and findings from the other Core EPA workgroups.

Back to Top | Article Outline

Outcomes

From this literature review, we identified three initial lenses through which to consider the entrustment process—the perceived trustworthiness of the learner, the workplace-based gestalt judgment of a supervisor working with a learner, and the summative formal entrustment decision for each Core EPA. We defined entrustment across each of the Core EPAs as the point at which learners both (1) possess the requisite knowledge, skills, and attitudes needed to perform the EPA and (2) demonstrate specific elements of trustworthiness both foundationally and within the context of entrustment decisions indicating that they are able to perform the EPA without direct supervision.1,6

Back to Top | Article Outline

Creating transparency in formal summative entrustment decision-making processes

We developed guiding principles for making formal summative entrustment decisions that are transparent to faculty and learners. We recommend the following principles to operationalize a formal process for entrustment:

  • Create a process to describe and maintain formal entrustment decisions by a trained group of administrators and faculty,
  • Base entrustment decisions on a longitudinal view of each learner’s performance,
  • Include day-to-day ad hoc workplace entrustment judgments by clinical supervisors in the body of evidence supporting formal entrustment decisions,
  • Explicitly measure attributes of learners’ trustworthiness as foundational to all the Core EPAs (in addition to EPA-specific knowledge and skills),
  • Gather multimodal performance evidence from multiple assessors,
  • Ensure a process for formative feedback, and
  • Ensure that each learner is an active participant in entrustment decisions.
Back to Top | Article Outline

Deconstructing trustworthiness

Kennedy and colleagues6 studied supervisors’ assessments of learners’ ability to work independently, and they identified four dimensions that guided supervisors’ decisions about learners’ trustworthiness for independence: knowledge/skill, discernment of limitations, truthfulness, and conscientiousness. We considered three of these dimensions—discernment, truthfulness, and conscientiousness—essential elements of entrustment that are foundational to all of the Core EPAs. We then further explored these three dimensions of trustworthiness. Like professionalism, trustworthiness includes cross-cutting behaviors and attitudes that appear in multiple contexts. Nonetheless, the individual dimensions of discernment, truthfulness, and conscientiousness unpack trustworthiness into specific observable behaviors. The fourth dimension described by Kennedy and colleagues—knowledge/skill—while foundational to each Core EPA is also context specific. Therefore, the individual Core EPA workgroups, rather than our concept group, were tasked with drafting knowledge/skill-specific developmental paths to entrustment for each of the 13 Core EPAs.3,7 Since trustworthiness crosses all of the Core EPAs, through an iterative group process, we constructed a developmental framework of trustworthiness that could be studied by the Core EPA pilot schools (see Chart 1).3,6,7

An important concept in the Core EPA pilot is continuity with GME. To align with efforts in GME, we conceptualized trustworthiness in the clinical context as a skill with the aspirational “proficient” level of the Dreyfus8 model anchored to the skill level that highly reliable residents possess (see Chart 2). This framework of trustworthiness as aspirational can be used by learners, faculty, and committees charged with making entrustment decisions, through the compilation of the results of multisource assessments mapped to the Core EPAs. Elements of this trustworthiness framework are being integrated into classroom and small-group professionalism assessments and embedded in assessments for specific workplace activities. During the next stage of the pilot, we will explore the consequences of different approaches to implementing this framework across the 10 pilot schools and evaluate the validity of the rating scale across sites.

Back to Top | Article Outline

Implementing an entrustment decision-making process

Each pilot school is exploring how the guiding principles for entrustment that we developed (listed above) may impact its structures and functions and what adaptations may be required. Adoption of these principles affects the prioritization of funding and faculty time and may challenge existing medical school culture. In Table 1, we summarize the plans for applying the guiding principles at the pilot schools as well as important remaining challenges.

Table 1

Table 1

The existence of the following attributes at some of the pilot schools has facilitated operationalizing the guiding principles:

  • Structures for longitudinal relationships between faculty and learners,
  • Portfolios that allow tracking of competency assessment data,
  • Analytic systems that allow the aggregation of competency assessment data into dashboards, and
  • Learner handovers across educational settings within UME.

We also identified common barriers to this work, the most notable being funding and faculty development. While longitudinal faculty–learner relationships facilitate the development of trust, schools with fewer resources find it challenging to create these programs. In addition, some schools have rigorous informatics systems in place to compile multisource assessment data into dashboards, while others are only beginning to explore this possibility and investigate the necessary funding. Finally, while some schools have conceptualized a process to render formal entrustment decisions, most are still in the exploratory phase.

Specific Core EPA–related curricula, point-of-care assessments, and faculty development are essential to adopting systems of formal entrustment decision making; these programs are being developed by the different concept groups and individual Core EPA workgroups.9,10 By providing a transparent model of entrustment for the Core EPAs, we hope to systematize ad hoc workplace entrustment judgments and provide learners with reliable feedback as they progress toward entrustment.

Back to Top | Article Outline

Next Steps

In most medical schools, a single advancement committee focuses on identifying struggling students, never discussing the majority of students who are not struggling. Therefore, monitoring every student’s progress and ultimately using aggregate evidence to render an entrustment decision for each Core EPA is a major shift. The Core EPA pilot schools are implementing a variety of EPA-specific assessments, as well as assessments of trustworthiness, in a variety of contexts across curricula. With this shift, the pilot schools are working to engage students and faculty, compile evidence, and pilot the work of entrustment committees. Our work, described above, raised key questions in three categories (discussed below) that require further exploration. Although each pilot school is expected to discover different solutions to these challenges, their collective efforts and the resulting lessons and conversations will help us to further understand these core concepts.

Back to Top | Article Outline

Evaluating the entrustment process

The pilot schools are in the process of defining developmental paths to entrustment that can be monitored as students progress through UME. We will examine the role of longitudinal supervisory or coaching relationships, student-driven assessments, trustworthiness-based assessments, supervisory scales, EPA-specific assessments, and enhanced transparency about students’ progression towards entrustment. The pilot schools will compare the amount and type of performance evidence that they are able to compile on individual students and discuss their confidence in that evidence. Since medical school graduates commonly transition to new institutions for residency, we must rely on one another’s entrustment decisions. Open discussions among the pilot participants regarding the limitations of our assessment evidence will help us to create a shared understanding of the curricula, workplace activities, assessment systems, and faculty development necessary to enable institutions to entrust students to perform the Core EPAs. The role of a formal educational handover from UME to GME is also being explored.

Back to Top | Article Outline

Compiling evidence of trustworthiness

We also plan to study explicit assessments of the elements of trustworthiness (discernment, conscientiousness, and truthfulness) as professional attributes that can be developed through coaching. We will seek evidence of trustworthiness in non-patient-care settings, in clinical settings independent of Core EPA performance, and within the act of performing a given Core EPA. Lapses in trustworthiness may occur in situations of high stress or extreme fatigue, so we will consider how students can anticipate such lapses, which lapses are remediable, and which are not. Providing students with this kind of feedback may profoundly impact their subsequent strategies and behaviors. Multisource assessment of students in the context of the Core EPA framework may show that evidence of trustworthiness is demonstrable throughout the curriculum. Longitudinal use of the trustworthiness framework (see Chart 1) could enable students to identify training situations in which consistent discernment, conscientiousness, or truthfulness is challenging to maintain. Furthermore, while we believe that our trustworthiness framework will be useful to inform entrustment decisions, the pilot schools have not yet begun testing or validating it as an assessment tool. Finally, in addition to studying the relevant components of the trustworthiness model created by Kennedy and colleagues, we plan to explore how factors such as communication skills, motivation, emotional intelligence, and systems orientation may impact supervisors’ assessment of students’ trustworthiness.4–7

Back to Top | Article Outline

Ensuring trust in entrustment

As trust is reciprocal, our efforts to promote a safe transition from UME to GME for our students and the safety of their future patients seek to engender trust from all stakeholders. The role of the student in the entrustment process is crucial; we must win the trust of our students by establishing the validity, reliability, and appropriateness of our new assessments and by instituting mechanisms to help students meet expectations for entrustment. The limited opportunities for students to participate directly in patient care and perform the Core EPAs in current educational environments may pose a significant systematic barrier to EPA measurement. The identification of developmental paths toward entrustment can assist in identifying appropriate activities for different levels of students for each of the Core EPAs. By sharing findings among institutions, we plan to explore which interventions in curriculum, assessment, faculty development, feedback, and remediation best support the entrustment process. These factors all contribute to determining whether it is possible under the current rules and structures of UME to entrust graduating medical students to perform all 13 Core EPAs.

The Core EPAs framework requires us to learn new terminology, make changes in faculty and student approaches to feedback and assessment, and modify the structures and processes of UME assessment systems. The work of the Entrustment Concept Group specifically, and the Core EPAs pilot as a whole, seeks to inform meaningful culture change in UME through a shared understanding of the assessment of trust and a shared trust in that assessment.

Acknowledgments: The authors thank the Association of American Medical Colleges, Carol Aschenbrener, Maryellen Gusic, and all of the members of the Core Entrustable Professional Activities for Entering Residency pilot for their support, inspiration, and contributions to this report. The members of the Entrustment Concept Group include Judith L. Bowen, MD, David Brown, MD, Stephanie Call, MD, MSPH, Ronald E. Drusin, MD, Abbas Hyderi, MD, MPH, Jeremy Moeller, MD, Philip R. Orlander, MD, Melvin Rosenfeld, PhD, Kyla Terhune, MD, Dianne P. Wagner, MD, Jamie B. Warren, MD, MPH, and Sandra Yingling, PhD.

Back to Top | Article Outline

References

1. Englander R, Flynn T, Call S, et alToward defining the foundation of the MD degree: Core entrustable professional activities for entering residency. Acad Med. 2016;91:1352–1358.
2. Chen HC, van den Broek WE, ten Cate OThe case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015;90:431–436.
3. Aylward M, Nixon J, Gladding SAn entrustable professional activity (EPA) for handoffs as a model for EPA assessment development. Acad Med. 2014;89:1335–1340.
4. Englander R, Aschenbrener CA, Call SA, et alCore entrustable professional activities for entering residency. MedEdPORTAL. 2014. https://www.mededportal.org/icollaborative/resource/887. Accessed November 3, 2016.
5. ten Cate O, Hart D, Ankel F, et alEntrustment decision making in clinical training. Acad Med. 2016;91:191–198.
6. Kennedy TJ, Regehr G, Baker GR, Lingard LPoint-of-care assessment of medical trainee competence for independent clinical work. Acad Med. 2008;83(10 suppl):S89–S92.
7. Brown DR, Gillespie CC, Warren JBEPA 9—Collaborate as a member of an interprofessional team: A short communication from the AAMC Core EPAs for Entering Residency pilot schools. Med Sci Educ. 2016;26:457–461.
8. Dreyfus SEThe five-stage model of adult skill acquisition. Bull Sci Technol Soc. 2004;24:177–181.
9. Lomis K, Amiel JM, Ryan MS, et alImplementing an entrustable professional activities framework in undergraduate medical education: Early lessons from the AAMC Core Entrustable Professional Activities for Entering Residency Pilot. Acad Med. 2017;92:765–770.
10. Favreau MA, Tewksbury L, Lupi C, et alConstructing a shared mental model for faculty development for the Core Entrustable Professional Activities for Entering Residency. Acad Med. 2017;92:759–764.
© 2017 by the Association of American Medical Colleges