Building a Framework of Entrustable Professional Activities, Supported by Competencies and Milestones, to Bridge the Educational Continuum

Abstract

The transition to competency-based medical education (CBME) and adoption of the foundational domains of competence by the Accreditation Council for Graduate Medical Education, Association of American Medical Colleges (AAMC), and American Board of Medical Specialties’ certification and maintenance of certification (MOC) programs provided an unprecedented opportunity for the pediatrics community to create a model of learning and assessment across the continuum. Two frameworks for assessment in CBME have been promoted: (1) entrustable professional activities (EPAs) and (2) milestones that define a developmental trajectory for individual competencies. EPAs are observable and measureable units of work that can be mapped to competencies and milestones critical to performing them safely and effectively.

The pediatrics community integrated the two frameworks to create a potential pathway of learning and assessment across the continuum from undergraduate medical education (UME) to graduate medical education (GME) and from GME to practice. The authors briefly describe the evolution of the Pediatrics Milestone Project and the process for identifying EPAs for the specialty and subspecialties of pediatrics. The method of integrating EPAs with competencies and milestones through a mapping process is discussed, and an example is provided. The authors illustrate the alignment of the AAMC’s Core EPAs for Entering Residency with the general pediatrics EPAs and, in turn, the alignment of the latter with the subspecialty EPAs, thus helping build the bridge between UME and GME. The authors propose how assessment in GME, based on EPAs and milestones, can guide MOC to complete the bridge across the education continuum.

The adoption of the six core competencies of the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) as the foundation for outcomes-based undergraduate medical education (UME) by the Association of American Medical Colleges (AAMC), for graduate medical education (GME) by the ACGME, and for board certification as well as maintenance of certification (MOC) by the ABMS has provided an unprecedented opportunity to create a seamless continuum of learning and assessment in medicine.^1–3 Parallel advancements in assessment through entrustable professional activities (EPAs)⁴ and educational milestones, the latter effort initiated by the ACGME and the ABMS,⁵ provide the foundation and the building blocks, respectively, to realize the implementation of this continuum.

EPAs, first introduced by ten Cate⁶ in the Netherlands in 2005, are receiving increasing attention internationally as a framework for meaningful assessment of physician competence at both the UME and GME levels.^7–11 As measureable units of observable work, EPAs describe important routine activities of a given specialist or subspecialist that require integration of competencies for safe and effective performance. In the aggregate, they define the specialty or subspecialty.⁴ The qualifier of “entrustable” aligns with the ultimate ability of an individual to perform the professional activity safely and effectively without supervision and has resulted in the proposal of rating scales with levels of supervision leading to readiness for unsupervised practice.^8,12,13 Specialties have developed EPAs for residents, and the AAMC has developed Core EPAs for Entering Residency.^7,9–11 The Core EPAs for Entering Residency set the expectation that trainees should be able to perform 13 EPAs on Day 1 of residency without direct supervision, but experts agree that ongoing observation and feedback are essential for further professional development.

Parallel to and complementing the work of EPAs, the ACGME and member boards of the ABMS partnered to initiate the process that developed the milestones, with each specialty creating a shared mental model of performance levels or milestones for competencies within the specialty.⁵

The pediatrics community has integrated the two frameworks of EPAs and milestones.¹⁴ This integration provides a pathway that leads to assessment across the educational continuum from UME to GME and from GME to practice. We wrote this article to (1) briefly describe the evolution of the Pediatrics Milestone Project and the identification of EPAs for the specialty and subspecialties of pediatrics, (2) discuss the value added to assessment by integrating the EPAs with competencies and milestones, (3) illustrate the bridge between UME and GME built upon alignment of Core EPAs for Entering Residency with pediatrics EPAs, and (4) propose how assessment in GME based on EPAs and milestones can guide MOC to complete the bridge across the educational continuum, not only for pediatrics but also for other specialties.

The Unfolding of the Pediatrics Milestone Project

In 2009, the ACGME and the American Board of Pediatrics (ABP) invited the pediatrics community to initiate the development of the milestones for pediatrics. Creation of the guiding principles, process, and outcome of this three-year endeavor has previously been described.¹⁵ The full pediatrics milestone document is published as a supplement to Academic Pediatrics.¹⁶

In brief, the pediatrics milestones are narrative descriptions of behaviors across the developmental continuum beginning with a novice learner (early medical student) and progressing along the continuum to advanced beginner, then to competent learner, proficient learner, and, finally, expert learner (after years of deliberate practice). The competencies (e.g., “gather essential and accurate information about the patient”) and thus their narratives are context independent and therefore do not define the complexity of the patient encounter or the clinical environment in which the skills will be demonstrated. Reporting of individual performance levels or milestones for 21 of the 48 competencies, within the six competency domains, on a semiannual basis began in 2014 as part of the data collection for the ACGME’s Next Accreditation System.^5*

Identifying General Pediatrics EPAs

As the first iteration of the milestones was being developed, a body of literature on EPAs was emerging. As early as 2005,⁶ ten Cate began to write about the construct of EPAs with a seminal article⁴ published in 2007 that spoke to the implementation of their use for assessment in obstetrics–gynecology in the Netherlands. By focusing on care delivery, which requires the integration of multiple competencies, EPAs align what clinicians assess with what they do in the authentic environment, adding meaning and value to assessment. Some of us (C.C., R.E., J.G.) began to identify the potential list of EPAs for the general pediatrician. Through an online survey, we vetted the initial list of EPAs with members of the Association of Pediatric Program Directors. In response to their feedback, we revised the list and then sent a second survey asking for additional refinements. We also communicated with other members of our community asking for their input. The final first iterations of the 17 EPAs defining the desired outcomes for the pediatrics residency-to-practice transition appear in the middle columns of Charts 1 and 2, with 6 in Chart 1 and the remaining 11 in Chart 2.

Chart 1: Examples of EPAs That Directly Align Across the Continuum of Pediatrics Education and Training^a

Chart 2: Examples of EPAs That Are Sequenced Across the Continuum of Pediatrics Education, With Subsequent EPAs Building on Preceding EPAs^a

As conceived by the pediatrics community, the EPA and competency/milestone frameworks should be integrated, not only in pediatrics but for other specialties as well:

The combination of the top-down and bottom-up approach to evaluating EPA performance is ultimately the most powerful. EPAs provide the clinical context for the assessment of competencies, which uses a “panoramic” lens for assessing learners who must integrate competencies to deliver care, and their milestones, which provide a “zoom” lens for assessing the learner at a more granular level.¹⁷

Pangaro and ten Cate¹⁸ suggest conceptual frameworks for assessment that can be categorized as analytic, synthetic, or developmental. The ACGME competencies would be considered an analytic framework, breaking down core domains into further detailed descriptions of competencies. Milestones are an example of a developmental framework, representing a logical progression of steps over time. Finally, EPAs represent a synthetic framework that combines multiple domains of competence. The framework we propose embraces the above models in that it is a framework of EPAs integrated with competencies and milestones.

Once identified, EPAs are judiciously mapped to those competencies and milestones critical to making an entrustment decision for that specific EPA.¹⁹ Through this mapping process we create, for pediatrics, a matrix of the critical competencies for a given EPA and their respective milestones (see Chart 3). The first column of the matrix (on the left) lists the competencies that are critical for making an entrustment decision for the given EPA. Each row represents the progression of milestones for the given competency. Columns two through six represent all of the milestones for a given level of performance (novice to expert) across the competencies in the map. For example, in Chart 3, the behaviors of the novice are described in the second column, advanced beginner in the third, competent learner in the fourth, proficient learner in the fifth, and expert in the sixth column.

Chart 3: The “Handover EPA”a Mapped to Competencies and Their Milestones That Are Critical to Making an Entrustment Decision, With Examples of Behaviors Associated With the Milestone for One of the Performance Levels^b

Our model suggests that reading down any column provides the requisite behaviors of a learner at a specific level of performance for all of the critical competencies required for an entrustment decision on that EPA. For example, integrating the behaviors in the third column of Chart 3 into a clinical vignette paints a picture of an advanced beginner performing the EPA. This same process would apply to each performance level, the end result being a picture of a learner at each performance level carrying out the EPA.¹⁴ This shared mental model of performance could then be applied to learner assessment, with faculty choosing the vignette that most closely aligns with the behaviors of the trainees that they have directly observed over some period of time to assess performance level.

Learners in the workplace will likely not demonstrate all the described behaviors of one column or performance level but, rather, some of the behaviors from two or possibly three performance levels. The shared mental models painted by the behaviors in each column are meant to be anchors that frame a discussion between the learner and a mentor or the learner and a clinical competency committee around which behaviors have been demonstrated and which behaviors are desired. The information gleaned about a learner’s performance level using this framework will be combined with many other data points that experts will use in making a balanced judgment regarding entrustment.

Regehr et al²⁰ provide some support for this assessment strategy. Creating video vignettes of each of the five performance levels provides an additional tool for faculty development in the assessment of EPAs and milestones.²¹ Using EPAs, with their dimension of entrustment, allows a formal decision to trust a learner to safely perform the professional activity without direct supervision at the UME-to-GME transition and without supervision at the GME-to-practice transition, thus bringing the required level of supervision as another data point into the assessment equation.¹³ Being mindful that context matters, verification of a learner’s readiness to practice at a given level of supervision through ongoing observation and feedback is warranted at transition points until readiness for unsupervised practice is truly achieved. Even then, significant changes in context may require self-assessment, peer observation, and coaching until readiness for unsupervised practice is regained.

As the development of the general pediatrics milestones was being completed, the ACGME required that each of the subspecialties within a given specialty develop subspecialty-specific milestones. The pediatrics milestones were unique in that they spanned the continuum from early medical students to practicing pediatricians, thus including performance levels applicable for fellows. Embedding the competencies in clinical contexts or EPAs relevant to each of the 14 pediatrics subspecialties with ABP certification was the next and best step for our community. The ACGME agreed that this plan represented a reasoned approach, and the ABP took the lead in this effort.

Identifying EPAs for the Pediatrics Subspecialties

In March 2013, the ABP, with the help of the Council of Pediatric Subspecialties (CoPS), convened a workshop for education thought leaders from each of the ABP-certified pediatrics subspecialties to identify EPAs common to the subspecialties.²² The two-day workshop began with faculty development around defining and distinguishing pediatrics EPAs, competencies, and milestones and illustrating their relationships to each other. The subspecialists then reviewed the general pediatrics EPAs, adopting five that spanned the generalist to subspecialist role and that applied to all subspecialties. Through a consensus process, the group identified two additional common subspecialty EPAs (see Charts 2 and 3), one on leading within the profession and a second on scholarly activity. With regard to the latter EPA, the ABP requires the completion of scholarly work during fellowship in order for the learner to sit for the subspecialty certification examination, which provides the rationale for the identification of an EPA on scholarly activity. The second day of the workshop concluded with a discussion of the approach to identifying subspecialty-specific EPAs to maximize consistency across subspecialties.

The participants left with two challenges: (1) to lead the effort to identify the subspecialty-specific EPAs for their community by developing a title, short description, and functions (list of tasks that one needs to do to perform the EPA) as well as to judiciously map each EPA to its critical competencies²³; and (2) with the help of CoPS, to obtain and incorporate feedback from their communities into the work product. Each subspecialty community identified between three and six subspecialty-specific EPAs to add to the seven common subspecialty EPAs. Leadership from the ABP worked with each subspecialty community to ensure a consistent approach.

The Bridge Across the UME-to-GME Continuum

Shortly after work began on the pediatrics EPAs, the AAMC constituted a multidisciplinary drafting panel to identify the Core EPAs for Entering Residency. A panel comprising approximately 100 thought leaders across the spectrum of medical education provided feedback to the drafting panel, resulting in an iterative process of creation, reaction, and revision. The final document details 13 EPAs that medical students should be entrusted to perform without direct supervision on Day 1 of residency training.⁷

With the creation of the Core EPAs for Entering Residency, the pediatrics community now had the infrastructure to build the bridge that connects EPAs for medical students with general pediatrics EPAs for all residents, which in turn connects to the pediatrics EPAs for fellowships. Review of the lists of EPAs illustrates the natural linkages that represent either a direct alignment (e.g., as a student, collaborating as a member of an interprofessional team; as a resident or fellow, leading an interprofessional health care team—see Chart 1) or where student EPAs serve as building blocks for residency EPAs, the latter serving as building blocks for fellowship EPAs (e.g., as a student, entering and discussing orders and prescriptions; as a resident or fellow, managing patients with common acute illnesses in a variety of settings—see Chart 2). This bridge across the continuum of learning and assessment is reminiscent of what Harden and Stamper²⁴ highlighted as the value of the “spiral curriculum,” where the “competence of students increases with each visit until the final overall objectives are achieved.”

The Missing Link: Guiding MOC With GME EPAs

On the basis of performance in residency and fellowship, one can envision specific EPAs and competencies forming the foundation for improvement during the first cycle of MOC and beyond. As a prerequisite, our community must decide the performance level that equates with entrustment for each EPA, for residents as well as fellows, and whether a level of supervision short of readiness for unsupervised practice for any of the EPAs will be acceptable for transition from residency or fellowship into practice.

The ABP just initiated research with our community to address these questions for pediatrics learners. For example, in a case where a learner is entrusted with 16 of the 17 EPAs, with the exception being the EPA to “manage information from a variety of sources for both learning and application to patient care,” would we accept a level of performance that equates with a requisite period of indirect supervision or guidance in a new practice by a peer/colleague who is more experienced in this professional activity? Or consider this scenario: A resident is entrusted to perform all 17 EPAs at the completion of training and enters a general pediatrics practice that cares for a large number of adolescent patients, many of whom are struggling with either an anxiety disorder or depression. While the new practicing physician had demonstrated the requisite competencies to be entrusted during residency to perform without supervision the EPA “Assess and manage patients with common behavior/mental health problems,” the more limited patient volume and experience during training did not allow for the development of proficiency. The new practicing physician realizes that patients will benefit if he or she continues to move along the developmental trajectory toward proficiency and expertise. Seen in this light, EPAs may focus reflection on practice that helps individuals identify gaps and seek learning activities to fill those gaps.

Of note, entrustment for unsupervised practice during training does not equate with expertise. In both of these examples, the opportunity to address identified gaps during the transition from residency to practice presents an opportunity to design a cycle of MOC that enhances the professional development of the pediatrician—or any other physician—toward expertise. Investment in improvement through continuous professional development and MOC should be used to facilitate a process of structured learning throughout a physician’s career. When gaps are identified during training, learning activities can be initiated as part of an individualized curriculum and continued into MOC. In addition, there may be new EPAs identified as a physician’s career evolves or changes in scope. According to ten Cate et al,²⁵ “entrustment decisions should … have an expiration date if no or too little practice has occurred.” Further work in this area holds promise of ensuring that MOC remains meaningful throughout a physician’s career.

The Critical Contributions of the Continuum to Competency-Based Education

One of the current concerns about trainees is that they may not be well prepared for real-world practice. The EPAs for residents and fellows are based on what one expects in practice, and thus a backward-visioning process is used to set expectations and outcomes for GME.²⁶ Thus, the continuum enables the alignment of the work of UME and GME with the desired outcomes for practicing physicians. For example, in the UME setting it is essential to lay the foundation for handovers by introducing a template, which includes situation awareness, illness severity, action planning, and contingency planning as well as the communication skills to effectively engage as either a sender or a receiver of the handover (e.g., skill at readbacks). These foundational skills as well as content knowledge must be advanced during GME, building on and reinforcing what came before to provide deeper learning and ultimately readiness for entrustment. In addition, faculty development for assessment can focus on experience with a given set of tools used across the continuum, developing expertise among faculty raters.

Next Steps

Although the road ahead is long, for the first time it is continuous. As the work of the pediatrics community indicates, bridges are available to connect us at the transition points from UME to GME and from GME to fellowship and practice for any specialty that has developed EPAs. As our response to those who will invariably question whether the road ahead will lead to a physician workforce that delivers better quality of care and who are reluctant to follow a path without proof, we offer three thoughts:

First, although we will not have proof that our proposed road leads to desired patient care outcomes until implementation and study, we have little evidence supporting the current model despite its entrenchment, and we have clear evidence that this discontinuous path leads us away from, rather than closer to, desired outcomes.^27,28

Second, program directors’ expectations of what new residents should be able to do are not aligned with residents’ actual skills.²⁹ First-year residents also report performing some professional activities with less supervision than clinical supervisors say they expect, particularly at night.³⁰ Misalignment between actual skills and expected skills or between an expected level of supervision and an actual level of supervision puts a patient’s safety at risk and puts the learner in a position of making a potentially preventable medical error.

Third, the scaffolding that supports the bridges we are building is constructed of sound educational theory.^20,31–33

We did not create our list of EPAs with broad input from practitioners or input from patients; however, practicing general pediatricians and pediatric subspecialists created the original lists. These lists were subsequently vetted among our various communities of practice, which include general pediatricians and the 14 subspecialties for which ABP certification is offered, and we revised the lists on the basis of their feedback. However, we need to “test” the lists, which we are currently doing, with studies at both the residency and fellowship levels in pediatrics. Of note, the AAMC is engaging a number of medical schools in piloting the Core EPAs for Entering Residency.³⁴ Testing in practice will come by having trainees carry EPAs from GME into practice for the first cycle of MOC as a way of moving them further along the trajectory from competent/proficient to expert. Plans for testing in practice are also on the horizon.

Dr. David Leach, a former executive director of the ACGME, a visionary who saw the road ahead before we could even begin to imagine it, said of assessment that it is

dependent on an integrated version of the competencies, whereas measurement relies on a speciated version of the competencies. The paradox cannot be resolved easily. The more the competencies are specified, the less relevant to the whole they become.³⁵

The integration of the EPAs, compe tencies, and milestones across the educational continuum provides the road map to resolving this paradox.

Acknowledgments: The authors wish to thank the I-PASS Entrustable Professional Activity (EPA) Working Group, I-PASS Institute, for review and input on the handover EPA.