In this month’s issue of Academic Medicine, Aylward and colleagues1 describe their process for implementing an entrustable professional activity (EPA) in a medicine–pediatrics residency program. The concept of EPAs, originally described by ten Cate2 and then further elucidated by ten Cate and Scheele,3 has been gaining traction in medical education, particularly in obstetrics–gynecology in the Netherlands and in psychiatry in Australia and New Zealand, where they form the basis for decisions around transitioning from residency training into practice. EPAs as a concept are gaining momentum throughout the world because of both the concept’s practicality—EPAs have great “face validity” for faculty at the front lines—and the trust and level of supervision they add to the assessment equation for learners. Medical education conferences across the globe are rich with workshops and presentations describing proposed and/or early uses of EPAs in the clinical learning environment.
The concept is so enticing to medical educators, in fact, that a drafting panel sponsored by the Association of American Medical Colleges (AAMC) used the EPA framework for their work in defining what a first-day intern should be able to do without direct supervision, even though the EPA concept had been previously applied only in the graduate medical education (GME) space. The resultant draft publication, “Core Entrustable Professional Activities for Entering Residency,” was introduced at the AAMC Annual Meeting in November 2013.4 The final product was disseminated in the spring of 2014.
Implementing EPAs: There’s the (Potential) Rub
As with any new educational intervention, implementation and study thereof is the key to getting uptake beyond the early adopters. Understanding what works, in what contexts and under what conditions, is critical to dissemination and adoption in different settings. Implementation of the EPA concept to date in the Netherlands has focused on defining the EPAs and then measuring progress in learners toward unsupervised practice. The assessments are based on where the learner is on a five-point level-of-supervision scale in which the fourth level is entrustment to effectively perform the EPA without supervision, and is the requirement for transition to practice. Using this system, then, competence is inferred rather than measured.5
One of the critical questions facing implementation of EPAs in the United States has been how to integrate the concept with the Accreditation Council for Graduate Medical Education (ACGME) competencies framework and, more important, the recent rollout of the ACGME’s Milestones Project.6 To meet program accreditation and individual certification requirements in the United States, EPA implementation requires a slightly different framing, linking EPAs more explicitly to the competencies and milestones, at least at the GME level.
Aylward and colleagues’ article is exciting because the model they describe for developing and implementing an EPA not only helps to fulfill program requirements but also explicitly links competencies and milestones to the EPA. This linkage allows a more definitive “picture” of the learner at each of the levels of supervision required, which can be shared with those directly observing the learner. In so doing, this linkage provides a potential model for moving the needle toward more reliable assessment of EPAs and decisions around entrustment. We also suspect that a clearer picture of learners at the various levels of performance can have enormous benefit to feedback discussions for the learners on the path to entrustment. The current state of the art for EPA performance feedback certainly provides the basis for a powerful discussion. This feedback might include, as an example, telling a learner that one is comfortable allowing him or her to perform an EPA when the supervisor is immediately available (i.e., under indirect supervision) but not yet comfortable with allowing unsupervised practice. We hope these discussions have a much greater impact than the traditional “great job,” or “needs to read more,” or “great team player.” However, EPA performance feedback in its current form requires implicit reasoning on the part of the supervisor to help the learner see the path to unsupervised practice. Our hope is that explicitly linking the EPA to its critical competencies (those absolutely required for entrustment to occur), and then to the developmental milestones for those competencies, will provide a much more comprehensive picture of the learner, which will enrich the supervisor–learner discussion and speed the journey to entrustment.
To make the distinction a bit clearer, current EPA implementation in locales such as the Netherlands, where there are no additional requirements of milestones, has used the decision around entrustment as a starting point. From the supervisor’s perspective, she is primarily responsible for deciding the level of supervision a learner requires for an EPA and then conveying why she has chosen that level, until a learner reaches the unsupervised practice level and is “entrusted.” She will undoubtedly use language around the functions or CanMEDS roles (the physician competency framework used in the Netherlands) that require integration for that EPA, but her language and determinations may be quite different from one of her colleague’s, regardless of whether they have reached the same or different conclusions about the learner’s required level of supervision. The learner shares responsibility for determining the required level of supervision through routine “defense” of her portfolio, but again this defense is based on a gestalt of performance of the EPA’s functions or the required roles using the CanMEDS framework, rather than a specific behavioral description of one’s level of performance compared with a standard.5 This approach might be seen as the “top-down” approach in which the supervisor’s and/or learner’s gestalt results in a determination of the level of supervision which then results in inference about competence.
What Aylward and colleagues have done is to start with the competencies and their milestones to build a picture of learners at the various levels—a more “bottom-up” approach. Their result is a behavioral description based on the critical competencies and their milestones for learners who merit each of the five levels of supervision (unable to perform even under supervision; able to perform with direct supervision; able to perform with indirect supervision; able to perform unsupervised; able to supervise).2 The supervisors are then using those descriptions to inform their “gestalt” or, as the authors call it, their “guided gestalt.” In this case, the global sense of the supervisor about the performance of the learner is still important, but it is informed by the standard descriptions. Interestingly, Aylward and colleagues did not find good interrater reliability despite the detailed behavioral descriptors, which they surmise was due to insufficient faculty development.
In our own work, we are creating scripts of clinical vignettes and recording videos of learners at each of the performance levels to accompany the behavioral descriptors as a further step toward the goal of informing the reliability and validity of entrustment decisions. We also suspect that supervisors will become increasingly expert in matching the identifying behaviors to the learner’s level over time.
We think the combination of the top-down and bottom-up approach to evaluating EPA performance is ultimately the most powerful. EPAs provide the clinical context for the assessment of competencies, which uses a “panoramic” lens for assessing learners who must integrate competencies to deliver care, and their milestones, which provide a “zoom” lens for assessing the learner at a more granular level. The former lens will be sufficient for most learners, but the latter becomes critical for those who struggle and need to remediate.
What EPAs Add to the Current Landscape
EPAs provide a wonderful strategy and context for promoting and assessing competence, which will by necessity require differences in implementation across countries and perhaps disciplines. But if the old way of assessing learners was essentially “I’ll know it when I see it,” then EPAs move us to “I’ll know what’s important for a learner to perform and I’ll know it when I see it.” The addition of behavioral descriptors to the equation moves us to “I’ll know what’s important for the learner to perform, I’ll know what specifically to look for so I can recognize it when I see it, and I’ll be looking for and recognizing the same thing as my colleague.”
Aylward and colleagues’ step-by-step process for implementing and assessing an EPA reminds us of the importance of not only the implementation process itself but also studying that process in disseminating educational innovations. We hope and trust that this work is one of many publications to come over the ensuing years that helps medical educators do the work of “bench to bedside” translation.
1. Aylward M, Nixon J, Gladding S. An entrustable professional activity (EPA) for handoffs as a model for EPA assessment development. Acad Med. 2014;89:1335–1340
2. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39:1176–1177
3. ten Cate O, Scheele F. Viewpoint: Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–547
4. Association of American Medical Colleges (AAMC). . Core Entrustable Professional Activities for Entering Residency. 2013 https://www.mededportal.org/icollaborative/resource/887
. Accessed April 7, 2014
5. Scheele F, Teunissen P, Van Luijk S, et al. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach. 2008;30:248–253
6. Accreditation Council for Graduate Medical Education (ACGME). . Milestones Project (Powerpoint). 2013 http://www.acgme.org/acgmeweb/Portals/0/PDFs/ACGMEMilestones-CCC-AssesmentWebinar.pdf
. Accessed April 7, 2014