Secondary Logo

Journal Logo

Articles

Faculty Development Revisited: A Systems-Based View of Stakeholder Development to Meet the Demands of Entrustable Professional Activity Implementation

Lupi, Carla S. MD; Ownby, Allison R. PhD, MEd; Jokela, Janet A. MD, MPH; Cutrer, William B. MD, MEd; Thompson-Busch, Angela K. MD, PhD; Catallozzi, Marina MD, MSCE; Noble, James M. MD, MS, CPH; Amiel, Jonathan M. MD; for the Association of American Medical Colleges Core Entrustable Professional Activities for Entering Residency Faculty Development Concept Group

Author Information
doi: 10.1097/ACM.0000000000002297
  • Free

Abstract

A premise of competency-based education (CBE) is that progression through a curriculum should be driven by a learner’s attainment of specific observable abilities that integrate knowledge, skills, and attitudes.1 The concept of CBE is powerful in its simplicity, but its implementation and uptake have been slow. Historically, efforts to implement CBE curricula in the health professions have faced at least two areas of substantial systemic barriers. First, assessment of competencies has proved difficult because of the fractured learning environment.2 Learners and faculty are often not in situations that allow for continuity and communication about learners’ skills; therefore, it becomes difficult for faculty to assess their competency attainment. The second, which some argue as the most important barrier, has been the lack of realistically resourced, deliberate faculty development.2 Faculty leading implementations of the CBE frameworks have commonly lacked the training, time, and institutional backing they need to adapt a CBE program to their local environment and train a large and complex group of clinician–educators. The broad taxonomies for describing the competencies of health professionals have also proven difficult to use as an organizing framework for effective training in assessment across institutions. Incomplete implementations have engendered angst and, understandably, skepticism within the medical educator community.3,4

Recently, we have entered a new era in CBE theory in the health professions that brings with it a potential way forward. The CBE construct has been extended to define health professionals’ specific observable abilities as tasks known as entrustable professional activities (EPAs). EPAs holistically synthesize competencies into the universal tasks of patient care and focus assessment on entrustability. In doing so, they offer a construct that aligns so well with current health care practices and with the assessment required to train learners for independent practice that some describe them as “intuitive.”5,6

Aimed at narrowing the gap between medical school graduates’ actual abilities and those expected of them on the first day of residency, the Association of American Medical Colleges (AAMC) commissioned a drafting panel to delineate the Core EPAs for Entering Residency (Core EPAs).5 The AAMC then established an interinstitutional group to pilot EPA implementation in 2015. Recognizing the assertion of the Core EPAs’ drafting panel that “the success of this work will require faculty development,” pilot organizers formed a concept group dedicated to faculty development. The group has previously described the four essential dimensions for development of faculty implementing the Core EPAs: workplace-based assessment (WBA); coaching and feedback; role modeling for informed self assessment and reflective practice; and peer guidance skills through a community of practice.7

In this second work, we take a systems-based view, recognizing that adequate training is essential to many participants in CBE implementation. We summarize five domains of faculty development as described by Steinert,8 enumerate the faculty and other stakeholders requiring development for Core EPA implementation, and describe their specific needs.

Domains of Faculty Development

Steinert has described the scope of faculty development as including five key domains:

  • Teaching improvement: including the design and planning of learning activities, teaching and supporting learners, and assessing and providing feedback to learners.
  • Leadership and management: whereby leadership produces change and movement, and management produces order and consistency.
  • Research capacity building: including using scholarly resources, study design (forming high-quality research questions, ethical conduct of research, appropriate data analysis), competing for grants, and disseminating scholarly work.
  • Academic and career development: including recruitment, orientation to roles and culture, assessment of achievement, ongoing professional learning, and retirement.
  • Organizational change: including developing shared goals and values, promoting cohesion, and aligning efforts.

All five domains are pertinent to CBE, but each cohort of faculty requires specific development within unique sets of domains, which this article delineates.

Faculty and Stakeholder Definitions

An EPA-based system of medical education must effectively scaffold the functions of workplace-based learning and assessment, didactic learning, assessment of professional behaviors wherever appropriate, overall monitoring and support of progress toward entrustment, summative entrustment decision making, curriculum planning with organizational change, and resource management. On the basis of these functions, we have identified key stakeholders in this system as follows:

  • Students
  • Didactic faculty
  • Residents and other postgraduate trainees
  • Short-term clinical supervisors
  • Longitudinal clinical supervisors and clinical course directors
  • Portfolio coaches
  • Entrustment committee members
  • Faculty and deans responsible for oversight of professional behaviors
  • Curriculum deans and resource managers
  • Faculty developers

Because the field is developing, and no single path to entrustment at the undergraduate medical education (UME) level has been vetted, institutions may distribute the work of various stakeholder groups somewhat differently than we contemplate here. We also recognize the possibility that individuals may simultaneously serve in two or more stakeholder roles, and move between these roles. Also, we use the term “clinical supervisor” to include the range of supervising health care professionals (e.g., nurses, therapists) and clinical consultants with whom medical students may work in the clinical setting.

Required Knowledge and Skills of Stakeholder Groups

To elaborate on our earlier shared mental model,7 we have considered each of these groups in relationship to Steinert’s8 five domains of faculty development, asking what are the requisite knowledge, skills, and values to fulfill each role in EPA implementation (Table 1). Further elaboration is offered in the following.

Table 1
Table 1:
Requisite Stakeholder Skills and Knowledge for EPA Implementation, Based on Steinert’s8 Domains of Faculty Development

Students

All models of CBE require active engagement on the part of students in their training.2 This requirement is heightened in an EPA-based system that emphasizes increasing trust as a dimension of progression, and the role of direct observation in judging trust as well as competence. Students earn trust through demonstration of clinical skill, truthfulness, conscientiousness, and discernment that results in self-directed learning and appropriate seeking of help and feedback.9

To participate in an EPA-based system, students will need orientation to their roles, a working knowledge of entrustment and the specific EPAs, and an understanding of the importance of direct observation. They will also need to espouse the mindset of assessment for learning, which will facilitate their adoption of newer skills in feedback seeking and processing and their development as self-directed learners.10,11

Finally, to participate in the implementation of EPAs, it will be important to include student voices. Representation of student perspectives in early institutional efforts will contribute to the steep curve of continuous quality improvement (CQI) that is necessary for operationalizing EPAs.

Didactic faculty

Didactic faculty are those who teach courses, conduct simulation exercises, spend time teaching and assessing students, and instruct students in other nonclinical settings. In the majority of U.S. medical schools, these faculty already contribute to formal assessment of student professional behaviors, which informs the determinations of trustworthiness underlying all EPAs (e.g., attendance, preparation, team-based interactions, responsiveness to feedback, conflict management).12 Therefore, they will need to understand the concept of entrustment and the importance of early assessment of truthfulness, discernment, and conscientiousness. Faculty development for this cohort should address barriers to the reporting of concerns and issues regarding those dimensions of entrustment. Importantly, these faculty may be the first with the opportunity to teach students to productively receive feedback and should be trained to do so. Also, they may incorporate coactivity scales into their instruction to help students begin to operationalize this approach to assessment. They also can role model informed self-assessment and reflective practice through co-participation with students in curricular CQI processes and reflection-in-action during their own teaching.

Some will need knowledge of the specific EPAs to which their curricular content contributes, and should participate in the curricular integration required to maximize student learning of those EPAs.13,14

Residents and other postgraduate trainees

Because residents make entrustment decisions earlier in their growth as supervisors, they most value conscientiousness (e.g., task completion and accuracy) and check the details of trainee work more frequently. This positions them to offer important observations on truthfulness and conscientiousness.15 Therefore, they will need instruction in the value of both entrustment and their potential contributions to entrustment decisions. They should be trained to identify students with behaviors requiring immediate corrective responses, to use supervisory language to frame feedback, and to assess early developing behaviors in at least some EPAs. The latter will require training in WBA and direct observation, training that may be facilitated through the intuitiveness of the EPA framework. Residents should be practicing informed self-assessment and reflective practice and, with a small amount of development, should explicitly model this for students.

Short-term clinical supervisors

This group includes traditional clinical supervisors who do not have continuous or longitudinal contact with students, and who lack the time and support to engage in the detailed faculty development techniques (e.g., frame-of-reference training)16 required to generate most quality assessment data.2 Examples include rotating inpatient ward attendings, faculty overseeing selectives, and other health care professionals who might be called on to provide assessment data based on brief interactions. Although they have traditionally constituted a substantial proportion of assessors of clinical performance in UME, their assessments lack validity and reliability.17

For these faculty, just-in-time assessments using only supervisory scales,18 which ask for a straightforward rating of how much they had to help the student, may permit valuable contributions where using behaviorally anchored scales have disappointed.

Data from the use of supervisory scales in graduate medical education are now repeatedly demonstrating moderate to high reliability with minimal to no faculty development, especially when the data are adjusted for the relative complexity of the task being observed.19 The advantage of supervisory scales stems from the fact that they are aligned to a set of decisions that clinical supervisors have already made—namely, how much the supervisor had to intervene. Therefore, their use does not impose the cognitive burden of translating observations into another framework of task assessment. Research also suggests that the use of supervisory scales may facilitate, again without additional development, the generation of more valuable feedback to learners, both face-to-face and in written evaluations.20

Longitudinal clinical supervisors and clinical course directors

Longitudinal clinical supervisors include faculty teaching in longitudinal integrated clerkships, coaches who provide learning and assessment opportunities over time through patient encounters, and faculty available for more intensive faculty development efforts. These educators must fully understand entrustment, the specific EPAs, and their role in an EPA-based system. They must have the motivation and resources to significantly engage in and identify with the educational mission and therefore participate in substantial development and a strong community of practice. These faculty should undergo frame-of-reference training or other detailed training techniques for specific EPAs and should guide students with targeted coaching and feedback along the developmental trajectory toward entrustment. To support learner improvement and to provide critical qualitative information for portfolios and summative entrustment decision making, these faculty should be skilled at crafting meaningful narrative comments.21 They will need to explicitly role model informed self-assessment and reflective processes.

Their longer-term relationships with students, including their more consistent availability to take time to carefully observe learner performance, position these faculty to offer highly effective feedback and judiciously engage learners in self-assessment.22 These relationships also position them to build the trust that facilitates effective feedback behaviors from all learners, and fosters self-directed learning.10,23 These relationships also position these supervisors to observe learners under the variety of conditions required to determine trustworthiness.7 Finally, they are the most logical group to have responsibility for remediating underperforming learners and therefore will require a substantial skill set in designing, implementing, and assessing individualized learning activities.

For this group to be effectively empowered and to fairly engage in monitoring and supporting learner progress, they will need training in the responsible communication of learner assessment information. Processes for sharing must focus on minimizing the long-debated pitfalls of sharing, or “forward feeding”—namely, biasing future assessors against or for students, sowing learner distrust of the process, and violating learner privacy.24 Instead, communication processes should maximize the benefits of feeding forward—namely, individualized coaching and mentoring, early identification and remediation of struggling learners, and fulfillment of the social contract to graduate competent students.

At least some of these faculty will require training to develop both the skills and the ability to teach in areas that their own training lacked, such as quality improvement and evidence-based medicine. At least some will also need the leadership and management skills to implement systems that support both quality patient care and robust clinical education.

Clinical course directors will need the full skill set of longitudinal clinical supervisors because they may serve in this role or oversee the work and some of the development of these faculty. Historically, clinical course directors (or clerkship directors) develop and oversee the clinical education of students. This includes a range of activities such as curriculum development, student assessment, and program evaluation, to name a few.25 An EPA-based system will require a greater degree of collaboration within this group to coordinate and integrate curricula and to identify the best possible permutations of assessment data to support entrustment decisions. Program evaluation will also be strengthened with greater input from them. Even without directly working with students, they access and synthesize multiple sources of assessment to support and monitor students in real time. Given their proximity to both students and faculty, they will need to understand and communicate the value of entrustment throughout the process of organizational transformation.

Portfolio coaches

Although portfolio systems have not been deemed absolutely necessary for CBE or EPA implementation, they are increasingly used to support CBE in UME.26 The goals for portfolio use in the EPA paradigm will be twofold: to support students’ learning trajectories and to assess student progress. Therefore, they should contain diverse components from multiple assessors as well as reflective components. Strong coaching in a safe environment is critical to student engagement in their use, and the absence of coaching leaves portfolios as “nothing short of bureaucratic hurdles in our CBE programs.”27

Kopechek and colleagues28 define a successful portfolio coach as any faculty trained to create a trusting and supportive relationship, inspire academic and professional growth in the student, and provide advice. Coaches should also undergo development to model self-directed learning skills and to engage in best practices of forward feeding.28,29 Because the trusting relationship is critical to the portfolio coaching process, choosing faculty who are attentive to this goal, as well as providing ongoing development opportunities emphasizing these skills, will be important.28

Working within the EPA framework, portfolio coaches will assess students’ abilities in informed self-assessment and reflective practice, and will themselves serve as role models for these activities. Along with some understanding of psychometric principles and data analysis, their familiarity must include the process of direct observation, the context and generators of assessment data to maximize meaning, and the identification of learning and growth opportunities for students across the spectrum of achievement.

Especially as they learn their new work, portfolio coaches will benefit from a robust community of practice based on a shared understanding of their roles in an EPA-based system. They will offer valuable expertise to the larger community of practice in the communication of learner assessment information and evolving perspectives on the adaptation of students and faculty assessors to EPA implementation. Their experience and observations, including those regarding the value and limitations of remediation and enrichment opportunities, may inform curricular change and potentially add significant and novel insights to the medical education literature.

Entrustment committee members

Lockyer and colleagues30 emphasize that CBE requires group processes to synthesize data from multiple assessors and methods into decisions regarding development toward competency and summative determinations of competency. Using an EPA framework, such groups, given the title of entrustment committees, would be tasked with decisions regarding student progress toward entrustment.

We expect that the division of labor between portfolio coaches, longitudinal clinical supervisors, and committees will undoubtedly vary between institutions. Entrustment committees may direct their efforts only at underperforming students, or actively engage in monitoring and supporting the progress of a larger proportion of students. In either case, members of these committees will need grounding in specific EPAs and entrustment and will need a robust working knowledge of the entire assessment system, including the available psychometrics of its tools and all relevant contextual information.

Evidence from graduate medical education clinical competency committees, which are similar in their use of multisource data to produce highly consequential on trainee progress, supports another development need for committee members: training for effective group decision making.31 This training should include a strong grounding in the shared understanding of both the goals of the group and the trajectory of trainee performance. Development should emphasize the value of sharing the full range of available and relevant information and the importance of resisting the tendency to focus on memory and personal experience with trainees. Finally, entrustment committees need leaders trained in effective facilitation, specifically with techniques and structures that solicit input from all members and foster elaboration and exchange to maximize meaning of that input.

Faculty and deans with oversight of professional behavior

These individuals typically have primary responsibility for guiding students in career planning as well as through academic and personal challenges. In these contexts, a strong understanding of the EPA framework, especially trustworthiness, is important in helping students who are struggling to progress toward entrustment. Determining why a student is struggling and how to support that student is especially important in a competency-based EPA framework. Students may need academic enrichment, counseling to overcome personal difficulties, learning strategy remediation, or simply patience if they are on a slower learning curve.

These faculty will likely need to work with leaders in the formal curriculum to design and implement systems of fair and purposeful communication of information about student professional behavior and personal difficulties. They will need to continue and perhaps even further develop programs addressing lapses in professional behaviors that support entrustment. They may work closely with curriculum deans to align advising with EPA coaching systems, negotiate the Liaison Committee on Medical Education’s separation of advising and evaluative roles, and serve as the liaison to program directors for postresidency match communications, in which case they should have a thorough grounding in specific EPAs as well. For these functions, they should have skills in leadership and management, organizational change, and teaching improvement. These faculty also have a unique opportunity to extend the professionalism literature more concertedly in the direction of trustworthiness and may benefit from development in research-building capacity to do this.

Curriculum planners and resource managers

The adoption of CBE represents a major transformation of professional, institutional, and organizational culture.32 Skilled leadership is critical to the success of this transformation.33 Drawing on a deep and system-wide understanding of curriculum and assessment, curriculum planners, acting as leaders of the program, will need to transform a culture of achievement and assessment for regulation into a culture of assessment for lifelong learning that drives safe patient care. The cornerstone of their success will rest on their ability to skillfully represent the value of EPAs in the broader educational mission to all stakeholders, a task that should be facilitated by the alignment of EPAs with health care practices and their connection to safe patient care. They will need to carefully identify barriers at their own institutions and develop strategies to address them.

Most important, they will need to judiciously develop, align, and use their most significant resource: the faculty. Leaders will need to appropriately select faculty for the most critical roles in CBE (longitudinal clinical supervisors, portfolio coaches, and entrustment committee members), identify those who can serve as part of early guiding coalitions, ensure the access to and use of the development and resources to fulfill roles and responsibilities, and pace change to optimize faculty engagement and minimize exhaustion.34 Their choices and allocation of resources should clearly support the development of communities of practice.35 Leaders must ensure the existence of strong evaluation processes that inform further definition of the system and provide for CQI. They should also engage in informed self-assessment of their own professional activities and have strong skills in the intentional and orderly use of bidirectional feedback.

As resource managers, leaders must also identify and allocate resources for the development and maintenance of elements critical to the EPA framework. These include first supporting faculty time for development, teaching, and coaching. They can then build program evaluation processes for CQI of new curricula, and develop information technology systems necessary to support a system that facilitates and manages multisource feedback.

Faculty developers

As these demands on faculty development in CBE and EPA implementation are far-reaching, curriculum planners and resource managers will need to carefully consider the allocation of faculty development work and the role of designated faculty developers.

Jolly36 offers recommendations for faculty developers responsible for programs supporting organizational change. These include engaging with leadership before development takes place to determine the scope and desired outcomes; tailoring and promoting efforts to make clear the link between faculty development and effecting organizational change; addressing factors that foster or impede change; and ensuring that the system will support the application and maintenance of new skills. Faculty developers will need the leadership skill of building communities of practice to achieve successful systems.35 These recommendations suggest the possibility of expanding the traditional faculty developers’ role from facilitating skills acquisition to explicitly leading organizational change, requiring therefore a strong skill set in leadership.

Leaders and resource managers will also need to address the imperative of CBE to expand research capacity building.37 As a complex educational intervention, CBE raises a multitude of questions requiring a complex array of study designs. Institutions may address this need through recruiting dedicated educational researchers and/or expanding the capacity of faculty developers to foster these skills.

Concluding Remarks

We have contemplated a scope of programmatic and targeted effort extending well beyond the traditional focus of faculty development on teaching improvement by taking a systems-based view that carefully delineates the requisite roles and responsibilities. The effort described here includes students, organizational leadership, resource managers, and postgraduate trainees among those whose skill sets must be developed. It brings faculty developers themselves, as well as those with oversight of professional behaviors, into the faculty fold.

Although much of this development pertains to the implementation of any CBE system, our work supports the possibility that the EPA framework offers a way forward and through some of the previously described challenges of faculty development for CBE implementation. We have noted that the value of EPAs for faculty development lies in their alignment with the daily work of health care and their direct relationship to safe patient care. Starting at the top, the transition to an EPA-based system must begin with leaders representing the value of EPAs to stakeholders, and stakeholders adopting them. A highly intuitive framework should facilitate both of these outcomes. If evidence from graduate medical education on supervisory scales holds true in the UME environment, EPAs, with their focus on entrustability, offer medical educators a way to capture valid assessment data from the many valuable clinical supervisors who typically do not participate in robust training. Finally, EPAs bring to the forefront the construct of trustworthiness for safe patient care based in truthfulness, conscientiousness, and discernment. This construct captures the essence while significantly circumscribing the extensive lists of professional attributes that define “professionalism.” In this way, trustworthiness becomes an intra- and cross-institutional shared mental model that puts a clear value, within the formal curriculum, on valid contributions of didactic faculty, residents, faculty, and deans overseeing professional behaviors, as well as on the contributions of students themselves.

With the participating stakeholders and their needs now identified, future work should aim to outline the most effective strategies for helping stakeholders fully engage in the EPA mission.

Acknowledgments:

The authors wish to thank the Association of American Medical Colleges and the Core Entrustable Professional Activities for Entering Residency Faculty Development Concept Group members: Linda Tewksbury, MD, New York University School of Medicine; Greg Trimble, MD, Virginia Commonwealth University School of Medicine, Inova Campus; Dana Dunne, Yale University School of Medicine; and Michael Ryan, MD, Virginia Commonwealth University. They also wish to thank Jan Bull, MA, Association of American Medical Colleges, and Alison Whelan, MD, Association of American Medical Colleges.

References

1. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: From Flexner to competencies. Acad Med. 2002;77:361–367.
2. Holmboe ES, Ward DS, Reznick RK, et al. Faculty development in assessment: The missing link in competency-based medical education. Acad Med. 2011;86:460–467.
3. Dath D, Iobst W. The importance of faculty development in the transition to competency-based medical education. Med Teach. 2010;32:683–686.
4. Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency-based medical education: Are we addressing the concerns and challenges? Med Educ. 2015;49:1086–1102.
5. Englander R, Aschenbrener CA, Call SA, et al. Core entrustable professional activities for entering residency. MedEdPORTAL. https://icollaborative.aamc.org/resource/887. Published May 28, 2014. Accessed April 24, 2018.
6. ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach. 2015;37:983–1002.
7. Favreau MA, Tewksbury L, Lupi C, Cutrer WB, Jokela JA, Yarris LM; AAMC Core Entrustable Professional Activities for Entering Residency Faculty Development Concept Group. Constructing a shared mental model for faculty development for the core entrustable professional activities for entering residency. Acad Med. 2017;92:759–764.
8. Steinert Y. Steinert Y. Faculty development: Core concepts and principles. In: Faculty Development in the Health Professions: A Focus on Research and Practice. 2014.New York, NY: Springer;
9. Kennedy TJ, Regehr G, Baker GR, Lingard L. Point-of-care assessment of medical trainee competence for independent clinical work. Acad Med. 2008;83(10 suppl):S89–S92.
10. Bowen L, Marshall M, Murdoch-Eaton D. Medical student perceptions of feedback and feedback behaviors within the context of the “educational alliance.” Acad Med. 2017;92:1303–1312.
11. Patel R, Tarrant C, Bonas S, Yates J, Sandars J. The struggling student: A thematic analysis from the self-regulated learning perspective. Med Educ. 2015;49:417–426.
12. Association of American Medical Colleges. Curriculum inventory reports: Assessment of professional behavior: Student evaluation of faculty/supervisors. https://www.aamc.org/initiatives/cir/429054/30.html. Revised 2017. Accessed April 24, 2018.
13. Schmidt H. Integrating the teaching of basic sciences, clinical sciences, and biopsychosocial issues. Acad Med. 1998;73(9 suppl):S24–S31.
14. Irby DM, Cooke M, O’Brien BC. Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med. 2010;85:220–227.
15. Sheu L, Kogan JR, Hauer KE. How supervisor experience influences trust, supervision, and trainee learning: A qualitative study. Acad Med. 2017;92:1320–1327.
16. Goodstone MS, Lopez FE. The frame of reference approach as a solution to an assessment center dilemma. Consult Psych J Pract Res. 2001;53:96–107.
17. Fromme HB, Karani R, Downing SM. Direct observation in medical education: A review of the literature and evidence for validity. Mt Sinai J Med. 2009;76:365–371.
18. Chen HC, van den Broek WE, ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015;90:431–436.
19. Weller JM, Castanelli DJ, Chen Y, Jolly B. Making robust assessments of specialist trainees’ workplace performance. Br J Anaesth. 2017;118:207–214.
20. Castanelli DJ, Jowsey T, Chen Y, Weller JM. Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training. Can J Anaesth. 2016;63:1345–1356.
21. Hodges B. Assessment in the post-psychometric era: Learning to love the subjective and collective. Med Teach. 2013;35:564–568.
22. Snow SC, Gong J, Adams JE. Faculty experience and engagement in a longitudinal integrated clerkship. Med Teach. 2017;39:527–534.
23. Goldman S. The educational kanban: Promoting effective self-directed adult learning in medical education. Acad Med. 2009;84:927–934.
24. Cleary LM. Pangaro LN, McGaghie WC. Chapter 4: Sharing student performance information across courses and clerkships. In: Handbook on Medical Student Evaluation and Assessment. 2015.North Syracuse, NY: Gegensatz Press;
25. Pangaro L, Bachicha J, Brodkey A, et al.; Alliance for Clinical Education. Expectations of and for clerkship directors: A collaborative statement from the Alliance for Clinical Education. Teach Learn Med. 2003;15:217–222.
26. Chertoff J, Wright A, Novak M, et al. Status of portfolios in undergraduate medical education in the LCME accredited US medical school. Med Teach. 2016;38:886–896.
27. Driessen E. Do portfolios have a future? Adv Health Sci Educ Theory Pract. 2017;22:221–228.
28. Kopechek J, Khandelwal S, Grieco CA, Post DM, Davis JA, Ledford C. Portfolio-associated faculty: A qualitative analysis of successful behaviors from the perspective of the student. J Biomed Educ. 2016. doi: http://dx.doi.org/10.1155/2016/4602704.
29. van Schaik S, Plant J, O’Sullivan P. Promoting self-directed learning through portfolios in undergraduate medical education: The mentors’ perspective. Med Teach. 2013;35:139–144.
30. Lockyer J, Carraccio C, Chan MK, et al.; ICBME Collaborators. Core principles of assessment in competency-based medical education. Med Teach. 2017;39:609–616.
31. Hauer KE, ten Cate O, Boscardin CK, et al. Ensuring resident competence: A narrative review of the literature on group decision making to inform the work of clinical competency committees. J Grad Med Educ. 2016;8:156–164.
32. Ferguson PC, Caverzagie KJ, Nousiainen MT, Snell L; ICBME Collaborators. Changing the culture of medical training: An important step toward the implementation of competency-based medical education. Med Teach. 2017;39:599–602.
33. Nousiainen MT, Caverzagie KJ, Ferguson PC, Frank JR; ICBME Collaborators. Implementing competency-based medical education: What changes in curricular structure and processes are needed? Med Teach. 2017;39:594–598.
34. Kotter JP. Leading Change. 2012.2nd ed. Cambridge, MA: Harvard Business Review Press;
35. Cruess RL, Cruess SR, Steinert Y. Medicine as a community of practice: Implications for medical education. Acad Med. 2018;93:185–191.
36. Jolly B. Steinert Y. Faculty development for organizational change. In: Faculty Development in the Health Professions: A Focus on Research and Practice. 2014.New York, NY: Springer;
37. Gruppen L, Frank JR, Lockyer J, et al.; ICBME Collaborators. Toward a research agenda for competency-based medical education. Med Teach. 2017;39:623–630.
Copyright © 2018 by the Association of American Medical Colleges