Secondary Logo

Journal Logo

Graduate Medical Education: Its Role in Achieving a True Medical Education Continuum

Aschenbrener, Carol A. MD; Ast, Cori MHSA; Kirch, Darrell G. MD

doi: 10.1097/ACM.0000000000000829
Perspectives
Free
SDC

Nearly half a century ago, Lowell T. Coggeshall recommended, through what has come to be known as the Coggeshall Report, that physician education—medical school (or undergraduate medical education [UME]), residency training (or graduate medical education [GME]), and continuing medical education (CME)—be “planned and provided as a continuum.” While the dream of a true continuum remains unfulfilled, recent innovations focused on defining and assessing meaningful outcomes at last offer the anchor for the creation of a seamless, flexible, and ongoing pathway for the preparation of physicians. Recent innovations, including a widely accepted competency framework and entrustable professional activities (EPAs), provide key tools for creating a continuum. The competency framework is being leveraged in UME, GME, and CME and is serving as the foundation for the continuum. Learners and those who assess them are increasingly relying on observable behaviors (e.g., EPAs) to determine progress. The GME community in the United States and Canada has played—and continues to play—a leading role in the creation of these tools and a true medical education continuum. Despite some systemic challenges to implementation (e.g., premedical learner formation, time-in-step requirements), the GME community is already operationalizing these tools as a basis for other innovations that are improving transitions across the continuum (e.g., competency-based progression of residents). The medical education community’s greatest responsibility in the years ahead will be to build on these efforts in GME—joining together to learn from one another and develop a continuum that serves the public and the profession.

C.A. Aschenbrener is founding chief medical education officer, Association of American Medical Colleges, Washington, DC.

C. Ast is director, executive activities, Office of the President, Association of American Medical Colleges, Washington, DC.

D.G. Kirch is president and chief executive officer, Association of American Medical Colleges, Washington, DC.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Correspondence should be addressed to Cori Ast, Association of American Medical Colleges, 655 K St., NW, Suite 100, Washington, DC 20001; telephone: (202) 448-6656; e-mail: cast@aamc.org.

When examining the future of medical education, Stephen Covey’s1 “Begin with the end in mind” seems salient advice. The ultimate outcome for medical education is physicians who consistently provide safe, effective, patient-centered health care.2 Over the last half-century, scholars, clinicians, insurers, and the public have relentlessly called for improvements to medical education to ensure this desired outcome. Among them was Lowell T. Coggeshall, 3 who authored Planning for Medical Progress Through Education (now widely known as the Coggeshall Report), which decried the “sharp separation” of the three phases of physician education: medical school (or undergraduate medical education [UME]), residency training (or graduate medical education [GME]), and continuing medical education (CME). Coggeshall recommended that medical education should be “planned and provided as a continuum.”3 Half a century later, realizing the vision of a continuum of learning for physicians may be the critical step to ensuring that medical education in the United States consistently prepares all physicians to enhance the health of individuals and populations.

While the dream of a true continuum remains unfulfilled, the past decade of innovations in GME has moved medical education closer to that goal by defining meaningful outcomes as physician competencies and identifying the developmental pathways to achieve those outcomes. In this Perspective, we propose that this competency framework at last offers an anchor for the creation of a seamless, flexible, and ongoing continuum for the preparation of physicians. We highlight signs that the continuum Coggeshall envisioned is emerging, and we encourage the medical education community to continue to pursue an outcome-anchored continuum—even in the face of significant challenges.

Back to Top | Article Outline

A Competency Framework as an Anchor for the Continuum

The key questions facing the medical education community still remain: How can we bring all physicians to at least the level of proficiency in performing all the integrated professional activities of their role, and how do we ensure that they maintain proficiency as populations, societal needs, technology, and contexts change? Multiple authors and organizations (including the Association of American Medical Colleges [AAMC]) have written reports and commentaries, highlighting challenges in medical education.4–7 Most recently, the Institute of Medicine has published a report on the financing of GME which criticizes GME as “lack(ing) the oversight and infrastructure to track outcomes, reward performance, and respond nimbly to challenges.”8 Changes to GME in isolation cannot shape physicians who will provide competent, compassionate, effective, and accessible care throughout their entire careers. Ensuring that desired result requires a continuum approach like the one envisioned by Coggeshall. We believe that two recent innovations applied mainly in GME—a widely accepted competency framework9 and emerging entrustable professional activities (EPAs)10,11—provide key tools for creating a continuum (see Table 1 below and Figure 2 in Core EntrustableProfessional Activities for Entering Residency11).

Table 1

Table 1

General and specialty-specific competency frameworks provide the requisite “end” to anchor the entire continuum. Much of the foundational work on defining physician competencies has come from GME-focused efforts in Canada and the United States, particularly the CanMEDS project of the Royal College of Physicians and Surgeons of Canada12 and the Outcomes Project of the Accreditation Council for Graduate Medical Education (ACGME).13 Launched in 1999, the ACGME Outcomes Project, in partnership with the American Board of Medical Specialties (ABMS), defined 36 specific competencies for all physicians within six domains (patient care; medical knowledge; practice-based learning and improvement; interpersonal and communication skills; professionalism; systems-based practice).13 This competency framework outlined by the Outcomes Project has been central in the ACGME’s transition to an outcomes-based assessment of residency programs.

An indication of broad agreement about the relevance and utility of this competency framework across the continuum of learning for physicians is the recent endorsement of the framework as foundational for the profession of medicine14 by the 11 organizations that constitute the regulatory community of medicine (see List 1). Highlighting the framework as an anchor throughout the lifetime of a physician, the consensus statement asserts:

Mastery of the competencies is developmental, beginning before entry into medical school and continuing throughout a career in clinical practice.… An individual’s performance in most competencies may improve throughout a career of clinical practice. Performance in at least some competencies may decline over time. The pace of growth will vary among individuals, and any individual likely will progress at different rates in different competencies.14

Back to Top | Article Outline

List 1 The 11 Member Organizations of the Coalition for Physician Accountability, Endorsing “a Framework for Professional Competence”14 Cited Here...

  • Accreditation Council for Continuing Medical Education (ACCME)
  • Accreditation Council for Graduate Medical Education (ACGME)
  • American Board of Medical Specialties (ABMS)
  • American Medical Association (AMA)
  • American Osteopathic Association (AOA)
  • Association of American Medical Colleges (AAMC)
  • Educational Commission for Foreign Medical Graduates (ECFMG)
  • Federation of State Medical Boards (FSMB)
  • Liaison Committee on Medical Education (LCME)
  • National Board of Medical Examiners (NBME)
  • National Board of Osteopathic Medical Examiners (NBOME)

In addition, competency frameworks are being used to align the initial certification process and maintenance of certification (MOC) processes of ABMS, as well as the maintenance of licensure process outlined by the Federation of State Medical Boards.15,16

Back to Top | Article Outline

Linking competencies to professional practice: EPAs

In the implementation of the ACGME/ABMS competency framework, educators have been stymied in their efforts to assess competencies that are not within the domains of medical knowledge and interpersonal communication. This obstacle is not surprising because competencies require assessment of abilities to perform professional tasks, not the individual tasks themselves. Further, in practice, most daily activities of physicians require integration of multiple competencies. The concept of EPAs (introduced in the Netherlands) offers a means of linking competencies to meaningful professional work.10,17,18 EPAs also enable clinical faculty to make determinations (“entrustment decisions”) about what level of supervision is appropriate for which residents performing specific tasks.10,18 A number of specialties, including family medicine, pediatrics, and cardiology,19–21 have outlined EPAs that, in the aggregate, define their specialty.

Back to Top | Article Outline

Building a continuum

Since the necessary components—a competency framework and observable, measurable, integrated behaviors that define a specialty (i.e., EPAs)—are now in place, the challenge is to use them to better link the phases of the continuum for the formation of physicians, beginning prior to medical school and continuing throughout professional practice. The goal of such a continuum is to achieve and maintain standardized outcomes while allowing flexible learning pathways to accommodate the varied skills and backgrounds of all learners, at all stages, in medical education.

Building a continuum of learning requires working backward from that goal. Some steps include identifying the requisite knowledge, skills, and attitudes (KSAs) for the competencies and EPAs; clarifying both the distinctions between and overlap of KSAs across competencies; designing learning experiences that facilitate the acquisition of the requisite KSAs; incorporating the appropriate balance of formative and summative feedback; designing assessments that will support entrustment decisions; and creating a means of providing information-rich hand-offs about individual learners as they move along the continuum.

Well conceived, an EPA can be mapped to the most relevant domains of competence, the most critical competencies within those domains, and the KSAs undergirding the critical competencies (see Figure 2 in Core Entrustable Professional Activities for Entering Residency11). Such maps, in the aggregate, can guide the design of linked curricula aimed at developing physicians proficient in the competencies.

A true continuum will incorporate pathways of sequentially relevant experiences without locking learners into a rigid, specific frame of block courses or rotations. Instead, a true competency-based continuum will allow learners to progress faster in some competencies, slower in others—all according to their individual abilities and experiences. The level of responsibility granted to each learner and the degree of supervision he or she requires will be linked, at least in part, to observation-based professional judgments about his or her demonstrated performance in specific competencies or EPAs, rather than time or process.

A continuum assumes that practicing physicians must continue to learn as context, demographics, technology, and scope of practice change, and it should stimulate the medical education community to collaborate in developing competency-based, just-in-time learning resources. A true continuum will incorporate periodic reassessment of the level of performance of physicians or physicians-in-training, especially of competencies and EPAs most susceptible to deterioration without regular practice.

Back to Top | Article Outline

Signs of an Emerging Continuum

A continuum of learning requires a coherent trajectory that aligns the components to ultimate outcomes; a guide for ensuring effective, information-rich hand-offs about learner progress at key transition points (e.g., the beginning of clinical clerkships, entry into residency, and transition to practice); and a framework to foster lifelong learning to maintain competence.

Back to Top | Article Outline

Implementing competencies as the first step in creating the continuum: Defining the pathways to competence

Since six standardized domains of competence (and 36 competencies) have already been defined, the creation of a continuum next requires explication of the developmental sequence for acquiring the competencies. The ACGME Milestones Project has now identified developmental pathways for increasing levels of performance (i.e., milestones) for each competency for all medical specialties.22 Although much remains to be done to validate these milestones and create reliable assessments, their implementation in GME signals a shift to standardizing the observable and measurable behaviors expected of all physicians. Once the behavioral anchors in the milestones are converted to narrative descriptors and vignettes, an approach pioneered by the American Board of Pediatrics, physician learners at all levels and those who assess them will have a shared basis for linking performance with level of responsibility and supervision, thereby enabling robust implementation of competency-based education.23

In their cogent Commentary on the lessons learned and challenges of competency-based medical education, Carraccio and Englander24 identify the need for a standardized language as a key means to creating a shared vision of what constitutes competence. Many different competency lists are available for medicine, for various specialties and subspecialties, for different nations, and for diverse health professions. A 2013 comparative analysis of all available competency lists has yielded a taxonomy of eight domains of competence for health professionals, adding “interprofessional collaboration” and “personal and professional development” to the six domains outlined by the ACGME and ABMS—along with a list of 58 competencies for physicians within those eight domains.9 Many medical schools are mapping their curricula and linking assessments to this “Reference List of General Physician Competencies.” While confusion about terminology persists, consensus on definitions is emerging with considerable guidance from the International Collaborators on Competency-Based Medical Education.13,25

Back to Top | Article Outline

Transitions within the continuum: Assuring readiness for progress

Each transition should be a process over time, not simply an arrival at a specific date on the calendar. Many medical schools have implemented transition activities, such as required levels of performance on simulations before engaging with patients, transition-to-clerkship and transition-to-residency “boot camps,” gateway assessments, structured acting internships, and specialty-specific standardized “transition” electives.26,27 Many GME programs also mandate skills “boot camps” at the beginning of residency to confirm that new residents can safely transition to supervised care of patients.28,29 With the notable exception of the standardized longitudinal curriculum in surgery developed by the American College of Surgeons,30 most transition activities are institution- or program-specific.

While more remains to be done to ensure information-rich handoffs of learners across the continuum, some progress is already occurring in facilitating better transitions. To address perceived gaps in readiness for GME, the AAMC convened an expert panel to define foundational clinical activities that residents are commonly expected to perform without direct supervision in the first weeks of residency. These 13 Core Entrustable Professional Activities for Entering Residency (CEPAER) represent the first effort to define specific skills that all medical students should be entrusted to do before receiving a medical degree.11 Although much remains to be learned about how to ensure appropriate learning experiences and make entrustment decisions, medical schools are rapidly embracing the use of EPAs. For example, more than half of U.S. MD-granting medical schools recently applied to participate in an AAMC program to pilot implementation of the 13 EPAs.

Back to Top | Article Outline

Evidence of extending the continuum through CME: Fostering lifelong learning

Recent changes in CME are also facilitating a continuum approach. Some of these changes include integration of new learning methodologies; migration from lectures to team learning, simulation, and workplace applications; and more offerings designed for interprofessional audiences. In addition, the need for new or expanded content (e.g., team care, patient safety, quality improvement, population health) is emerging so rapidly that CME programs may include learners from different stages (as well as different professions, institutions, and specialties). Such programs illustrate a shift from role-based to theme- or skill-based offerings. With the implementation of practice performance assessment via Part 4 of MOC in most ABMS-certified medical specialties, the shift to CME that is both aligned with specialty-specific competencies and linked to measured improvement in performance will only increase.15 Indeed, “CME” is being replaced by the concept of “continued professional development,” in which the aim of learning is to increase each physician’s level of performance in specific competencies; thus, continued professional development connects to the previously cited efforts in UME and GME that support medical education as a continuum of growth for each individual.

Back to Top | Article Outline

Overcoming Challenges to Implementation of a Continuum

Across many sectors, organizations with decades of tradition and success tend to respond to suggestions for change with a list of obstacles. Although understandable, such a response is not acceptable for a profession of learned individuals, who, collectively, bear responsibility for human life and still enjoy the privilege of self-regulation. The challenges to implementing a true continuum (e.g., the need for faculty development, the time and effort required, the potential for lost revenue) have been well articulated,24 and some promising approaches to addressing those challenges are emerging.

Preparing faculty for new roles as designers and facilitators of learning experiences, as providers of meaningful behavioral feedback, and as contributors to entrustment decisions is a major endeavor.31,32 Pursuing innovation in the customary way—each institution creating its own learning and assessment resources—will indeed be too time consuming. Collaboration is imperative. Given their potential impact on patient care, transinstitutional collaborations might seek private or even public funding for a transition period to expedite the development and testing of novel educational resources.

To address concerns about the loss of tuition revenue that may result from a time-variable program, medical schools might shift to charging for the degree over several years, rather than by semesters. The transition to variable time for residency must be made without any adverse effects on patient care and without creating an undue financial burden to the sponsoring entities.33 One approach to navigating this tension might entail requiring a specific period of service of residents while allowing them—once they have met the criteria for entrustment for unsupervised practice of specific EPAs—to progress to advanced levels in some competencies and allowing such entrusted residents to bill for these specified services. Another idea may be to consider alternatives to the traditional rotational approach34 used in many GME programs.

Back to Top | Article Outline

Premedical education as the gateway of the continuum

As medical schools focus more attention on specific competencies in the admissions process, the role of premedical education (PME) is coming under increasing scrutiny. At present, PME is not well aligned with the changing emphases in medical education, but some movement toward better alignment with the competency framework is under way. For example, there have been efforts to define both academic and personal competencies (e.g., ethical responsibility to self and others, a service orientation, ability to work in a team) for premedical students.35–37 The next step will be to identify approaches that not only encourage students to pursue, but also enable the assessment of, these foundational competencies. For undergraduate institutions, this step may entail offering more interdisciplinary courses and/or experiences that foster personal growth.38

Notably, the use of holistic review represents some progress in emphasizing and evaluating desired qualities in applicants beyond academic achievement,39 though the practice is not yet mainstream. The introduction of MCAT 2015 also emphasizes the importance of competencies such as problem-solving skills.40 Although these two initiatives indicate progress toward better preparation for transition to UME, more innovation is needed.

Accountability for physician formation begins as early as college and is shared by aspirants, their teachers and advisors, and the members of medical school admission committees who must intentionally seek entrants who demonstrate the requisite competencies to become caring, competent physicians.41

Back to Top | Article Outline

Time-variable advancement through the mastery of competencies

Medical schools, residencies, and continuing education programs are experimenting both with approaches to the valid assessment of competencies and with flexible learning pathways that would allow individuals to progress at different rates. These efforts address what can be significant systemic barriers (e.g., tuition structures and time-in-step certification requirements). These system challenges, however, pale in comparison to the complexity of designing a truly time-variable system grounded in standardized outcomes; thus, the medical education community can learn a great deal from the rare pilots.

To test the feasibility of advancement according to competencies across UME and GME, in 2009 the AAMC began sponsoring a pilot project, called Education in Pediatrics Across the Continuum, for small groups of students who know early in medical school that they want to be pediatricians. With a multiyear commitment from medical school and pediatric residency programs, the four pilot institutions have designed and begun implementing pathways that allow time-variable progression linked to performance on the CEPAER, on the general competencies for physicians, and on the milestones established by the American Board of Pediatrics.42

Another example of innovation is the orthopedic surgery residency at the University of Toronto, which is pioneering a competency-based, time-variable program which began with the cohort that entered in 2009. Early results have been encouraging: The first 14 residents slightly outperformed their colleagues in the traditional, time-based program; two members of this first cohort finished the program in four years rather than five; and two who had knowledge deficiencies were identified early and were still able, albeit with additional time, to perform at the level required by the program.43 The program has since been expanded to include all entering residents.

Back to Top | Article Outline

Using a continuum approach to collaborate on quality and safety

One of the most urgent needs is to ensure that all physicians in practice demonstrate the ability to identify and address personal behaviors or system problems that may compromise patient safety or quality of care. The clinical learning environment for GME is increasingly recognized as one of the most powerful influences shaping physicians’ behavior.44 Habits learned in residency, for good or bad, tend to persist in practice.45,46 Additionally, physicians, physician leaders, and physician educators increasingly recognize that some core professional behaviors (e.g., respect for patient needs that supersedes self-interest) must be ensured across all programs in all specialties to mitigate unacceptable variations in quality of care.

The ACGME has identified patient safety and improving the quality of care as two of six focal points in their nonevaluative reviews of the clinical learning environment (CLER Pathways to Excellence). Collaborations are developing to address needs for learners at different levels, and these activities could be connected as a continuum, linked to the appropriate phase of competence development rather than to a predetermined year of education. Both a medical student and midcareer practitioner must understand the principles of quality improvement before being able to apply an iterative process to addressing gaps in performance. Currently, the Institute for Healthcare Improvement’s Open School focuses on modules or “foundations of improvement, safety, system design, and leadership” that are appropriate for students, residents, or practitioners.47 Additionally, the AAMC sponsors several programs for improving quality, including conferences and project-based quality improvement initiatives for training health care professionals and bringing interested faculty to mastery level.48,49 These efforts and others could be consciously aligned and linked to outcomes and milestones so as to define a coherent pathway. In turn, the pathway could be “installed” as a continuum of learning and assessment not linked specifically to any particular stage of education or training. Lessons from this collaborative effort could then inform other innovators who are collaborating to develop additional competency-based approaches.

Back to Top | Article Outline

Encouraging Continued Pursuit of an Outcome-Anchored Continuum

During a 1992 conference on GME sponsored by the Josiah Macy Jr. Foundation, keynote speaker Samuel O. Thier, former president of the Institute of Medicine, asserted that “GME cannot transform the health care system.”50 Most in academic medicine would likely agree with his assessment. We believe, however, that GME can be the pivot for transforming medical education from a siloed, relatively lock-step process into a continuous system focused on clearly defined outcomes that encourages flexible pathways for learning and lifelong improvement in performance.

The transition of GME to focus on standardized outcomes for learning and assessment is occurring through a process of continuous quality improvement: planning, experimentation, adjustment, and implementation. The formation of a continuum must occur likewise. The medical education community can make progress by identifying needs for improvement in physician performance and collaborating to implement appropriate learning and formative assessment at all stages from PME through continued learning in practice.

The need for change has long been identified, the anchors for effecting change have emerged, and a sense of readiness for change is springing from within the academic and learner communities. It is now up to us—the leaders and learners in medical education—to join efforts to learn from one another and build a continuum that serves the public and the profession. The transition to a coherent continuum of learning and assessment will be neither easy nor swift, but it is essential to meet the diverse needs of the public, students and trainees, and the institutions that provide health care. Creating a truly seamless, flexible, ongoing continuum of learning will be the medical education community’s greatest responsibility over the next decade and, given the talent and creativity of the community, the goal is within reach.

Acknowledgments: The authors wish to thank Robert Englander, MD, and David Davis, MD, for their invaluable assistance in readying this manuscript.

Back to Top | Article Outline

References

1. Covey SR. The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change. 1989 New York, NY Free Press
2. Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. 2001 Washington, DC National Academy Press
3. Coggeshall LT. Planning for Medical Progress Through Education: A Report Submitted to the Executive Council of the Association of American Medical Colleges. 1965 Evanston, Ill Association of American Medical Colleges
4. McGaghie WC, Miller GE, Sajid AW, Telder TV. Competency-based curriculum development in medical education: An introduction. 1978 Geneva, Switzerland World Health Organization http://whqlibdoc.who.int/php/WHO_PHP_68.pdf. Accessed May 29, 2015
5. Cooke M, Irby DM, O’Brien BCCarnegie Foundation for the Advancement of Teaching. Educating Physicians: A Call for Reform of Medical School and Residency. 2010 San Francisco, Calif Jossey-Bass
6. Association of American Medical Colleges. Physicians for the Twenty-First Century: Report of the Project Panel on the General Professional Education of the Physician and College Preparation for Medicine. 1984 Washington, DC Association of American Medical Colleges
7. Association of American Medical Colleges. Educating Medical Students: Assessing Change in Medical Education—The Road to Implementation. 1992 Washington, DC Association of American Medical Colleges
8. Institute of Medicine, Committee on the Governance and Financing of Graduate Medical Education. Graduate Medical Education That Meets the Nation’s Health Needs. 2014 Washington, DC National Academies Press
9. Englander R, Cameron T, Ballard AJ, Dodge J, Bull J, Aschenbrener CA.. Toward a common taxonomy of competency domains for the health professions and competencies for physicians. Acad Med. 2013;88:1088–1094
10. Ten Cate O.. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157–158
11. Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency: Curriculum Developers Guide. 2014 Washington, DC Association of American Medical Colleges www.aamc.org/cepaer. Accessed May 29, 2015
12. Frank JR. The CanMEDS 2005 Physician Competency Framework. 2005 Ottawa, Ontario, Canada Royal College of Physicians and Surgeons of Canada http://www.royalcollege.ca/portal/page/portal/rc/common/documents/canmeds/resources/publications/framework_full_e.pdf. Accessed May 29, 2015
13. Swing SR.. The ACGME outcome project: Retrospective and prospective. Med Teach. 2007;29:648–654
14. Coalition for Physician Accountability. Consensus statement on a framework for professional competence. Approved August 19, 2014. http://www.physicianaccountability.org/pdf/Coalition_Competencies_Consensus_Statement.pdf. Accessed May 29, 2015
15. American Board of Medical Specialties. 2015 http://www.abms.org/maintenance_of_certification/ABMS_MOC.aspx. Accessed May 29, 2015
16. Chaudhry HJ, Rhyne J, Cain FE, Young A, Crane M, Bush F.. Maintenance of licensure: Protecting the public, promoting quality health care. J Med Regul. 2010;96(2):13–20
17. Laan RF, Leunissen RR, van Herwaarden CL.. The 2009 framework for undergraduate medical education in the Netherlands. GMS Z Med Ausbild. 2010;27:Doc35
18. ten Cate O, Scheele F.. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–547
19. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J Jr. Entrustable professional activities in family medicine. J Grad Med Educ. 2013;5:112–118
20. Carraccio C, Englander R, Gilhooly JAmerican Board of Pediatrics. Entrustable professional activities. www.abp.org/entrustable-professional-activities-epas. Accessed May 29, 2015
21. American College of Cardiology. Competencies: Background. www.acc.org/education-and-meetings/products-and-resources/competencies/background. Accessed May 29, 2015
22. Sullivan G, Simpson D, Cooney T, Beresin E.. A milestone in the milestones movement: The JGME milestones supplement. J Grad Med Educ. 2013;5(1 suppl 1):1–4
23. Accreditation Council for Graduate Medical Education; American Board of Pediatrics. The Pediatrics Milestone Project. 2013 http://acgme.org/acgmeweb/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf. Accessed May 29, 2015
24. Carraccio CL, Englander R.. From Flexner to competencies: Reflections on a decade and the journey ahead. Acad Med. 2013;88:1067–1073
25. Frank JR, Snell LS, ten Cate OT, et al. Competency based medical education: Theory to practice. Med Teach. 2010;32:638–645
26. Laack TA, Newman JS, Goyal DG, Torsher LC.. A 1-week simulated internship course helps prepare medical students for transition to residency. Simul Healthc. 2010;5:127–132
27. Okusanya OT, Kornfield ZN, Reinke CE, et al. The effect and durability of a pregraduation boot camp on the confidence of senior medical student entering surgical residencies. J Surg Educ. 2012;69:536–543
28. Krajewski A, Filippa D, Staff I, Singh R, Kirton OC.. Implementation of an intern boot camp curriculum to address clinical competencies under the new Accreditation Council for Graduate Medical Education supervision requirements and duty hour restrictions. JAMA Surg. 2013;148:727–732
29. Cohen ER, Barsuk JH, Moazed F, et al. Making July safer: Simulation-based mastery learning during intern boot camp. Acad Med. 2013;88:233–239
30. American College of Surgeons. 2015 Chicago, Ill American College of Surgeons www.facs.org/education/program/resident-prep. Accessed May 29, 2015
31. Holmboe ES, Hawkins RE, Huot SJ.. Effects of training in direct observation of medical residents’ clinical competence: A randomized trial. Ann Intern Med. 2004;140:874–881
32. Hauer KE, Ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS.. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014;19:435–456
33. Josiah Macy Jr. Foundation. Ensuring an Effective Physician Workforce for the United States: Recommendations for Reforming Graduate Medical Education to Meet the Needs of the Public. 2011 New York, NY Josiah Macy Jr. Foundation http://macyfoundation.org/docs/macy_pubs/JMF_GME_Conference2_Monograph%282%29.pdf. Accessed May 29, 2015
34. Holmboe E, Ginsburg S, Bernabo E.. The rotational approach to medical education: Time to confront our assumptions? Med Teach. 2011;45:69–80
35. Association of American Medical Colleges–Howard Hughes Medical Institute Committee. 2009 Washington, DC Association of American Medical Colleges; https://www.aamc.org/download/271072/data/scientificfoundationsforfuturephysicians.pdf. Accessed May 29, 2015
36. Association of American Medical Colleges. 2011 Washington, DC Association of American Medical Colleges https://www.aamc.org/download/271020/data/behavioralandsocialsciencefoundationsforfuturephysicians.pdf. Accessed May 29, 2015
37. Koenig TW, Parrish SK, Terregino CA, Williams JP, Dunleavy DM, Volsch JM.. Core personal competencies important to entering students’ success in medical school: What are they and how could they be assessed early in the admission process? Acad Med. 2013;88:603–613
38. Dienstag JL.. Relevance and rigor in premedical education. N Engl J Med. 2008;359:221–224
39. Witzburg RA, Sondheimer HM.. Holistic review—Shaping the medical profession one applicant at a time. N Engl J Med. 2013;368:1565–1567
40. Kirch DG, Mitchell K, Ast C.. The new 2015 MCAT: Testing competencies. J Am Med Assoc. 2013;310:2243–2244
41. Kirch DG, Gusic ME, Ast C.. Undergraduate medical education and the foundation of physician professionalism. J Am Med Assoc. 2015;313:1797–1798
42. Powell DE, Carraccio C, Aschenbrener CA.. Pediatrics redesign project: A pilot implementing competency-based education across the continuum. Acad Med. 2011;86:e13
43. Ferguson PC, Kraemer W, Nousiainen M, et al. Three-year experience with an innovative, modular competency-based curriculum for orthopaedic training. J Bone Joint Surg Am. 2013;95:e166
44. Nasca TJ, Weiss KB, Bagian JP.. Improving clinical learning environments for tomorrow’s physicians. N Engl J Med. 2014;370:991–993
45. Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ.. Evaluating obstetrical residency programs using patient outcomes. J Am Med Assoc. 2009;302:1277–1283
46. Bowen JL, Salerno SM, Chamberlain JK, Eckstrom E, Chen HL, Brandenburg S.. Changing habits of practice. Transforming internal medicine residency education in ambulatory settings. J Gen Intern Med. 2005;20:1181–1187
47. Institute for Healthcare Improvement. 2015 http://www.ihi.org/education/ihiopenschool/Pages/default.aspx. Accessed May 29, 2015
48. Association of American Medical Colleges. Teaching for Quality: Integrating Quality Improvement and Patient Safety Across the Continuum of Medical Education. Report of an Expert Panel. 2013 Washington, DC Association of American Medical Colleges
49. Davis NL, Davis DA, Johnson NM, et al. Aligning academic continuing medical education with quality improvement: A model for the 21st century. Acad Med. 2013;88:1437–1441
50. Morris TQ, Coimbra MS Taking Charge of Graduate Medical Education: To Meet the Nation’s Needs in the 21st Century. 1993 New York, NY Josiah Macy Jr. Foundation
Back to Top | Article Outline

Reference cited in Table 1 only

51. Schumacher DJ, Bria C, Frohna JG.. The quest toward unsupervised practice: Promoting autonomy, not independence. J Am Med Assoc. 2013;310:2613–2614
    © 2015 by the Association of American Medical Colleges