Secondary Logo

Journal Logo

CORR®Curriculum—Orthopaedic Education: Competency-based Medical Education—How Do We Get There?

Dougherty, Paul J. MD1,a; Andreatta, Pamela PhD, EdD, MFA, CHSE2

Clinical Orthopaedics and Related Research®: June 2017 - Volume 475 - Issue 6 - p 1557–1560
doi: 10.1007/s11999-017-5313-x
CORR Curriculum — Orthopaedic Education
Free

1Department of Orthopaedic Surgery, University of Florida, 655 W 8th Street, 2nd Floor ACC, 32209, Jacksonville, FL, USA

2College of Medicine, University of Central Florida, Orlando, FL, USA

ae-mail; paul.dougherty@jax.ufl.edu

Received February 15, 2017/Accepted March 3, 2017; previously published online March 9, 2017

A note from the Editor-in-Chief: We are pleased to offer the next installment of CORR® Curriculum-Orthopaedic Education, a quarterly column. The goal of this column is to focus on the mechanics of resident education. We welcome reader feedback on all of our columns and articles; please send your comments toeic@clinorthop.org.

The authors certify that neither they, nor any members of their immediate families, have any commercial associations (such as consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.

All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request.

The opinions expressed are those of the writers, and do not reflect the opinion or policy of CORR® or The Association of Bone and Joint Surgeons®.

Back to Top | Article Outline

Introduction

Competency-based medical education (CBME), a pedagogical approach that advances residents as they master a set of defined performance standards [5, 8, 9, 13], is gradually replacing graduate medical education's traditional time-based structure in places like Canada and the United Kingdom (UK). Why the move away from the traditional time-based educational approach? The amount of time spent learning a surgical skill does not define how well a person performs that skill. Additionally, because of the renewed focus on resident working hours, the amount of time available for hands-on clinical training has decreased [10]. As a result, orthopaedic residency programs outside the United States are adopting CBME programs where residents have more control in determining when they graduate and faculty have more assessments at their disposal to determine resident competency. Given that there are no CBME orthopaedic surgery residency programs currently implemented in the United States, the question should be asked: Will US residency programs eventually follow suit and adopt CBME?

Back to Top | Article Outline

Improved Training

CBME is certainly an attractive option for both the residency program and the resident. Compared to traditional time-based educational programs, CBME offers more-precise criterion-based performance objectives for both residents and faculty, provides frequent formative and summative assessments to residents, and allows the resident to advance through the program at his or her own pace.

Back to Top | Article Outline

Performance Objectives

CBME programs require a demonstration of mastery in a targeted domain supported by criterion-referenced and objective performance-based assessments [1, 3]. In a CBME model, faculty members provide customized instruction to the resident, addressing the strengths and weaknesses of each individual student [2]. Progressing through the program requires attaining intermediary prespecified competency levels along the way [7]. Once criterion-based performance objectives are set, teaching focuses on those areas needed to meet the performance objectives.

Back to Top | Article Outline

Reliable Assessment Tools

The global end-of-rotation evaluation provides insufficient feedback for most aspects of patient care, and inadequately covers the range of operative skills required. CBME requires both formative tests that show progress towards proficiency targets and summative tests that show competency through performance-based demonstration of abilities. Summative tests are not measured relative to others [6]. Instead, they indicate whether the learner has achieved the specified performance within a range of acceptable standards [11].

Back to Top | Article Outline

Pace

Since each resident progresses at his or her own rate in a CBME model, completing the program may potentially take less time than the prescribed 5 years seen today. There is potential to get into practice earlier, or, in some situations, going slower to accommodate other needs, such as having children. Having this flexibility makes it more likely to accommodate a broad range of individuals for the specialty.

Back to Top | Article Outline

CBME in Canada and the UK

Two of the most recent examples of residency programs transitioning to CBME can be found in Canada and the UK.

Although the University of Toronto orthopaedic surgery residency implemented a competency-based program just 7 years ago, the Royal College of Physicians and Surgeons of Canada has mandated that all residency programs must be competency based by 2020 [5, 9]. In July of this year, Queen's University in Toronto will become the first institution in Canada to fully transition all of its postgraduate residency programs to a CBME program [12].

The Royal College of Physicians and Surgeons of Canada breaks down the elements of orthopaedic surgery residency into modules (with topics like hip fractures, arthroscopy, arthroplasty, for example), and faculty teaches them to residents with appropriate feedback and assessment. Teaching specific modules for mastery in a certain area is a hallmark of CBME.

A recent study [5] followed the first 14 residents in a CBME program at the University of Toronto and compared them to those in a more-conventional residency program. The authors found that two residents finished the program earlier and two finished later, while the remaining residents finished the program in the same average time as those in time-based education programs. Despite extra coaching and attention, particularly in the PGY 1-3 years, no discernible differences in competence between the two groups were found. Moreover, those in the competency-based program required more faculty time and support. While the authors did not put a dollar figure on this extra amount of support, they felt that earlier graduation could be a cost-savings for the resident [5]. But CBME does not appear to be a cost savings for the University of Toronto. The need for extra faculty for a surgical simulation center plus increased assessments makes the CBME model at the University of Toronto a more resource-intensive proposition.

In the United Kingdom, as part of the 2005 mandate for Modernizing Medical Careers, the British Orthopaedic Association and Specialist Advisory Committee in Trauma and Orthopaedics developed the Orthopaedic Competence Assessment Project to improve orthopaedic resident (registrar) education.

This program consists of instruments to help with registrar assessment, setting agendas, and coaching. All training programs are mandated to have documented workplace-based assessments, including small clinical exercises, procedure-based assessments, case-based discussions, and peer assessments as part of their portfolio.

Prior to the rotation, the trainer describes his or her practice, the types of patients seen and treated, and commonly performed surgical techniques. The registrar provides his or her portfolio, which documents their prior experiences. Both will come to a consensus on the registrar's educational goals for the rotation (called attachments) including scope of cases and improving orthopaedic knowledge. The registrar enters operative cases into an eLogboook, so that procedures can be reviewed and their progress documented. The eLogbook includes an assessment for every procedure. In building a more competency-based framework, the UK programs now have more prescribed educational goals for the overall program and for each attachment. Additionally, the registrar is assessed more frequently on operative procedures compared to time-based education.

While the examples from Canada and the UK show potential, they are far from finished products, as both programs lack reliable criterion-based evaluation tools to cover the breadth of orthopaedic education.

Back to Top | Article Outline

US Orthopaedic Residency Programs and CBME

Although we believe CBME will eventually be a part of US residency programs, implementing it for all aspects of orthopaedic surgery is unlikely at this time. Before a transition to CBME can take place, stakeholders and performance assessment experts must agree on criterion-based objectives and the particular assessments needed to provide robust formative and summative feedback.

Those same experts must also consider that an increased number of assessments could place a heavy burden on faculty members. With the advent of recent changes in graduate medical education over the last few years (ie, the Next Accreditation System and Milestones [4]), many programs are already feeling overworked, without necessarily having increased resources.

Back to Top | Article Outline

How Do We Get There?

Clearly, transitioning to a CBME program will take many years. Mandating a shift in educational philosophy on a short timeframe will result in implementing expensive, but ineffective assessment tools that will overburden faculty and fail to improve resident competence.

Orthopaedic surgery residency programs can avoid those pitfalls by first gathering longitudinal data to determine the most-essential criterion-based assessments for the transition to CBME. The Accreditation Council for Graduate Medical Education (ACGME) or American Board of Orthopaedic Surgery (ABOS) should fund this research, as they will play a major role in implementing any future changes to the curriculum.

After gathering data on CBME from a few sites over 5 years, a broad group of orthopaedic surgeons and performance-assessment experts with psychology and education backgrounds can develop accurate criterion-based performance objectives. Once the new criterion-based assessments are established and proven to be effective, they can be applied to all programs.

Overburdening faculty members with additional assessments is a valid concern. But as the residency program changes, so too will the faculty, likely in both expertise and sheer size. In order to provide effective performance assessment, orthopaedic surgery departments will likely need to hire surgeons who have an advanced educational mission. While MBA degrees are commonplace among present-day leadership, program directors should hire candidates with a Master's level or above degree in Surgical Education to teach other faculty members about CBME.

Another reasonable intermediate step might be to start developing a transition plan based on the 16 ACGME milestones, which represent a cross-section of general orthopaedic surgery. The ACGME requires reporting of Milestones data every 6 months for every resident, based on the previous 6 months’ rotations. Presently, where a resident performs within this framework is based on the judgement of the residency program's Clinical Competency Committee. Currently, the committee makes decisions on the ACGME milestones with very limited data, thus leaving the accuracy of the Milestones in doubt. Therefore, development of criterion-based performance assessments within the framework of the Milestones might be an ideal way to integrate aspects of CBME into the structure already in place.

Given the public demand for more accountability, as well as the lack of effective assessment systems, some form of CBME will need to take place. How this will take place, and at what expense, should be a part of the discussion by leadership in orthopaedic surgery in the near future.

Back to Top | Article Outline

References

1. Argüelles A, Gonczi A. Competency Based Education and Training: A World Perspective 2000;Noriega EditoresMexico City, Mexico.
2. Bloom BS. Bloom BS, Hastings JT, Madaus G. Learning for mastery. Handbook of Formative and Summative Evaluation of Student Learning 1971;New York, NYMcgraw-Hill.
3. Burke JW. Competency Based Education and Training 1989;Bristol, PAThe Falmer Press, Taylor & Francis Group.
4. Dougherty PJ. What the ACGME's next accreditation system means to you. Clin Orthop Relat Res. 2013;479:2746-2750 10.1007/s11999-013-3117-1.
5. Ferguson PC, Kraemer W, Nousiainen M, Safir O, Sonnadara R, Alman B, Reznick R. Three-year experience with an innovative, modular competency-based curriculum for orthopaedic training. J Bone Joint Surg Am. 2013;95:e166 10.2106/JBJS.M.00314.
6. Glaser R. Glaser R. Psychology and instructional technology. Training Research and Education 1962;Pittsburgh, PAUniversity of Pittsburgh Press.
7. Hambleton RK, Eignor DR. Jaeger RM, Tittle CK. Competency test development, validation, and standard setting. Minimum Competency Achievement Testing: Motives, Models, Measures, and Consequences 1980;Berkeley, CAMcCutchan Publishing Corporation.
8. Houston WR. Houston WR. Competency based education. Exploring Competency Based Education 1974;Berkeley, CAMcCutchan Publishing Corporation.
9. Nousiainen MT, McQueen SA, Hall J, Kraemer W, Ferguson P, Marsh JL, Reznick RR, Reed MR, Sonnadara R. Resident education in orthopaedic trauma. The future role of competency-based medical education. Bone Joint J. 2016;98B:1320-1325.
10. Orthopaedics One. Time-based or competence-based residency training. Available at: http://www.orthopaedicsone.com/display/Viewpoint/Time-based+or+Competence-based+Residency+Training. Accessed February 21, 2017.
11. Popham WJ, Husek TR. Implications of criterion- referenced measurement. In: Popham WJ, ed. Criterion-referenced Measurement. New Jersey; Educational Technology Publications: 1969.
12. Queen's University. Competency-based medical education. Available at: https://meds.queensu.ca/education/postgraduate/cbme. Accessed February 21, 2017.
13. Tuxworth E. Burke JW. Competence based education and training: Background and origins. Competency Based Education and Training 1989;Oxon, UKFalmer Press.
© 2017 Lippincott Williams & Wilkins LWW