As innovations in medical education move toward outcome-based training,1 specifically in postgraduate medical training, many organizations advocate identifying and assessing competencies as tools for defining the outcomes we should attain with our doctors in training. Competency-based training frameworks, although not new in medical education,2 have only recently been introduced into postgraduate training on a nationwide scale in several countries, and their impact has been unprecedented.3–7 This shift may be justified, but many involved in medical training are not convinced that a competency-based framework will lead to better training and health care, and some consider it no more than another educational fad.8–10
Some authors even worry that the competency-based movement will create new obstacles for sound training: “the real challenge is to produce expert professionals in a culture that emphasizes competency rather than expertise.”11 Competency rather than expertise? How is this confusion possible? Would professional medical educators and sensible governing bodies, such as the Royal College of Physicians and Surgeons of Canada (RCPSC), the Accreditation Council for Graduate Medical Education (ACGME) in the United States, the General Medical Council (GMC) in the United Kingdom, and the Central College of Medical Specialties in The Netherlands really introduce educational models that would discourage the development of expertise? This seems to be the perception of several authors.12,13 However, others seem to have found the key to success in postgraduate training with the introduction of competency-based training.14–16 Do these two groups of authors fundamentally disagree, or is there misunderstanding because they speak a different language when it comes to defining competency? It seems time to take a closer look at the confusion about competence terminology, and to find out what is needed to bridge different viewpoints about competency-based training.
A Competency Is a Personal Quality, Not an Action
Competency frameworks for postgraduate training usually read as logical sets of general qualities that every medical specialist—in other words, every doctor—should acquire. They have been reviewed by many individuals and committees for their relevance and comprehensiveness. Nevertheless, they still form a theoretical construct.
When it comes to translating the framework into a curriculum and into an assessment program, the meanings of competence and competency are not so clear. On one hand, competencies are usually formulated as broad, general attributes of a good doctor. On the other hand, as soon as we attempt to assess competencies, they tend to get reduced to detailed skills or activities: interpreting an ECG, counseling parents of a stillborn child, or performing a cesarean section. Lists of competencies tend to include individual activities (assessment of muscle atrophy, palpation of the spine) or even just the pathology in question (spontaneous abortion, multiple pregnancy).
We should be cautious here. Of course, authors of these lists mean to say that the trainee should posses the competence to interpret an ECG, or that the trainee should demonstrate the ability to handle a multiple pregnancy. But what happens in a competency-based assessment program is that the conceptual state of a person is mixed with his or her performance behavior, with an activity, or even with a medical term (Peter's competencies include genetic anomalies). This blurs the meaning of competence.17 We strongly support the view that competence and competency should be used only as they are primarily defined in the Oxford Dictionary of English: “The ability to do something successfully.”18 Competence and competency—both words may be used with the same meaning—are a “quality or state of being.”19 In the examples above, the actual competence seems specifically stated, but it is actually not defined at all. If we state that the doctor should have the competencies to be able to handle a multiple pregnancy, then our next question should be, what are these competencies? This question is rarely answered.
The origin of this linguistic confusion is not difficult to understand. As clinical educators implement competency frameworks into assessment programs, they must make competencies concrete so that they may be clearly assessed. Supervisors and trainees need to know which educational targets are important to attain, and they must know what, specifically, will be assessed. A general competency is difficult to assess, but the ability to execute a specific activity can be observed and appraised. So, clinical educators feel the need to specify competencies. However, this line of thinking tends to produce long checklists of specific competencies, formulated as activities that, in the end, do not quite reflect the original meaning of the general competency. In addition, linguistic laziness tends to make us shorten phrases where possible. But when the transcription of competency shifts from the ability to execute activity X to activity X, this adds to the terminology confusion.
Throughout this article we will, therefore, use the term competency in its original meaning—the ability to do something successfully.
Competencies Should Be Limited to General Attributes
General competencies are what most governing bodies and educational leaders responsible for physician training like to stress. For instance, the ability to perform the RCPSC's seven Canadian Medical Education Directions for Specialists (CanMEDS) roles3 is essentially the possession of general competencies that are demanded by the profession and society in general. In the day-to-day work environment, these roles are not automatically recognized because the competencies they represent are indeed so general. Residents are not asked to play a health advocate role on Monday, be a communicator on Tuesday, collaborator on Wednesday, and an expert on Thursday. All roles are intertwined in a complex way that makes them less visible and measurable.
It is the day-to-day activities that keep residents and their supervisors occupied. Running the consultation office, performing a diagnostic procedure, chairing a meeting, presenting a patient, and other such activities are the worries of the working day. Keeping a portfolio to document progress in the “vague” fields, such as health advocacy or professionalism, may easily feel like an extra burden on top of a heavy workload, distracting from the “real” concerns of the doctor. No wonder some supervisors start complaining when asked to keep tabs on these activities. On the other hand, keeping a logbook merely of lists of functional activities reduces the profession to that of a mechanic, checking off technical tasks as they are completed. The competent professional is clearly more than the sum of many detailed operational competencies.
We therefore propose to disentangle competencies from activities when constructing competency-based training programs. We argue that competencies should be defined as general attributes of a doctor and should be confined to a limited set of qualities. In our view, formulating long lists of specific competencies is not necessary and is confusing, and therefore we do not recommend it. Activities, on the other hand, are the constituting elements of professional work. For the purpose of constructing competency-based postgraduate training, they may well be specified to a limited number of so-called entrustable professional activities (EPAs).20,21 We will explain this concept and show that competencies and EPAs are related in a two-dimensional matrix.
Separate Competencies from Activities, but Use Them Both
From an educationalist's point of view, learning objectives must be specified. If they are phrased as competencies, there is a tendency to split each of these objectives into more detail. This has been done in the ACGME and CanMEDS frameworks. The ACGME model describes 28 more detailed competencies.4 The CanMEDS 2005 model also discerns 28 key competencies; however, 126 more specific enabling competencies are described on a more detailed level.3 When translating these competencies to a specific postgraduate course, a tendency to specify these competencies even further will arise. On the work floor, some clinicians get lost in the complexities of the competency-based assessment model. They are perfectly able to state which professional activities need to be carried out adequately, but they have trouble valuing these activities as competencies.
It is not necessary to choose between competencies and activities. Rather, by acknowledging that both are relevant pieces of the training process and that each represents a different dimension of the same overall objective of professional training, we can reconcile the concepts of competence and clinical practice. Chart 1 illustrates this reconciliation for postgraduate medical training in obstetrics–gynecology. An employer, a colleague, or a patient would probably be more interested to know whether a new obstetrics–gynecology resident has demonstrated enough competence to be entrusted with the professional activities in the top row of the figure than whether he or she has received high marks for the general competencies in the vertical column. However, at the same time, they would probably realize that these general competencies are important prerequisites for day-to-day practice.
When it comes to demonstrating competence, it is not one or the other—it's both. We should only fully trust colleagues or trainees to carry to out a critical activity once they have attained all the competencies that are needed to adequately complete this activity. Almost invariably, these activities require several (groups of) competencies, as is illustrated in Chart 1.
When we build a competency-based curriculum in this fashion, distinguishing important activities and emphasizing general competencies, it is helpful to identify the critical activities that constitute a specialty—all the elements that society and experts consider to belong to that profession, the activities of which we would all agree should be only carried out by a trained specialist. Critical activities as a term may be too limited, because it usually signifies only a few crucial, decisive events. We aim to identify all professional activities that we would agree a specific medical specialist can be asked to do. These activities can include executing an operational procedure, conveying bad news to a patient, chairing an interdisciplinary meeting, and many others. We have labeled these activities EPAs.20 Patients' and instructors' trust in a trainee and their entrustment of responsibility to that trainee are essential concepts in this approach, because they reflect the most important outcome of postgraduate training: a trainee's readiness to bear professional responsibility.21
EPAs and Statements of Awarded Responsibility
EPAs are those professional activities that together constitute the mass of critical elements that operationally define a profession. If we think of a competent obstetrician, we should be able to list those activities that form the core of that profession. Each of these activities may be defined as a unit of work that should only be entrusted upon a competent enough professional. EPAs are critical activities in a medical discipline that, according to opinion leaders in the field, must be assessed and approved of at some point during training. To be precise, EPAs have been specified with a number of conditions to be met, to demarcate them from daily activities that do not require specialist training (List 1).20
EPAs should be considered units of work that may be awarded a more or less formal qualification at the moment when supervisors confirm that the trainee is ready to assume responsibility for such activities. This entrustment does not require a ceremony, because it can happen at any moment. During a night shift, a supervisor on call must decide whether to trust a trainee to handle a complex case. In designing a competency-based curriculum, the entrustment of an EPA may be acknowledged more formally, in a statement that implies that a trainee has demonstrated enough competence to carry out the activity in question independently from now on. A statement of awarded responsibility (STAR) for a specific EPA may mark the threshold on which it is entrusted to a trainee to be carried out independently. Awarding STARs for EPAs over time requires supervisors to make deliberate decisions about their trainees' competence to perform all critical activities at some point in time.
Why create acronyms for these terms and pretend to introduce new concepts? The concepts in themselves may be not so novel. However, in our discussions with program directors and curriculum committees, we have found these acronyms to be extremely useful in explaining how to think in terms of competency-based training. Every clinical supervisor has experience with granting trust to a trainee, be it during a night shift or at other moments. Making this decision more formal by awarding a STAR helps supervisors understand and develop competency-based training. It strikes us how often residents are not allowed to carry out critical professional activities independently until very late in a course, or not at all during their training period, but the day after their specialist registration, they are expected to execute the breadth of all these activities independently. We have also used the EPA concept to guide the curriculum design for public health physicians, which generated great confidence among public health employers that their staff would be trained in skills that really matter.22
The proposed new national obstetrics–gynecology curriculum framework in the Netherlands suggests awarding a STAR when residents have reached a predetermined level of proficiency in executing an EPA. This curriculum acknowledges five levels of proficiency: (1) has knowledge, (2) may act under full supervision, (3) may act under moderate supervision, (4) may act independently, and (5) may act as a supervisor and instructor.23 This progressive curriculum is in line with the philosophy of learning-oriented teaching.24 Level 4 reflects the level of entrustment of the activity that may be awarded with a STAR. Chart 1 is derived from this curriculum.
Competency frameworks are increasingly used in undergraduate curricula as well. We believe that competencies should be related to the professional environment in which they must be demonstrated. In the terminology of entrustment, it only makes sense to give trainees responsibility for professional activities in the clinical environment. In the clinical clerkships, medical students may reach levels of competence that may be sufficient to execute well-defined, limited activities independently, ranging from measuring blood pressure to managing initial intake of patients and performing focused diagnostic and therapeutic procedures. In some medical schools, senior medical students receive responsibility to run a student-led ward.25
Assessing Competence: A Practical Approach
Supervisors are used to making judgments about trainees' entrustability in the execution of EPAs, even if it is identified by different terminology. Misjudgments in this area could be harmful for patients. Increasingly, in competency-based curricula, tests that are considered surrogate markers for clinical expertise are used for assessing trainees. Knowledge tests, simulations, logbooks, mini clinical evaluation exercises (MiniCEX) procedures, Objective Structured Assessment of Technical Skills, video observation, instruments to assess professional behavior, and many more procedures are being developed in an attempt to cover assessment of all relevant competencies. Clinical observations during the job and multisource feedback procedures have gained popularity, and the test battery is often summarized in a portfolio. Both the trainee and the responsible clinical teacher review and comment on these portfolios. Clinical supervisors make regular evaluations of competency development and set tasks for the next stage of training. Competence is expressed within a prescribed competency framework, be it the CanMEDS model, the ACGME model, the GMC model,6 or another set of general competencies. Many clinicians will confirm that these roles are all important, and many will be motivated to comply with the framework in their day-to-day teaching and assessment practices.
In practice, however, there are drawbacks to these widely used models. At first, the understanding, execution, and interpretation of many tests demand educational and psychometric expertise. For high-stakes decisions this may be justified, but many clinicians have too little time, expertise, and organizational and financial resources to profit fully from these sophisticated methods. However, they are often able to judge clinical performance reasonably well in clear-cut clinical tasks. They already experience varying levels of confidence when asking trainees to execute specific tasks. Working with portfolio-based competency frameworks that seem rather theoretical and disconnected from these clinical activities, and being asked to assess competency roles under these circumstances, may make many clinicians feel somewhat alienated from their own world.
We do not want to discourage the use of innovative methods of training and assessment. On the contrary, we do believe that many of these innovations are extremely valuable. At the same time, though, we are worried about the clinical supervisor. Moving away from the daily activities of the clinical ward to focus on sophisticated assessment procedures may lead to a decrease in a clinician–educator's interest and engagement in clinical teaching rather than an increase in these qualities. In our view, supporting the individual supervisor in the daily practice of clinical teaching should be the central focus of any innovation in postgraduate training, not just to optimize the success of innovations in postgraduate training. In addition, we believe that the supervisor's subjective but expert judgment is potentially a richer source of information than most other methods of assessment. Finally, a supervisor granting trust to a trainee to execute critical patient care implies assessing not only the trainee's competence, but also his or her performance. It is a powerful approach to assessment because it involves the supervisor personally. A failing resident often implies a failing supervisor.21
In the construction of an assessment program for postgraduate trainees, the described matrix relation between competencies and (entrustable) professional activities should, in our opinion, be the central focus. We believe that performance assessment of trainees, based on EPAs, can often be done with simple, concrete procedures. A valid assessment of general competencies often cannot be realized, but this type of assessment may not always be necessary. From a psychological and a psychometric research point of view, sophisticated procedures to measure general competencies may serve as valuable reference instruments in validation procedures. But, in general, we may well focus on the inference of general competence once enough EPAs have been rewarded with STARs. In fact, as EPAs reflect several general competencies at the same time, and as individual competencies show themselves in different EPAs, competencies are naturally achieved when all relevant EPAs for a particular competency have been rewarded with a STAR. If the profession is analyzed, all EPAs are listed, and a matrix is constructed in a way that is abbreviated in Chart 1, the curriculum can do justice both to the general educational and societal needs for general competencies and to the practice of clinical training on the ward.
We do not advocate a return to unreliable, traditional personal observations of trainees. Promising procedures, such as the MiniCEX, are being developed to support the quality of observations in a way that supervisors can handle. In addition, we need to objectify what types of observations deserve a STAR. Probably, information already available in the clinical environment may be used to determine these standards. In any case, we believe the responsible supervisor must play a central role in assessing his or her trainees. Future research may help to establish which sources of information best help supervisors make decisions of entrustment.
Using EPAs and STARs to Build a Competency-Based Curriculum
An analysis of any specialty should identify all relevant EPAs. There may be some debate about the breadth and comprehensiveness of an EPA, but it is important to note that EPAs always reflect a critical activity that may be fully entrusted to a trainee, once he or she is competent. We believe that 50 to 100 EPAs should be able to cover the objectives of a full postgraduate medical course of five to six years. List 1 specifies the conditions that should be met in compiling a list of EPAs. Next, EPAs should be contrasted with competencies in a framework such as the one illustrated in Chart 1.
Subsequently, program planners should construct a standard time line for the expected attribution of STARs for the different EPAs (Table 1). Deviations from the schedule should be evaluated regularly.
The time to achieve the STAR in a specific EPA will vary and will depend on at least four factors:
1. The EPA
When dealing with a complex and high-risk EPA, supervisors should expect trainees to have a slow learning curve. For an EPA that occurs frequently in a trainee's experience, the trainee should be expected to have a steep learning curve.
2. The working environment
If the clinic needs many clinicians with a particular STAR (e.g., for ventouse delivery on the labor ward), if the backup system for cases in which a trainee fails the EPA is well organized, or if the curriculum demands a STAR in the first years of training, a steep learning curve may be expected.
3. The trainee
A gifted trainee with a high level of motivation and sufficient self-confidence may have a relatively steep learning curve.
4. The clinical teacher
If the clinical teacher is comfortable with an EPA, is able to use the assessment procedure properly, and is able to reach a judgment, a steeper learning curve may be expected.
If the evaluation of a trainee shows a slower learning curve expressed in the level of competence of several EPAs, besides all specific information gathered in the test batteries, evaluators should consider these four factors. In a competency-based curriculum constructed with the help of EPAs and STARs, the notion of a flexible training length emerges as obvious.
Meeting the Assessment Needs of Both Theory and Practice
Competency-based curricula that mainly use test batteries to assess competencies from general competency frameworks risk losing the connection with the clinical work floor. We suggest using EPAs, critical professional activities of a medical discipline, as the central focus of curriculum building, without disregarding general competencies. EPAs and general competencies should relate in a two-dimensional framework, as illustrated in Chart 1. EPAs addressing all general competencies can be identified to ensure thorough, competency-based training. In our opinion, work-based assessment should focus both on EPAs and on general competencies, as it does in our model.
The standard time schedules we suggest can define when trainees should collect STARs for different EPAs. Using this model, evaluators can easily detect slow learners and conduct additional assessments to evaluate them if necessary. Fast learners will also be identified and may benefit from additional training experiences, may be asked to supervise tasks, and may eventually benefit from earlier certification.
We believe that this practical EPA-based approach to assessment will do justice to both educational theory and clinical teaching practice.