Continuing professional development (CPD) is the process by which health professionals maintain and enhance their clinical performance to meet the needs of patients in today’s rapidly evolving health care systems. In many countries, participation in accredited CPD activities is mandated by professional or regulatory bodies or stimulated by other incentives.1 CPD should not only improve knowledge but also change professional behavior in daily practice. However, most certified CPD activities focus on knowledge dissemination that serves to reassure health professionals about the quality of their practice without producing a significant change in their behavior.2–5 CPD activities that might lead to change in clinical practice are therefore needed.
Much current effort by CPD providers focuses on developing strategies to assess the outcomes of the activities they offer. Many conceptual frameworks for CPD outcomes assessment are derived from Kirkpatrick’s model,6 which assesses training effectiveness by measuring four distinct levels: participants’ satisfaction; participants’ knowledge, skills, or attitudes; transfer of learning to practice (i.e., behavior); and organizational outcomes such as productivity and quality (see Table 1). Thus, CPD activities specifically targeting the behavior change component of improving training effectiveness can be assessed using Level 3 outcome measures.6 In reality, however, most CPD providers only assess Level 1 and 2 outcomes, using postactivity self-administered questionnaires. Level 3 and 4 outcomes have been measured in the context of research projects using health services methods,7,8 using self-reported questionnaires,9 or during professional inspection visits by regulatory bodies.5
The stated learning objectives of CPD activities describe their desired outcomes and thus should help shape the activities. According to most continuing medical education guidelines, learning objectives should clearly state, in measurable terms, the behavior the learner is expected to adopt on completion of the activity.10–12 They should also indicate the context in which the behavior should occur and the criteria for acceptable performance.13
Bloom’s taxonomy14 is one tool used by CPD providers to structure the learning objectives of CPD activities. This taxonomy defines three types or domains of learning: affective, psychomotor, and cognitive. Affective learning relates to attitudes in clinical practice; psychomotor learning relates to physical skills. Cognitive learning is divided into six levels of increasing complexity: knowledge, comprehension, application, analysis, synthesis, and evaluation (Table 2). Although the lowest levels of cognitive learning may enable CPD participants to comprehend and apply knowledge in their current practice, they are not equipped to analyze or critique it. Participants in a CPD activity that targets higher cognitive levels are more likely to implement a behavior change in their clinical practice. For example, synthesis involves using knowledge to produce a set of operations, and evaluation involves judging knowledge in terms of external criteria.15 Thus, the more complex the level of cognitive learning achieved during a CPD activity, the more effective the activity is likely to be at translating knowledge into clinical and organizational behavior change (Levels 3 and 4 of Kirkpatrick’s model).15,16
In this study, we sought to identify which of Bloom’s domains and levels of cognitive learning14 are targeted by the learning objectives of CPD activities offered by medical associations, regulatory bodies, and academic institutions in the province of Quebec, Canada.
CPD activities search
This study was a secondary analysis of an existing database that originated in a before-and-after study for the development of a theory-based instrument to assess the impact of CPD activities on clinical practice.17,18 These data on CPD activities were collected from all eight of the Quebec institutions responsible for accredited CPD activities: the College of Family Physicians of Canada, Médecins francophones du Canada, the Fédération des médecins omnipraticiens du Québec, the Fédération des médecins spécialistes du Québec, and four medical schools (McGill University, Université de Montréal, Université Laval, and Université de Sherbrooke), which together offer an average of 1,200 accredited activities per year (data available from authors). We included each CPD activity that was offered to practicing physicians between November 2012 and March 2013 and that met the following criteria: It was accredited by a medical association, a regulatory body, or an academic institution in the province of Quebec; it was related to a behavior relevant to clinical practice; and it was a group-based educational activity, preferably conducted as a live activity for groups of 50 participants or fewer, because smaller-scale interactive sessions are more likely to change physician behavior than larger-scale didactic sessions.2 Activities could occur in any setting (e.g., a university, a conference center, a practice setting) and could use any single or combination of instructional methods (e.g., lectures, workshops, case studies, demonstrations) and material (e.g., audience response systems, videos, card games, real or simulated patients). Activities embedded within large programs offering several activities in one setting, such as two-day conferences, were also eligible and were evaluated individually.
Analysis of CPD activity learning objectives
We extracted all learning objectives from eligible activities. Two reviewers (A.F. and P.T.L.) independently classified the stated objectives of each CPD activity according to the definition of the learning domains described in Bloom’s taxonomy.14 Both reviewers were experienced in data extraction and in designing and performing systematic reviews. We computed the frequency of each learning domain targeted by the learning objectives of each activity to evaluate how often the different levels of learning were targeted in CPD activities. We computed the interrated reliability of agreement between reviewers. Discrepancies in classifications were resolved through consensus. We also evaluated the verbs used in the learning objectives to describe intended behaviors. First, we classified these verbs using the list of sample verbs used in many continuing medical education guidelines for assessing Bloom’s learning domains10–12 (e.g., “cite” and “list” are verbs used to communicate knowledge). However, as some verbs belong to more than one learning domain (e.g., “discuss” can be used to “communicate comprehension” or for “synthesis,” depending on the context), reviewers evaluated all verbs in the context of the full wording of the learning objective. Finally, we calculated the frequency of each learning domain according to the type of instructional method used.
Ethical approval for the project was received from the institutional review board of the Centre Hospitalier Universitaire de Québec.
Characteristics of CPD activities
Out of 110 eligible activities (see Table 3), the majority (92) were embedded within large programs offering several activities in one setting, such as a medical conference. A mix of different instructional methods was used. However, confirming what is currently described in the literature,19 the majority of the activities analyzed (96) were lectures, the method most commonly used by CPD providers. Nine activities used interactive workshops, and 5 others used case studies as instructional methods. Two activities sed a combination of simulated patients and videos as instructional material. The target population for 85 activities was physicians only, whereas 25 activities targeted health care professionals in general, including nurses, pharmacists, and occupational therapists. The duration of the identified activities varied from one hour to two and a half days.
Evaluation of the learning objectives
A total of 404 learning objectives described the goals of the 110 CPD activities identified in this study (Table 4), representing an average of four learning objectives (SD = 1.6, range 1–10) per activity. We observed that the majority of the learning objectives (389) targeted the cognitive domain of learning, and most of these concentrated on “knowledge” (94) and “comprehension” (94). These levels correspond to Kirkpatrick’s Level 2; that is, the outcomes of these activities relate to the recall of previously learned material and the ability to grasp its meaning. According to Bloom’s taxonomy, these represent the simplest levels of understanding. Out of 122 verbs, the terms most often used in objectives classified under these two levels were “recognize” (30), “know” (22), “describe” (21), “identify” (21), and “explain” (10). Ninety-eight learning objectives analyzed targeted the “application” level of the cognitive domain. In this level, learners apply what they know and comprehend to a given situation. The three verbs most used in this level were “perform” (13), “use” (11), and “apply” (10). Higher levels of the cognitive domain (i.e., the most complex) were less often targeted by the learning objectives analyzed. “Analysis” (65), “synthesis” (18), and “evaluation” (20) together represented only 25% of the learning objectives analyzed.
In addition, we observed that the other two of Bloom’s learning domains, affective and psychomotor, were targeted by only 1.5% (6) and 2.2% (9) of learning objectives, respectively. The affective learning domain refers to learners’ attitudes, interests, or emotions toward a subject. Verbs found in this domain were “humanize,” “balance,” “enhance,” “reflect,” “encourage,” and “advise.” The psychomotor domain is related to the use of motor skills in clinical practice. The verbs found to represent this domain were “handle” (e.g., a defibrillator), “examine” (e.g., a newborn), “master” (a technique), and “perform” (e.g., a knee exam).
The learning objectives were not influenced by the type of instructional method used (see Table 5). On average, 45% of the learning objectives of all activities analyzed targeted knowledge and comprehension levels. Interactive workshops were least likely to target knowledge and comprehension (37%) and more likely to target the application level. Higher levels of the cognitive learning domain were targeted on average by 25% of the activities (26% of lectures, 20% of interactive workshops, and 28% of case studies). As CPD activities often described learning objectives that corresponded to several different learning levels at once, we also analyzed how these levels were distributed among the different activities. We found that for 49% of the 110 total CPD activities analyzed, 50% or more of learning objectives belonged in the “knowledge” or “comprehension” levels. Moreover, in 78% of the total CPD activities analyzed, 50% or more of learning objectives belonged in the “knowledge,” “comprehension,” or “application” levels. Only 16% of CPD activities focused on the higher cognitive skills of “analysis,” “synthesis,” or “evaluation.” The interrater reliability between reviewers was moderate (Kappa = 0.63). To enhance the reliability of our findings, they discussed all discrepant classifications until consensus was reached.
A pathway to clinical change
Our findings show that half of the learning objectives from our sample corresponded to the “knowledge” and “comprehension” levels of the cognitive domain. Learning objectives for an introductory course may be appropriately concentrated in the lower levels, such as “knowledge” and “comprehension,” whereas those for upper-level courses will normally be concentrated in the upper levels of the cognitive domain.14,15 The overall goal of CPD activities is not only to convey information.1 It is also to encourage critical thinking and reasoning skills (i.e., more complex and applicable forms of knowledge) among health professionals in order to promote a behavior change related to better outcomes for patients.1,20 Thus, the acquisition of higher thinking skills should be the norm in most CPD activities. Of all the activities in our sample, 78% of them addressed “knowledge,” “comprehension,” or “application” levels of cognitive learning. Although at this level participants are theoretically able to comprehend and apply what they know to a given situation, it is important to note that these skills may not equip health professionals to differentiate old practices from new ones (analyze) or critique them (evaluate) in order to plan (synthesize) and implement new clinical behavior, which should be the main goal of CPD activities. In addition, verbs often used in the learning objectives analyzed, such as “know,” “learn,” “appreciate,” and “understand,” can be interpreted in many different ways and are inappropriate for describing observable behaviors.10–12
For health professionals to maintain and enhance their clinical performance in order to meet the needs of patients, they must achieve higher learning levels in all domains (cognitive, affective, and psychomotor). Through dissemination of more complex levels of knowledge, CPD providers can transfer skills that are more tangible and operational and thus more likely to change participants’ clinical practice.6,15
Towards new models?
Not all physicians are at the same learning stage when they attend a CPD activity, and hence not all of them are ready to change their practice.21 Social cognitive theories, such as the theory of planned behavior (TPB), explain how people acquire and maintain certain behavioral patterns, while also providing the basis for intervention strategies.22,23 The TPB has long been used to investigate health professionals’ behavior changes in clinical practice. A systematic review evaluating the intentions and behaviors of health care professionals suggested that although other theories better capture the dynamic underlying intention, the TPB is the most appropriate theory for predicting behavior.23 Our systematic review of interventions targeting health care professionals found that CPD planners rarely use the TPB (or any other sociocognitive theory) in designing their activities.24 The TPB indicates that health professionals’ clinical behaviors will be determined by their intention to engage in those behaviors. Intention, in turn, is the result of the attitude toward the behavior, the influence of significant others, and the perceived ease or difficulty of behavioral performance. These variables could be helpful to CPD providers because they correspond well to Kirkpatrick’s four-level model for assessing the success of a CPD activity. The learning outcomes related to attitude and skill improvement (Kirkpatrick’s Level 2) determine behavioral intention, whereas Level 3 outcomes predict behavior related to clinical practice. The necessary transition from Level 2 to Level 3 can only occur as a result of changes promoted by the content and format of the CPD activity. As required by the TPB, CPD activity learning objectives should specify the population, clinical condition, behavior of interest, and, wherever possible, the timeframe required to perform the behavior.22 One study has demonstrated how a CPD activity based on the TPB proved to have a more positive impact on the uptake of evidence-based practice than CPD activities that were not theory based.25 We recommend that CPD providers benefit from these well-known theories to design CPD activities that equip physicians and health care providers with the skills needed to change their practice. This in turn will lead to improved clinical performance for the benefit of patients, health care professionals themselves, and the health care system.
A first step for a CPD provider would be to clearly delineate the clinical behavior to be reinforced using the TACT principles (target, action, context, and time). A CPD activity could, for example, aim for a group of family physicians (target) to implement recommendations on screening for breast cancer in average-risk women aged 40 to 74 years (action)26 in their clinical practice (context) within the next three months (time). Next, sociocognitive theories propose that interventions must also be designed according to determinants of behavior if they are to achieve behavior change. According to the TPB, those determinants are attitude, subjective norm, perceived behavioral control, and intention. Thus, the CPD provider should find out beforehand which of these determinants need to be addressed in order for the family physicians to make the desired behavior change. If the group has negative attitudes toward the screening recommendations, the activity could include presenting the research that led to the new recommendations, and demonstrating its benefits to patients. However, for a group with very positive attitudes toward the recommendations, such an approach is unlikely to enhance uptake. This focus on physicians’ judgment of and attitude toward the screening recommendations targets higher learning domains than providing information about the recommendations. Thus, a focus on behavior change will require a new form of planning from CPD designers, which includes (but is not limited to) identifying relevant determinants of behavior change and formulating learning objectives that focus on the higher cognitive skills and on affective and psychomotor domains.
This study has some limitations. First, we acknowledge that many factors other than the learning objectives of a CPD activity can influence their effect on performance outcomes. CPD activities that disseminate knowledge can lead to clinical practice change even if that change is not an explicit learning objective. However, addressing factors that are known to determine behavior change, such as motivation to change or the clinical context, may be a better strategy for achieving desired performance outcomes than simply promoting knowledge acquisition.27 Second, over the last two decades, there have been attempts to identify the instructional strategies for learning activities that are most effective in either changing professional behavior or improving patient outcomes. A number of articles2,19,28,29 report that traditional CPD activities such as large conferences are not effective in changing professional behavior, whereas interactive sessions do appear to promote a change in professional practice and in patient outcomes.2 However, we believe that a successful CPD activity—that is, one that constitutes an effective intervention for clinical behavior change—will align the planning of learning objectives based on sociocognitive theories with the content taught, the instructional strategies, and the evaluation methods. Finally, further research should be done to assess the available resources and structures already in place in the CPD community and the impact of CPD activities that address the various learning domains, including cost–benefit analysis. This could lead to the development of more efficient CPD activities.
The learning objectives of CPD activities in their present form do not promote change in clinical behavior. This is at least partly attributable to the fact that their learning objectives do not target a desirable degree of cognitive complexity. Most objectives focus on transmission of knowledge rather than application of knowledge. We suggest the use of established theoretical frameworks, such as the TPB, to develop CPD activities that encompass all necessary constructs for tangible change in professionals’ clinical behavior and that will ultimately lead to improved patient outcomes.
Acknowledgments: The authors wish to thank Louisa Blair and Dr. Patrick Archambault for editing this article.
1. Peck C, McCall M, McLaren B, Rotem T. Continuing medical education and continuing professional development: International comparisons. BMJ. 2000;320:432–435
2. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282:867–874
3. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: A systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423–1431
4. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. Int J Technol Assess Health Care. 2005;21:380–385
5. Goulet F, Hudon E, Gagnon R, Gauvin E, Lemire F, Arsenault I. Effects of continuing professional development on clinical performance: Results of a study involving family practitioners in Quebec. Can Fam Physician. 2013;59:518–525
6. Kirkpatrick DL Evaluating Training Programs: The Four Levels. 1994 San Francisco, Calif Berrett-Koehler Publishers Inc.
7. Moore DE Jr, Cervero RM, Fox R. A conceptual model of CME to address disparities in depression care. J Contin Educ Health Prof. 2007;27(suppl 1):S40–S54
8. Laprise R, Thivierge R, Gosselin G, et al. Improved cardiovascular prevention using best CME practices: A randomized trial. J Contin Educ Health Prof. 2009;29:16–31
9. Goulet F, Gagnon RJ, Desrosiers G, et al. Participation in CME activities: Family physicians in practice for more than 25 years versus those in practice for less than 25 years. Can Fam Physician. 1998;44:541–548
11. Conseil de l’éducation médicale continue du Québec. Continuing Medical Education Handbook [in French]. 1998 Montreal, Quebec, Canada Conseil de l’éducation médicale continue du Québec
13. Mager RF Preparing Instructional Objectives. 19842nd ed. Belmont, Calif Lake Publishing Co.
14. Bloom BS, Krathwohl DR. Taxonomy of educational objectives: The classification of educational goals by a committee of college and university examiners. Handbook 1: Cognitive Domain. 1956 New York, NY Longmans
15. Krathwohl DR. A revision of Bloom’s taxonomy: An overview. Theory Pract. 2002;41:212–218
16. Koedinger KR, Roll IHolyoak KJ, Morrison RG. Learning to think: Cognitive mechanisms of knowledge transfer. In: Oxford Handbook of Thinking and Reasoning. 20122nd ed. New York, NY Cambridge University Press:789–806
17. Légaré F, Borduas F, Jacques A, et al. Developing a theory-based instrument to assess the impact of continuing professional development activities on clinical practice: A study protocol. Implement Sci. 2011;6:17
18. Légaré F, Borduas F, Freitas A, et al.CPD-KT Team. Development of a simple 12-item theory-based instrument to assess the impact of continuing professional development on clinical behavioral intentions. PLoS One. 2014;9:e91013
19. Harrison C, Hogg W. Why do doctors attend traditional CME events if they don’t change what they do in their surgeries? Evaluation of doctors’ reasons for attending a traditional CME programme. Med Educ. 2003;37:884–888
20. Moore DE Jr, Green JS, Gallis HA. Achieving desired results and improved outcomes: Integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29:1–15
21. Slotnick HB. How doctors learn: Physicians’ self-directed learning episodes. Acad Med. 1999;74:1106–1117
22. Ajzen I The Theory of Planned Behavior. 1985;Vol 1 London, UK Sage
23. Godin G, Bélanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals’ intentions and behaviours: A systematic review of studies based on social cognitive theories. Implement Sci. 2008;3:36
24. Légaré F, Ratte S, Stacey D, et al. Interventions for improving the adoption of shared decision making by healthcare professionals. Cochrane Database Syst Rev. 2010(5):CD006732
25. Casper ES. The theory of planned behavior applied to continuing education for mental health professionals. Psychiatr Serv. 2007;58:1324–1329
26. Canadian Task Force on Preventive Health Care. . Recommendations on screening for breast cancer in average-risk women aged 40–74 years. CMAJ. 2011;183:1991–2001
27. Kanouse DE, Jacoby I. When does information change practitioners’ behavior? Int J Technol Assess Health Care. 1988;4:27–33
28. Cantillon P, Jones R. Does continuing medical education in general practice make a difference? BMJ. 1999;318:1276–1279
29. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700–706