Purpose: Physicians frequently are asked to sign commitments to change practice, based upon their involvement in continuing medical education (CME) activities. Although use of the commitment-to-change model is increasingly widespread in CME, the effect of signing such commitments on rates of change is not well understood.
Method: Immediately after a CME session, 110 physicians were asked to specify a change they intended to make in practice and to designate a level of commitment to change. To determine the effects of a signature on rates of change, physicians were randomly assigned to control (signature) and experimental (non-signature) groups. Follow-up surveys were conducted at two and three months to determine rates of change.
Results: In all, 88 physicians completed the first questionnaire, and 64 of them completed the follow-up. Consistent with prior studies involving the commitment-to-change model, those expressing an intention to change were significantly more likely to change on follow-up (p = .035). There was no significant difference between signature and non-signature groups (p = .99), regardless of age or gender.
Conclusions: Signatures appear unimportant to assuring compliance with commitments to change used in CME conferences. A physician's behavior can be expected to change if the specified change is consistent with the physician's beliefs and sense of what is important. The relative influences of components of the commitment-to-change model require further study to determine more clearly their roles in causation and measurement.
Dr. Mazmanian is associate dean, Continuing Medical Education; Dr. Johnson is associate professor, Department of Biostatistics; and Mr. Boothby is director of conferences and alternative instruction, Office of Continuing Medical Education, all at Virginia Commonwealth University School of Medicine. Ms. Zhang is an instructor, Center for International Programs, Virginia Commonwealth University. Ms. Yeatts is regulatory boards administrator, Department of Health Professions, Commonwealth of Virginia. All are in Richmond.
Correspondence and requests for reprints should be addressed to Dr. Mazmanian, Associate Dean, Office of Continuing Medical Education, Virginia Commonwealth University School of Medicine, Box 980048, Richmond, VA 23298-0048; telephone: (804) 828-0492.
Formal agreements are common in medical education. Faculty evaluations, performance plans, and learning contracts generally include goals and expectations that physicians agree upon with supervisors, mentors, or teachers. Physicians who attend continuing medical education (CME) activities frequently are asked to agree to make changes in practice based upon their participation in the CME activity. Strictly speaking, the goals that faculty, CME participants, or others have in mind are concerned with the outcomes of actions, and intention drives action when a physician agrees or commits to embark upon change.1 Commitment to change involves mindful reflection upon goals as well as outcomes of actions.
“Commitment to change” is a planning model for enabling and measuring behavioral change.2 Its methods are conceptually tied to the expectation that the learner who understands what is required and who is motivated to achieve a clearly specified task is more likely to succeed than is one who is uncertain or not motivated.2 The methods of the commitment-to-change model include a series of questionnaires administered to learners who agree to make a change and rate the strength of their commitment to successfully implement it. Ordinarily, in this model, the physician signs the agreement to change.2–4
Questions remain about the commitment-to-change model. Can a clearly articulated change with the highest level of commitment function as a surrogate for more expensive follow-up interventions and evaluation? Do follow-up mailings prompt change or collect data? We sought to discover whether learners who signed their commitments to change were more likely to change than were those who did not. Even though prior work in the social sciences2–6 provides evidence that expressed agreements to make specified changes are important to assuring success, the literature of law and business confirms only that mechanisms of agreement and contract, while common, attract no special attention until they go bad. A search of Education Abstracts, ERIC, Lexus-Nexus, and Psychological Abstracts revealed no prospective study linking signatures to higher rates of change. The paucity of information about the impact of signatures on physicians' agreements to make changes in their practices prompted us to test the value of a signature in the standard hard-copy approach to commitment to change.
We conducted a randomized controlled trial using the commitment-to-change model in conjunction with a CME conference. In planning for the conference, academic physicians, medical practitioners, and educators assessed learning needs by reviewing prior course evaluations, clinical guidelines, and advances in basic and clinical research.7–9 They generated instructional objectives to inform learners, develop lectures, and guide evaluation. During the session, three lectures with question-and-answer sessions were presented: recognizing selected treatments for athletes with exercise-induced asthma, defining risks associated with sports participation for athletes with common infections, and developing a treatment plan for adolescent women suspected to have disordered eating, amenorrhea, and osteoporosis.
On registration forms for the CME conference, enrollees designated their names, professions, and specialties. As registration forms were received, the data were entered into a spreadhseet. To minimize differences between the experimental and control groups and to reduce the overall standard of error, a gender proxy was used in sampling (based on information obtained from the conference registration forms), and the groups were matched according to the gender variable.
To isolate the effects of a signature, a single questionnaire with the attendee's name and degree printed at the top was prepared prior to the conference. It asked attendees to indicate whether they would or would not make a change in practice as a result of their participation in the CME session, or whether they were uncertain about making a change. Those intending to change their clinical behaviors were asked to specify the changes in writing and to indicate on a rating scale of 1 to 5 (1 indicating the lowest level of commitment) how committed they were to making the changes. A signature line was displayed on the questionnaire for the control group; the experimental group received the questionnaire without a signature line. Attendees were blinded to the difference between the two groups as well as to the research question. Conference faculty and staff also were blinded to avoid contamination of the research design.
At the conference site, each enrollee received the pre-coded questionnaire in an envelope with his or her name on it. At the beginning of the session, the conference chairman announced the study and explained the medical school's interest in evaluating learning and change in the lives of physicians. At the end of the session, the associate dean for CME reiterated the medical school's interest in physicians' change, learning, and ongoing program evaluation. He explained that participation in the study was not mandatory. Four medical school faculty and staff collected completed questionnaires, as attendees left the room.
Two months after the conference, a packet of information was sent to all those from whom questionnaires had been collected. The packet consisted of a cover letter, the originally completed questionnaire, and a new questionnaire asking the physicians to report whether they had changed their practices as a result of their participation in the conference. In March 1999, three months after the session, a second follow-up mailing of the same information was sent to those who had not responded to the first follow-up mailing. All data were entered into a database, and the results were analyzed with a standard statistical software package. The statistical significances of differences between means were assessed by two-sample t-test. Differences between percentages were assessed with the chisquared test.
A total of 112 physicians enrolled in the CME activity. Two enrollees did not attend the conference and the remaining 110 were assigned randomly to either the control group, which had the opportunity to sign the questionnaire, or the experimental group, whose questionnaire had no signature line.
In all, 88 physicians (80%) completed the initial questionnaire, 45 of the signature group (82%) and 43 of the non-signature group (78%). The average age of the participant was 42 years (range 28 to 66 years). Sixteen (18%) of the participants were women, and these women physicians were an average of 6.4 years younger than were the men (36.6 versus 43.0 years old, respectively). The survey instrument took one to five minutes to complete. The responses of the 88 participants are summarized in Table 1.
A total of 75 physicians (85%), 39 of the signature group (87%) and 36 of the non-signature group (84%), expressed an intention to change. No significant association was observed between the groups and intentions to change (p = .68). An intention to change was expressed by 15 (94%) of the women physicians and 60 (83%) of the men, but again the difference was not statistically significant (p = .29).
Sixty-four of the participating physicians (73%) responded to the follow-up questionnaire. Response rates were virtually the same for the signature group and the non-signature group, 33 (73%) and 31 (72%), respectively. The women physicians were as likely to respond as were the men physicians, 12 (75%) and 52 (72%), respectively. The respondents did not significantly differ in mean age from those who responded originally, 41.6 versus 42.7 years old, respectively.
Immediately after the CME activity, 13 physicians expressed no intention to change. Two of the six (33%) physicians from the signature group and one of the seven (14%) physicians from the non-signature group expressed no intention to change immediately after the CME, but reported changes on follow-up. Differences in rates of change between these two groups were not significant.
In all, 41 of 75 physicians (55%) expressing intentions to change reported success on follow-up. There was no significant difference between the signature group and the non-signature group with respect to the numbers of physicians who changed, 21 (54%) versus 20 (56%), respectively; p = .99. Successful changes were reported by ten (67%) women physicians and 31 (52%) men physicians who had expressed intentions to change, a difference that was not statistically significant (p = .30). A significant association was also observed in the combined study groups between intention to change (yes, no) and changed behavior (change, no change), when 24 non-respondents—19 who intended to change and five who did not—were added to those who reported no change (p = .035). The relationship retained significance (p = .041) when the 24 non-respondents were omitted from the calculation.
Fifty-two physicians who committed to change indicated their levels of commitment and responded to the follow-up questionnaire (see Table 2). Most of these physicians (n = 32) had designated a level 4 commitment, of whom 24 (75%) indicated on follow-up that they had implemented the changes. Ten of the 14 physicians (71%) who had indicated the strongest commitment (level 5) reported successfully changing their practices. Five of six (83%) who had indicated moderate commitment to change (level 3) went on to make changes. One who had indicated a less-than-moderate commitment (level 2) reported a successful change. Again, there was no significant difference in these rates of change between signature and non-signature groups (p = .85).
We found no significant difference in the rates of change between physicians who signed a commitment to change and those who did not. If signature is not predictive of behavior change, what is?
Our results agree with those of prior research: Those who specify a change and indicate a commitment to implement it, with or without a signature, are more likely to succeed than are those who do not.2–4 One might reason that a physician's behavior can be expected to change if the specified change and commitment are consistent with the physician's beliefs and desires to perform, irrespective of a signature or designated level of commitment. For the learner, the very act of specifying an intended change may reveal something worth caring about, and seeking or ascribing intention tips the odds in favor of success for a planned or agreed behavioral change.
When using the commitment-to-change model, all follow-up mailings ordinarily are distributed within 90 days of physicians' designating their intents to change. During our study, more than half of the physicians who indicated that they intended to change but reported the changes had not been made also reported seeing no appropriate patients in that time. Especially illustrative of this point was one physician who described exercise-induced asthma as a seasonal problem in his practice. Optimal use of the commitment-to-change model requires that an intended change be made within the data-collection period. Continued improvement of the model could be facilitated by developing a system to account for categories, phases, or other increasingly sensitive measures of change.
A potential confounding factor for the data and a subject for further research is evident in the response of one physician who intended to change but did not. He expressed gratitude for the mailed reminder. Researchers using the commitment-to-change model expect that the follow-up packet will be a value-free data-collection instrument, but others may see it as an additional exposure to learning and a stimulus for change.
Regardless of whether commitment is followed by a signature, one must ask whether completing a questionnaire is a true pledge. Spending time to complete initial and follow-up questionnaires may, in and of itself, reflect a seriousness about the intent to change, which may, in turn, predict action. The form and context of agreement as well as the meaning of a signature should be studied. More exploratory work may better explain what triggers an intention and what enables physicians to follow their intentions through to successful implementations of change.
As the interests of organized medicine and the public converge around ways to assure the continued competence of practitioners, academic medicine must provide not only the educational activities to meet the accreditation, credit, and regulatory requirements influencing physicians, but also the research to guide effective learning and policy making. What are the effects of reporting requirements? A physician voluntarily executing a nonbinding learning contract at a local medical school may regard his or her signature in vastly different terms if the learning contract is to be shared with an organization overseeing medical relicensure or staff privileges. The physician who affixes an electronic signature to a knowledge self-assessment completed online may ascribe different meaning to the signature than when the same test of knowledge is completed, signed, and submitted in person or for specialty recertification. Finally, more objective measures must be applied to validate whether change in practice actually occurs as a result of any agreement or commitment to change.