The UCSF Office of CME's risk score for commercial influence was not associated with perceived bias, as we found no significant difference in perceived bias between CME activities classified as lowest, intermediate, and highest risk (P = .63). Similarly, level of commercial support was not associated with perceived bias (P = .90). With the exception of the course director being a first-time chair, none of the individual risk factors we assessed were associated with perceived bias (Table 3). Simultaneous inclusion of all of these variables in a single model yielded limited results, with 3% of the total variance in perceived bias between activities explained by these risk factors.
Next, we evaluated several other event characteristics which we did not a priori hypothesize would be associated with perceived bias. Number of registrants was not associated with perceived bias (P = .46). However, the response rate to course evaluations was associated with perceived bias (P < .001), with an adjusted mean “bias-free” rating of 95.8% in activities with below-median response rates versus 96.9% in those with above-median response rates. There was also a statistical trend for activities conducted during later years of the study to be considered less free of bias (adjusted mean “bias-free” ratings of 97.1% for activities in 2005 versus 96.1% for activities in 2006 and 2007, P = .08). Together, number of registrants, response rate, and course year explained 11% of the total variance in perceived bias between activities. When all other predictor variables were added to the model, the total explained variance was 15%.
Finally, we evaluated whether potential risk factors for commercial bias were associated with the overall quality of the activity. As with perceived bias, neither the risk score nor level of commercial support was associated with the overall quality of the activity. Similarly, the only variable significantly associated with overall quality was the presence of a first-time course chair (mean Likert score 4.23 for activities with first-time chair versus 4.40 for activities without first-time chair, P = .01). When all risk factors used in the risk score were included in a single model, 9% of the variance in overall quality score was explained by these risk factors.
In this study of 213 educational programs organized through an academic provider of CME, the vast majority of CME activities were perceived by participants to be free of commercial bias. The extent of commercial support, a variety of other event characteristics, and a summative risk index were not associated with the level of perceived bias or the perceived overall quality of CME activities. Stated otherwise, rates of perceived commercial bias were consistently low regardless of the presence or absence of risk factors for commercial bias.
There is a paucity of other research that has directly evaluated the impact of commercial support on the content of CME.6 However, other studies of bias in CME and on relationships between industry and medicine can shed light on potential explanations for our findings. The most direct interpretation of our results is that CME activities in general are free of commercial bias. Under rules of the ACCME, all accredited providers of CME must abide by the Council's Standards of Commercial Support. These standards require that accredited activities be balanced and that conflicts of interest be disclosed and managed.8,9 However, these regulations cannot prevent all forms of commercial influence, and experts in drug industry promotion have identified widespread opportunities for commercial influence in CME and have documented industry marketing strategies that rely heavily on clinician education to boost drug sales.10–20 Other reports have documented that many physicians believe that industry-supported CME is biased.13,21,22 Finally, many commentators have noted that it is unlikely that industry would expend $1.2 billion per year to support CME if it did not help companies' bottom lines.1–3,11,23,24 Thus, although one interpretation of our findings is that industry support does not lead to bias in CME, other interpretations are possible as well.
One such alternate explanation is that the screening process instituted by the UCSF Office of CME successfully rejected activities with commercial bias that might have been permitted by other CME providers. This screening process is substantially more rigorous than is prescribed in the ACCME's Standards of Commercial Support. For example, although doing so is not required by the standards, the UCSF Office of CME rejects proposals with a single industry supporter or those with no cost to the participant. This screening process may have weeded out CME activities at higher risk of commercial bias, leaving only activities in which commercial bias was more subtle or entirely absent. Furthermore, activities that were allowed to proceed but flagged as having intermediate or higher risk of bias were subject to mitigation procedures before and/or during the activity.
Finally, our findings of low rates of perceived bias may in part be explained by the insensitivity of the simple “yes/no” question used to assess learners' perceptions of bias. Given that commercial influence is often subtle, a single binary question may fail to fully capture the range of learner perceptions of commercial influence.25 Moreover, learner perceptions of bias may not precisely correspond to actual bias in CME activities. It is possible that some suspicious observers may have perceived bias where, in fact, there was none. However, a wealth of literature in the medical and social sciences suggests that physicians (and people in general) often fail to perceive bias, particularly when these biases are subtle.26–28
Prior work investigating mechanisms of bias in CME has highlighted the importance of subtle, often subconscious and unintentional forms of bias that may be difficult for screening processes to identify and for learners to perceive.1,15,29 Course organizers may focus course curricula more on drug or device-based therapies and less on lifestyle-based treatments, or devote special attention to one aspect of disease management for which a supporter's drug (or class of drugs) may be commonly used.30 In addition, the medical literature from which evidence-based recommendations are made is subject to commercial influence, as noted in studies which have shown strong and consistent associations between industry support of clinical trials and the outcomes and interpretation of those trials.31,32 Speakers may also play an important role in introducing subtle biases, for example, through their choice of topics and presentation of course material.
In each of these cases, bias is introduced not through a direct quid pro quo but through subconscious attitudes and feelings of reciprocity that can arise when a speaker or course director has benefited from industry, for example, through funding of an educational or research grant, service on a speakers' bureau, or receipt of personal educational materials, food, or samples for one's clinic. Because attitudes and feelings of reciprocity largely occur on a subconscious level, physicians often fail to recognize the ways in which they might be influenced and overestimate their ability to resist commercial influence.23,26 Of note, in 2007 only 20% of speakers at UCSF CME activities reported conflicts of interest, such as grant funding or speakers' bureau honoraria relevant to their talks; however, this number may be substantially higher elsewhere.
To the extent that influence on the content of CME can often be subtle and occur without the conscious knowledge of course organizers and speakers, bias in CME can be difficult to detect even for the watchful observer.23,33 In addition, in many cases it can be challenging, if not impossible, to determine whether a speaker is emphasizing a certain topic or recommending a certain treatment strategy because of subtle commercial influence on that speaker or because that speaker has a truly independent, scientifically valid opinion that reaches the same conclusion.22 Paradoxically, CME activities in which commercial influence is subtle are likely to be more effective at changing participants' attitudes and behaviors than activities with overt influence, because physicians are more receptive to information that seems objective and may reject information they perceive to be promotional.30
Given the difficulty of disentangling unvarnished educational goals and scientific opinion from subtle forms of commercial bias, there have been increasing calls to fundamentally change the relationship between the health care industry and CME providers. In a seminal article, Brennan et al29 recommended eliminating industry support of individual CME activities at academic medical centers and instead suggested a mechanism in which industry donations would be pooled into a central account that would be used to help fund individual CME activities without industry input. Several universities have adopted this model, with a handful forsaking all industry funding of their CME programs.34 A recent conference commissioned by the Josiah Macy Foundation went a step further, recommending the complete elimination of industry support of accredited CME.16 Not surprisingly, these recommendations are controversial, in part because many CME providers are financially dependent on industry, and because physicians are concerned about increases in course registration fees if industry support were to be substantially reduced or withdrawn.4,13,21,35
Our results could be interpreted as either supporting or contradicting these recommendations. On the one hand, one might conclude that the safeguards offered by the UCSF Office of CME's screening practices resulted in a series of CME programs with little if any bias. On the other hand, our results are consistent with the observation that commercial bias is difficult to detect, both by organizers and recipients of CME, and that the only way to safely guard against industry influence is to eliminate or limit industry involvement in CME. To resolve this and other questions, more research is needed to systematically investigate the presence and impact of bias in CME, including in-depth explorations of course content and learner perceptions of bias. Of note, definitive research will likely require large sample sizes and sophisticated measures, making it unlikely that a substantial body of new research will be available in the near future. Until then, policy makers will need to carefully interpret existing data and apply lessons learned from research on other types of interactions between physicians and industry to help inform their judgments.
Our study has several limitations. As noted earlier, our sample of CME activities were organized by a single institution with a distinct set of criteria for vetting proposed activities. It is thus unclear to what extent our findings would be generalizable to other academic and nonacademic providers of CME with less stringent review criteria and a different set of administrative procedures and institutional culture. In addition, as noted above, the question we used to determine perceptions of bias was limited to a binary yes/no response, preventing us from capturing subtlety in participant responses. Third, our study evaluated only activities which had passed an initial screening process in which proposals deemed to have an unacceptably high risk of commercial influence were rejected (whereas such activities might be approved by other accredited providers of CME). As a result, our study design prevents us from assessing perceived bias in activities believed to carry the highest risk of industry influence. Finally, our study evaluated a moderate but limited number of CME activities. Nonetheless, it is unlikely that a larger sample of activities would have changed our conclusions, because even if the observed differences in perceived bias between lower- versus higher-risk activities were statistically significant, they would not be meaningfully different.
In summary, in this study of 213 CME activities, rates of perceived commercial bias were very low, with no differences in perceived bias between activities with and without potential risk factors for commercial influence. These findings suggest that rigorous review criteria effectively screened out activities with explicit commercial bias. Further research will be needed to evaluate the presence of subtle forms of commercial bias and risk factors to predict these biases in CME.
The authors thank John Boscardin, PhD, and Kathy Fung, MS, for their help with the statistical analyses.
Funding/Support: Funding for this work was provided by VA Health Services Research and Development Service (CDTA 01-013) and by the National Institute on Aging (K23 AG030999) (both for Dr. Steinman).
Other disclosures: None.
Ethical approval: This study was exempted from review by the Committee on Human Research at the University of California, San Francisco and by the Research and Development Committee at the San Francisco VA Medical Center.
Disclaimer: Dr. Steinman had full access to all data for this study and takes full responsibility for the results. Dr. Steinman served as an unpaid expert witness for the plaintiff in United States ex. rel Franklin versus Pfizer, Inc., litigation which alleged that Pfizer and Parke-David illegally marketed gabapentin (Neurontin) for uses not approved by the FDA, including the use of medical education for marketing purposes. Dr. Steinman also helped to found an online archive of drug industry marketing documents from this and other litigation, including soliciting a gift of start-up funds from the lead lawyer in the gabapentin litigation. In addition, Dr. Steinman contributed unpaid efforts to an educational grant funded with money from an Attorney General Settlement Fund created through an out-of-court settlement of this litigation.
1Steinbrook R. Financial support of continuing medical education. JAMA. 2008;299:1060–1062.
3Steinman MA, Baron RB. Is continuing medical education a drug-promotion tool?: YES. Can Fam Physician. 2007;53:1650–1657.
4Relman AS. Industry support of medical education. JAMA. 2008;300:1071–1073.
5Barnes BE, Cole JG, King CT, et al. A risk stratification tool to assess commercial influences on continuing medical education. J Contin Educ Health Prof. 2007;27:234–240.
7McCullaugh P, Nelder JA. Generalized Linear Models. 2nd ed. London, UK: Chapman and Hall; 1989.
12Relman AS. Separating continuing medical education from pharmaceutical marketing. JAMA. 2001;285:2009–2012.
13Mueller PS, Hook CC, Litin SC. Physician preferences and attitudes regarding industry support of CME programs. Am J Med. 2007;120:281–285.
14Smith JL, Cervero RM, Valentine T. Impact of commercial support on continuing pharmacy education. J Contin Educ Health Prof. 2006;26:302–312.
15Association of American Medical Colleges. Industry Funding of Medical Education: Report of an AAMC Task Force. Washington, DC: Association of American Medical Colleges; June 2008.
16Fletcher SW. Chairman's Summary of the Conference: Continuing Education in the Health Professions: Improving Healthcare Through Lifelong Learning. New York, NY: Josiah Macy, Jr. Foundation; 2008.
17Carlat D. Diagnosis: Conflict of interest. New York Times. June 13, 2007.
18Steinman MA, Bero LA, Chren MM, Landefeld CS. Narrative review: The promotion of gabapentin: An analysis of internal industry documents. Ann Intern Med. 2006;145:284–293.
19Fugh-Berman A, Ahari S. Following the script: How drug reps make friends and influence doctors. PLoS Med. 2007;4:e150.
20Bowman MA, Pearle DL. Changes in drug prescribing patterns related to commercial company funding of continuing medical education. J Contin Educ Health Prof. 1988;8:13–20.
21Rutledge P, Crookes D, McKinstry B, Maxwell SR. Do doctors rely on pharmaceutical industry funding to attend conferences and do they perceive that this creates a bias in their drug selection? Results from a questionnaire survey. Pharmacoepidemiol Drug Saf. 2003;12:663–667.
22Cornish JK, Leist JC. What constitutes commercial bias compared with the personal opinion of experts? J Contin Educ Health Prof. 2006;26:161–167.
23Association of American Medical Colleges. The Scientific Basis of Influence and Reciprocity: A Symposium. Washington, DC: Association of American Medical Colleges; June 12, 2007.
24Casebeer LL, Shillman RS. What's ROI got to do with CME? Med Mark Media. 2002:101–105.
25Takhar J, Dixon D, Donahue J, et al. Developing an instrument to measure bias in CME. J Contin Educ Health Prof. 2007;27:118–123.
26Pronin E. Perception and misperception of bias in human judgment. Trends Cogn Sci. 2007;11:37–43.
27Chren MM, Landefeld CS, Murray TH. Doctors, drug companies, and gifts. JAMA. 1989;262:3448–3451.
28Dana J, Loewenstein G. A social science perspective on gifts to physicians from industry. JAMA. 2003;290:252–255.
29Brennan TA, Rothman DJ, Blank L, et al. Health industry practices that create conflicts of interest: A policy proposal for academic medical centers. JAMA. 2006;295:429–433.
30Fugh-Berman A, Batt S. “This may sting a bit”: Cutting CME's ties to pharma. Virtual Mentor. 2006;8:412–415.
31Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: Systematic review. BMJ. 2003;326:1167–1170.
32Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: A reflection of treatment effect or adverse events? JAMA. 2003;290:921–928.
33Avorn J, Chen M, Hartley R. Scientific versus commercial sources of influence on the prescribing behavior of physicians. Am J Med. 1982;73:4–8.
© 2010 Association of American Medical Colleges
34Harris G. Stanford to limit drug maker financing. New York Times. August 26, 2008.