Zinner, Darren E. PhD; DesRoches, Catherine M. DrPh; Bristol, Steffanie J.; Clarridge, Brian PhD; Campbell, Eric G. PhD
The current debate over industry's role in biomedical research within government and academic settings revolves around a central disagreement about the balance of the risks and benefits of these relationships. One stance advocates a near-total ban on relationships with industry, citing the need to curtail or eliminate all conflicts of interest in medicine. Proponents of this stance believe that the presence of any industrial relationships undermines the public's trust in the medical establishment, increases secrecy in science, and biases medical decision making toward company products and services.1,2 Opponents argue that industry relationships are simply one form of external scientific collaboration and have positive benefits for research and knowledge transfer. They believe any policies that restrict such relationships may isolate researchers and impede scientific advancements that may ultimately improve health outcomes and save lives.3
These opposing viewpoints were accentuated in 2004, when conflicts of interest within the intramural research program of the National Institutes of Health (NIH) became the subject of several newspaper investigations, federal inquiries, and congressional hearings.4–6 In response, then NIH Director Elias Zerhouni revamped the agency's ethics policies, establishing final rules in August 2005 that effectively banned most (compensated or uncompensated) employment, consulting, and advising by NIH employees to biopharmaceutical firms, supported research institutions, and health care insurers and providers. The agency formed this policy based on three guiding principles7:
1. The public must be assured that research decisions made at NIH are based on scientific evidence and not by inappropriate influences.
2. Senior management and people who play an important role in research decisions must meet a higher standard of disclosure and divestiture than people who are not decision makers.
3. To advance the science and stay on the cutting edge of research, NIH employees must be allowed interaction with professional associations, participation in public health activities, and genuine teaching opportunities.
These principles emphasize the central importance of the public's perception of government–industry relationships (GIRs) and conflicts of interest at the NIH for both scientists and administrators (herein referred to as “faculty” of the NIH for ease of writing). But are these principles contradictory? By requiring that all relationships be submitted for approval, the regulations have created additional bureaucratic hurdles and may have inadvertently deterred “good” external scientific relationships. In an effort to reduce conflicts of interest, have the rules made the agency more insular by harming the professional discourse between NIH intramural faculty and external peers?
Although the NIH conducted an internal survey of perceptions to the rules immediately after the introduction of these policies,8 there has been no published study in the peer-reviewed literature of the subsequent impact of the NIH policy, especially on behaviors of NIH research personnel. The main purpose of this study was to determine how the rules affect the patenting and publishing performance of NIH scientists, as well as their collaborations with industry, professional organizations, and academia. In addition, we sought to gauge perceptions of conflicts of interest and research integrity among NIH intramural faculty and to examine how the 2005 ethics rules affected those perceptions. Emulating the guiding principles above, we also examined the perceived effect on public trust.
We collected the data presented here from a survey we administered to a sample of NIH faculty between October 2008 and January 2009. We identified eligible participants from the central NIH directory and individual institutes' Web pages using criteria detailed below. These lists of potential participants were verified against individuals listed as principal investigators on intramural research over the previous two years on Computer Retrieval of Information on Scientific Projects, which is now called NIH RePORTER (projectreporter.nih.gov). Because the primary focus of the NIH ethics rules was to ensure that research decisions were made without undue conflicts of interest, we limited the sample to NIH research scientists and administrators. Consequently, we removed from these lists any individual who was not listed as an investigator, scientist, or senior staff member, as well as faculty employed within policy/planning, scientific review, extramural grants management, communications, or veterinary suboffices of the intramural program. This left 1,636 eligible faculty from 21 divisions of the NIH. Each individual was assigned a random number and then sorted in ascending order. From this list, the first 900 NIH scientists and administrators were selected to ensure a random, representative sample of all faculty in the intramural program at the NIH.
Survey design and administration
We used a modified version of a survey instrument administered to 3,080 life science academic faculty in 2007,9 that we adapted and expanded on the basis of 10 confidential personal interviews with current and former NIH faculty. In addition, experienced survey researchers pretested the new survey items using 11 cognitive interviews of NIH faculty that were not included in the final sample.
The survey focused on the faculty members' knowledge and perceptions of the ethics rules, as well as the perceived impact of those rules. It also included questions on their rates of publishing, patenting, and external affiliations before and after the implementation of the rules. Because the ethics rules pertain to all external relationships, we queried respondents regarding both industry relationships and academic/professional relationships. Respondents were asked about each relationship individually over two time periods; for example, one set of questions asked, “Before the ethics rules did you serve in any of the following roles for companies as an NIH employee?” Types of industry relationships included serving as a company founder, board of directors member, officer/executive, scientific advisory board member, consultant, equity holder, recipient of funding for students, recipient of royalties, or corporate research collaborator. Academic and professional relationships included journal editor, officer of a professional association, foundation board member, recipient of honoraria, or academic research collaborator. Questions regarding the faculty's opinion on the various impacts of the ethics rules were based on a five-point Likert scale (1 = large positive impact, 2 = small positive impact, 3 = no impact, 4 = small negative impact, 5 = large negative impact).
We asked respondents about all relationships, regardless of whether they were preapproved by NIH administration. The survey did not distinguish between compensated and uncompensated service on a scientific advisory board, nor did it distinguish between relationships that were approved or not approved by NIH. We did not ask respondents about gender, race/ethnicity, or worksite characteristics.
The Center for Survey Research at the University of Massachusetts, Boston conducted the survey by mail in a protocol identical to several previous projects.10,11 We conducted telephone follow-up with all individuals who did not return the separate postcard indicating participation. The phone calls had three main purposes: to make sure individuals had received the questionnaire and to correct addresses if they had not; to answer any questions an individual might have about the study; and to allow the individual to say he or she did not wish to participate simply by saying, “No.” Those who declined to participate were coded as refusals, and no follow-up contact was initiated. We mailed a final set of questionnaires to all those who had not returned the card and had not refused when reached by phone, with the aim of boosting the final response rate.
This study was approved by IRBs at both the Massachusetts General Hospital and the University of Massachusetts, Boston.
We divided the sampled individuals into three organizational categories: administrators, investigators, and staff scientists/clinicians, based on the answer to the question, “What is your organizational title(s)? (Please check all that apply.)” Possible answers were administrator (specify title), senior investigator, investigator, staff scientist, staff clinician, and other (specify title). The lead author (D.E.Z.) analyzed the data using standard statistical techniques. Bivariate analysis employed chi-squares for testing differences in simple proportions and analysis of variance for differences in means. Multivariate logistic analyses were used to examine differences in respective proportions and means when adjusting for multiple predictor and control variables.
Of the 900 faculty in our sample, 93 were ineligible because they were retired, no longer intramural faculty, too ill to participate, or deceased. Of the remaining 807 faculty members, 566 completed the survey, for an overall response rate of 70.1%.12 The sensitive nature of the study and the subsequent need for anonymity of the questionnaires prevents us from a more detailed analysis of nonrespondents. Although respondents were generally thorough and answered 97.8% of all possible questions, missing data from individually skipped questions slightly reduces the denominator of individual calculations and varies slightly across survey items.
Characteristics of respondents
Table 1 shows the characteristics of the 564 respondents (2 respondents chose not to answer questions about their professional characteristics). Of these faculty, 65 (11.5%) were administrators, 344 (61.0%) were investigators, and 155 (27.5%) were staff scientists or staff clinicians. About 95% (n = 541) were employed at the NIH before the introduction of new ethics rules in August 2005, and over half of all administrators and investigators had worked at the NIH for more than 15 years (P < .001 compared with scientists/clinicians). In terms of their educational background, 34% (n = 22) of administrators and 20% (n = 70) of investigators held MD degrees compared with 9.7% (n = 15) of staff scientists (P < .001).
GIRs and perceptions of research integrity
Of the faculty employed by the NIH since before the introduction of the ethics rules (n = 541), 280 (51.8%) had some form of GIR (formal or informal, paid or unpaid) related to their area of scientific expertise during their career at the agency (administrators: n = 42; investigators: n = 206; scientists/clinicians: n = 32; P < .05 for all comparisons). Of those with GIRs, 120 faculty members (42.9%) indicated that the industry relationship had contributed to their most important scientific work at the NIH.
Industry collaboration also demonstrated negative effects on relationships among NIH faculty (Table 2). Respondents indicated that in their experiences with other NIH scientists who had GIRs, 65 had concerns about the validity of the involved scientist's work, 49 had been unable to replicate published research, 32 had a research idea stolen, and 30 had refused to share information, data, or materials with the involved scientist. These proportions did not differ significantly by NIH title or by past record of industry involvement.
Perceptions of the impact of the ethics rules
Almost all respondents (n = 556; 98.2%) had at least some familiarity with the ethics rules. Of those, 444 (79.9%) believed the rules were “too restrictive” or “much too restrictive,” 91 (16.4%) felt they were “about right,” and 1 person (0.2%) felt they were “too loose” (16 respondents answered “don't know,” and 4 chose not to answer).
According to faculty, the NIH rules had a number of negative effects. Nearly half of faculty reported that the ethics rules had a negative impact on collaborations with industry (n = 316) and with academia (n = 266). About two-thirds (n = 364) cited a negative impact on their job satisfaction. Although a majority of faculty indicated that the ethics rules had no impact on their financial situation (n = 336), 210 respondents felt that the rules negatively affected their finances, and 152 were forced to divest from a financial holding because of the implementation of the ethics rules. At the same time, nearly half of all faculty respondents (n = 241) believed that the rules had a positive impact on the NIH's credibility with the public (Figure 1).
Faculty with GIRs were significantly more negative about the effects of the new ethics rules compared with their peers without GIRs. Faculty with GIRs were less likely to say that the ethics rules had a positive impact on public credibility (OR = 0.66 [0.47–0.94], P = .02) and more than twice as likely to believe that the rules made it harder for the NIH to complete its mission (OR = 2.63 [1.72–4.01], P < .001). Those with GIRs were more likely than those without to believe that the rules had negative effects on both industrial (OR = 4.71 [3.20–6.95], P < .001) and academic collaborations (OR = 1.43 [1.02–2.01], P = .037) and were more likely to report negative personal effects derived from the rules (ORs between 2.18 and 4.07 for questions regarding effects on individual faculty, P < .001 for all).
In general, there were no significant differences between the responses of administrators and investigators, with two exceptions. First, 91.8% of administrators (56 of 61 responses) believed that the ethics rules made it more difficult for the NIH to complete its mission, compared with only 79% of investigators (263 of 333 responses, P < .01). Second, a significantly larger proportion of administrators reported a greater effect on faculty recruitment, with 78% (46 of 59 responses) citing a negative impact versus 53% for investigators (178 of 335 responses, P < .01).
Changes in behavior stemming from the ethics rules
The ethics rules were associated with a significant decline in GIRs among NIH faculty employed both before and since the implementation of the rules (n = 541). As seen in Figure 2, the number of responding faculty who consulted to industry was reduced by about three-fourths (n = 179 before versus n = 42 after, P < .01), and the proportion of respondents serving on scientific advisory boards was cut in half (n = 170 versus n = 86, P < .01). Significant declines were also seen for NIH faculty serving as company founders or as members of a board of directors, as well as significant declines in the receipt of royalties and the prevalence of industrial funding for students. Among those with an existing GIR (n = 280), 101 faculty terminated an industrial collaboration, and 195 declined an honorarium from a company (results not shown in Table 2).
Figure 2 also shows that the ethics rules had little perceived impact on professional service roles of NIH faculty, except for a decline in the proportion of faculty serving in advisory roles for foundations (n = 171 before versus n = 133 after, P = .01). Although the ethics rules did not generally cause faculty to stop performing these activities, 134 of 280 faculty (47.9%) who serve as editors of a professional journal indicated that the new rules made it harder for them to fulfill this role. Similarly, 63 of 97 faculty (64.9%) who have been elected as officers in a professional association claim the rules make this role more difficult. Over one-third (n = 199) responded that they had limited attendance at scientific meetings because of the ethics rules.
To assess the impact of faculty involvement with NIH-supported research institutions, we asked about their relationships with academic colleagues. Per the NIH rules, NIH faculty were no longer able to accept honoraria or speaking fees from universities or academic medical centers that receive support from the NIH. NIH faculty may have been forced to end collaborations with academia if they were paid as consultants or the principal investigator had an irresolvable conflict of interest. Alternatively, NIH faculty may have ended a collaboration because the burden of administrative rules overwhelmed the benefit of the partnership. In comparison with the numbers for industrial collaborations above, 54 faculty terminated an academic collaboration and 368 declined an honorarium from an academic institution.
Faculty with GIRs averaged a significantly higher rate of publication than their peers who did not have GIRs (6.7 versus 3.8 articles per year, P < .01, data not shown in tables). There was no change in publishing behavior of NIH faculty before and after the implementation of the ethics rules (5.29 articles per researcher per year from 2002 to 2005 versus 5.26 articles per year from 2005 to 2008; P = .88). A differences-in-differences regression failed to detect any change in publishing productivity between faculty with and without previous GIRs (P = .25). Similarly, there was no statistical difference in the proportion of faculty applying for a patent before and after the policy change (P = .39).
Beliefs on the repeal or extension of the ethics rules
Beliefs among surveyed NIH intramural faculty (n = 566) about the potential effects of relaxing the ethics rules were mixed. As Table 2 shows, 177 respondents believed that a repeal of the ethics rules would increase bias in NIH science, and 236 felt it would lead to more secrecy among faculty. Whereas 167 staff members (30%) indicated that such repeal would be likely to overemphasize applied investigations, 468 believed it would lead to new research that would not otherwise be possible. When asked whether the ethics rules should be extended to academic scientists to reduce conflicts of interest, the majority of faculty believe the rules should apply to academic scientists who are NIH grantees (n = 315), and nearly half felt they should apply to all scientists in academic institutions (n = 252); these views about extending the NIH ethics rules did not differ significantly by title or prior relationship with industry.
The 2005 ethics rules implemented by the NIH represented a radical shift in managing industry–research relationships. Unlike open-disclosure practices commonly used to manage industry relationships in health care arenas such as physician practice patterns and continuing medical education, the NIH rules presume that all industrial collaborations are inappropriate unless proven otherwise and require advanced permission before they begin. As such, the policy fundamentally modifies the calculus by which the balance of risks and benefits is measured in research relationships.
This report makes three important contributions to this debate. First, this study assesses the perception of research integrity among NIH faculty. The overall results suggest a strong association between industrial research relationships and perceptions of conflicts of interest. Approximately 1 in 10 faculty questioned the validity of research as a result of a peer faculty's involvement with industry, and nearly one-third believe that relaxing the ethics rules would lead to increased bias and secrecy or place an undue emphasis on applied research.
The second major contribution of this study is documenting the prevalence of GIRs at the NIH, before and after the implementation of the 2005 ethics rules. Although previous studies have shown the pervasiveness of corporate affiliations within academia, this is the first empirical study to document a strikingly similar proportion of GIRs among NIH faculty. A comparable survey of 3,080 academics from the top 50 grant-receiving institutions in 2006–2007 suggested that 52.8% of academic research faculty maintained some form of relationship with industry, whereas the data presented above show that 51.8% of all NIH research faculty were linked with industry before the introduction of 2005 ethics rules.9 Our study further documents that even in the government research sector, the most productive scientists are most likely to have relationships with industry; NIH faculty linked to industry published at nearly twice the rate of their peers (6.7 versus 3.8 articles per year), a ratio that is nearly identical to that seen in academia.9 Unfortunately, the cross-sectional nature of this study does not allow us to test whether industry relationships make scientists more productive or whether industry seeks out the most productive scientists with whom to collaborate.
Third, this study helps estimate the impact of the 2005 ethics rules. In general, the policy seems to have accomplished much of what it was intended to do: limit relationships with pharmaceutical, medical device, and related organizations while maintaining the researchers' association with non-industry-related external scientific and professional organizations. The rules did not eliminate GIRs—one-third of all NIH faculty still work with industry in some capacity (and it is important to note that this does not imply these faculty are in violation of federal regulations). This large remainder may reflect the agency's recognition of the importance of limited external partnerships to clinical development and technology transfer. But the proportion of faculty who consulted was reduced by three-fourths, and the fraction serving on scientific advisory boards was cut in half. Simultaneously, there was no significant drop in the researchers' participation in professional service roles. Our findings suggest that a ban on industry relationships can be achieved without harming other important relationships—at least in the governmental setting.
However, the 2005 ethics rules evoked several unintended consequences that may lead to negative long-term implications for the NIH. Most appreciably, they have substantially reduced personnel morale, job satisfaction, and the progress of faculty research. Over 80% of all faculty believe the new rules are too restrictive, including 72.7% of those without prior industry relationships.13 Although documented instances of recruiting and retention problems are limited,14 we found that NIH personnel, especially administrators, believe the rules are hampering successful recruitment of new faculty to the agency, presenting potential future workforce challenges. This may become even more important as the current generation of aging researchers nears retirement.15
Although the new rules did not eliminate professional and scientific affiliations, faculty believe that the rules have strained these connections. A large proportion of NIH faculty have terminated academic collaborations, limited attendance at scientific meetings, and reported that serving in professional roles has become more difficult. As indicated in the guiding principles of the ethics rules, this decreased interaction may hinder the ability of NIH faculty to stay on the forefront of pertinent scientific advancements.
Further, it is important to consider whether the ethics rules have limited valuable lines of new research and clinical development. Of the faculty who had industrial relationships, nearly half claimed their past industry relationships contributed to their most important work at the NIH. The vast majority, including those without industry affiliations, believe that relaxing the conflict-of-interest rules would encourage new types of research. However, our analysis of average publication rates before and after the implementation of the ethics rules did not detect a significant decline in scientists' productivity. Additional research is needed to more fully understand the long-term effects of the 2005 rules on the nature, productivity, and impact of NIH intramural research.
Several limitations should be noted. As with all survey research, this study suffers from weaknesses inherent with self-administered questionnaires. Respondents may have underreported socially undesirable relationships or behaviors, and their perceptions of whether or not their colleagues have industry relationships may not have been accurate. Further, the study sample consisted of currently employed intramural faculty and the results are not directly applicable outside this population. To the extent that NIH Web sites were not updated regularly, the sample may have underrepresented new faculty and excluded any scientist who left NIH because of the ethics rules. Lastly, it is important to emphasize that interim conflict-of-interest policies were discussed and promulgated within the NIH before enacting the final ethics rules in August 2005.
The issue of industry relationships and conflicts of interest has become increasingly salient for universities and academic medical centers, especially in light of proposed federal regulations for NIH extramural grantees.16 These institutions may look to the NIH experience for guidance as they develop relationships with industry and disclosure policies. Although the parallels to academic and other research settings are considerable, differences do remain. Public perceptions of credibility may be more important at the NIH because it is funded through direct public investment. On the other hand, NIH faculty are not required to continually find and renew grants to support their salaries like many academics, potentially making them less susceptible to industry influences. Our data suggest that the NIH faculty themselves are evenly split in their own assessments of the applicability of these rules in a wider setting.
Overall, our results further underscore the delicate balance of the benefits and the risks of industry relationships with researchers. Our findings suggest that the same policy that increased faculty's perceptions of public credibility of the NIH also made it more difficult for the organization to complete its mission. The success of more stringent conflict-of-interest policies will center on their ability to mitigate the negative aspects of industry relationships while maintaining incentives for collaboration and scientific development. These policies must consider both the intended and unintended consequences of industry research collaborations.
The authors would like to acknowledge the contributions of Sandra Feibelmann, MPH, of the Department of General Medicine at Massachusetts General Hospital, for her paid research assistance in assembling the survey sample.
This research was supported by a grant from Office of Research Integrity through the National Institute of General Medical Sciences of the National Institutes of Health (5R01GM0844455-01, PI Eric G. Campbell).
Dr. Zinner is partially supported by Brandeis University's Health Industry Forum, an independent policy consortium of major health care insurers, delivery systems, and biopharmaceutical and medical device firms. No other disclosures are reported.
This study was approved by institutional review boards at both the Massachusetts General Hospital and the University of Massachusetts, Boston.