Resnik, David B. JD, PhD; Dinse, Gregg E. ScD
Integrity is an essential aspect of the conduct of research in biomedicine and other scientific disciplines. Publication, peer review, collaboration, mentoring, data sharing, research with human and animal subjects, and other scientific activities rely on adherence to ethical standards. Unethical behavior may have detrimental consequences for science and for society, such as corruption of the research record, harms to public health or the environment resulting from regulatory or policy decisions based on faulty evidence, erosion of public support for research, and even threats to national security.1
In the late 1980s and early 1990s, a growing awareness of ethical problems, such as data fabrication and falsification, plagiarism, and conflicts of interest, led U.S. government agencies to take steps to promote integrity in federally funded scientific research. These efforts included formulating policies on research misconduct and conflict of interest, investigating cases of misconduct, and promoting research, education, and training in the responsible conduct of research (RCR).2
In 1989, the National Institutes of Health (NIH) implemented requirements for instruction in RCR for graduate students supported by NIH training grants. The NIH has revised these mandates several times since then and now requires that all students, trainees, fellows, and scholars supported by training grants, career development awards, research education grants, or dissertation research grants receive RCR training through the institutions with which they are affiliated.3 The NIH strongly encourages in-person instruction, recommending “a combination of didactic and small-group discussions,” which can be complemented by online training.3 (Per NIH policy, online-only training is not acceptable.) To meet NIH requirements, RCR training programs should include at least eight contact-hours of instruction in specific core content areas, including misconduct, data management, authorship, mentoring, collaboration, publication, peer review, and social responsibility.3 In 2001, the NIH extended its RCR training requirements to cover all intramural scientists, trainees, students, and staff who are substantially involved in research. These individuals must participate in initial online training and annual, in-person updates.4
Recently, the scope of federal RCR training mandates expanded to include research funded by the National Science Foundation (NSF). In 2007, the U.S. Congress passed the America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act (the America COMPETES Act). Section 7009 of the act requires that any institution receiving NSF funds must have a plan to provide training on the responsible and ethical conduct of research to undergraduate students, graduate students, and postdoctoral researchers supported by those funds.5 The NSF implemented Section 7009 in August 2009, requiring institutions to certify in their funding applications that they have such a plan and, if funding is received, to verify that the appropriate students and postdoctoral researchers receive RCR instruction.6 The NSF does not specify the content or format of the training.
Now that two major federal research funding organizations require instruction in RCR for funded students and trainees, it is important to understand whether U.S. research institutions are meeting—or exceeding—these mandates. This is a crucial issue: Many experts agree that extensive institutional support for RCR education throughout the undergraduate and graduate curricula is necessary to promote integrity in research and to build an ethical culture within an institution.7–9 To examine this, we surveyed top-funded research institutions to capture a snapshot of current RCR training programs. We determined whether RCR training requirements go beyond those of the NIH and NSF and, if so, whom the institutions require to receive RCR training. We also explored whether the types of individuals required to receive RCR training differed between institutions with and without medical schools.
During May to June 2011, we sent a brief e-mail survey to officials responsible for overseeing RCR training at the 200 top-funded research institutions in the United States and Puerto Rico. We compiled the list of institutions using the latest data then available (from 2007) from the Center for Measuring University Performance (CMUP) at Arizona State University, a program that collects and analyzes data on U.S. universities.10 We searched these institutions’ Web sites using the terms responsible conduct of research, research integrity, research ethics, and research compliance until we found an appropriate contact person. In some cases, the Web site identified the person responsible for overseeing RCR training; in other cases, we were referred to that person by the individual we initially contacted. People responsible for overseeing RCR training included faculty members, research compliance officials, research integrity officers, and high-level administrators (e.g., vice president for research).
We kept the survey brief to help promote a high response rate, and asked recipients to reply by e-mail. Our survey invitation assured respondents that their responses would be kept confidential and that we would not publish responses identifying specific institutions, although we might publish a list of responding institutions (available from the authors). The survey consisted of the following five questions:
1. Does the institution have an RCR training program?
2. Who is required to take RCR training?
3. Is training centralized (e.g., handled by the office of research or graduate school), decentralized (e.g., handled by various colleges or departments), or both?
4. Is training online, in-person (e.g., classes or workshops), or both?
5. If training is available online, which online resource is used (e.g., the Collaborative Institutional Training Initiative or CITI)?
Regarding the last question, CITI is an online ethics training program, operated by the University of Miami, which provides training in RCR as well as in the protection of human and animal subjects and biosafety.11
We sent a reminder e-mail after one week if no response was received, and we sent a second reminder after two weeks. In the few cases in which more than one individual at an institution responded to our request for information, we combined the responses and resolved minor discrepancies by giving preference to the response from the institutional official most directly responsible for overseeing the training program.
We obtained information on each institution’s funding (amount and ranking) and control (private versus public) from the CMUP. We classified each institution as having or not having a medical school and as belonging to one of four geographic regions according to U.S. census categories (i.e., Northeast, South, Midwest, and West). We counted an institution as having a medical school if it was a university with an associated medical school (e.g., Johns Hopkins University) or it was a medical school or an academic medical center listed separately by the CMUP (e.g., Baylor College of Medicine). We did not combine data for universities and associated medical schools that were listed separately by the CMUP.
The NIH Office of Human Subjects Research Protections determined that the federal human research protections regulations did not apply to this study because the study did not gather private information on human subjects.
Data coding and analysis
The first author (D.B.R.) coded the survey responses twice. Initially, he coded the data while developing the coding framework. Later, he coded the data a second time (in a blinded fashion), resolved differences between the codings by comparing them with the e-mail responses, and made further refinements to the coding framework.
We used the answers we received to the open-ended survey question “Who is required to take RCR training?” to develop the following nine dichotomous (yes/no) categories for classifying responses: (1) only individuals mandated by federal requirements (NIH/NSF), (2) all graduate students, (3) all doctoral-level students, (4) all students in selected programs (e.g., engineering, biomedicine), (5) all postdoctoral students or fellows, (6) all students participating in externally funded research, (7) all faculty/staff participating in externally funded research, (8) all faculty/staff participating in human subjects research, and (9) all faculty/staff participating in animal research. We used categories 2 to 9 to describe additional types of individuals some institutions required to take RCR training; those institutions also required training for persons mandated by the NIH and/or NSF.
We calculated the proportion of institutions with an RCR training program and the proportion that required individuals in each category to be trained in RCR. We also calculated the proportions of institutions with and without medical schools that required each category of individuals to receive RCR training. Finally, we tabulated the proportion of institutions with certain training types (e.g., centralized training, in-person training, online training, online training using CITI).
We used SAS version 9.2 (SAS Institute, Cary, North Carolina, USA) to perform statistical analyses. For each of the nine dichotomies, we applied Fisher’s exact test to assess whether the proportion of institutions with those training requirements differed according to whether an institution had a medical school and whether an institution was privately or publicly controlled. We also applied a two-sample t test, separately for each dichotomy, to assess whether institutions that did or did not require certain individuals to receive training differed with respect to their research funding. We checked the results obtained from the Fisher exact tests and t tests by using multiple logistic regression to simultaneously assess the effects of all explanatory factors considered together, as some factors may be correlated with others. All tests were performed at the α = .05 significance level.
We received responses from individuals responsible for RCR training at 144 (72.0%) of the 200 institutions we contacted. These responses indicated that all 144 institutions (100%) had an RCR training program.
This suggestion of complete RCR training compliance is reasonable if the 144 responding institutions are representative of all 200 institutions from which we solicited information. However, if nonrespondents were less likely than respondents to have an RCR program, we would expect the proportion of institutions offering RCR training to be lower, possibly as low as 72% (144/200). To investigate this, we compared the known institutional factors of survey respondents and nonrespondents (data not shown). We found no statistically significant differences in medical school status, private/public status, or geographic region; however, respondents’ average funding level was higher than nonrespondents’ ($253 million versus $183 million; P = .017). We do not think that funding differences biased our results, and the remaining analyses restrict attention to the 144 responding institutions.
Among the 144 responding institutions, the average amount of research funding was $253 million, with a standard deviation of $233 million. The median funding was $186 million, with a range of $36 million to more than $1.5 billion. Eighty-five (59.0%) of the responding institutions had a medical school, and 110 (76.4%) were publicly controlled (Table 1). Ninety (62.5%) of the institutions offered both centralized and decentralized RCR training, and 121 (84.0%) offered a mix of in-person and online training. Of the 129 (89.6%) that offered online training, 95 (73.6%) used only CITI for that training. Table 1 provides additional details on training as well as geographic distribution.
RCR training requirements varied substantially across the 144 responding institutions: 69 (47.9%) of the institutions required RCR training only for individuals covered by federal mandates, whereas the other 75 (52.1%) required training for additional individuals as well. Institutions’ additional training requirements varied according to the category of individuals, ranging from a high of 34 (23.6%) requiring instruction for all students in selected programs (e.g., engineering, biomedicine) to a low of 3 (2.1%) requiring instruction for all faculty/staff participating in animal research. Table 2 provides a breakdown of requirements for all nine categories.
Training requirements differed according to medical school status: 32 (37.7%) of the 85 institutions with a medical school and 37 (62.7%) of the 59 institutions without a medical school required RCR training only for individuals covered by federal mandates (Table 3). Thus, the proportion of institutions with requirements that went beyond the NIH/NSF mandates was significantly higher among those with a medical school than among those without (P = .004). Institutions with a medical school were significantly more likely than those without to require RCR training for all doctoral-level students (P = .042), all students in selected programs (P = .027), and all faculty/staff participating in human subjects research (P = .042). RCR training requirements of privately and publicly controlled institutions did not differ significantly for any of the nine dichotomies, nor were there any significant differences in funding level (or rank) between the two categories associated with any of the nine training requirement dichotomies (data not shown). These results regarding medical school status, institutional control, and funding were based on applying t tests and Fisher exact tests to each of those factors individually, but we reached the same conclusions when using logistic regression to assess all three factors simultaneously whenever such an analysis was possible (data not shown). The fact that no institutions without medical schools required RCR training of all doctoral-level students, all faculty/staff participating in human subjects research, or all faculty/staff participating in animal research prevented the use of logistic regression analysis for these three dichotomies.
The most important finding of this study is that more than half of the research institutions in our sample required RCR training for individuals not mandated by the NIH or the NSF to receive such training. Although this is encouraging, perhaps more institutions should consider going beyond the federal mandates. In our opinion, requiring RCR training only for students and trainees supported by NIH or NSF grants may send a mixed message: If students are doing the same type of research, why should only those supported by such a grant be required to receive RCR training? We believe it would make more sense for institutions to broaden their training requirements to send a clear signal to students and faculty concerning the importance of the ethical conduct of research. There is also a practical advantage to going beyond federal mandates: It may be easier to track compliance by requiring all students and trainees of certain types to take RCR training rather than determining on a case-by-case basis whether particular individuals are funded by an NIH or NSF grant and therefore need instruction in RCR.
Another important finding is that a significantly higher proportion of institutions with a medical school than those without had RCR training requirements that went beyond the federal mandates. A possible explanation for this difference is that institutions with a medical school may have more experience in dealing with RCR training requirements. They may be more likely to have funding from the NIH, which has required RCR training since 1989; the NSF only started to require such training in 2009. Over the years, institutions with medical schools may have decided to expand their training requirements to include entire classifications of students: Our findings suggest that a higher proportion of these institutions than of those without a medical school require RCR training for students in specific programs (e.g., biomedicine).
An interesting finding is that there was substantial variation with regard to expanded RCR training requirements, suggesting that institutions’ philosophies about RCR instruction differ: Some focus on graduate or postdoctoral students, whereas others target students in specific programs or even faculty/staff involved in specific types of research (e.g., human subjects or animal research). This variation may change over time as institutions expand their RCR training programs and learn from their experiences with RCR instruction.
Our study had several limitations. We did not ask in-depth questions concerning the content of RCR programs, the qualifications of the instructors, the use of case studies or videos, difficulties with implementing RCR programs (e.g., funding or staffing needs), institutional rationales for implementing additional training requirements, the evolution of institutional policies in response to federal mandates, or trainees’ perceptions of RCR instruction. Although collecting such information would be useful, our goal was to take a snapshot of current RCR instructional programs in the United States, which we accomplished by designing a brief survey to promote a high response rate. Our analysis was exploratory and descriptive; we did not attempt to formally evaluate RCR programs to determine how well they were meeting federal mandates. In the future, other researchers may be interested in conducting more detailed, follow-up surveys.
In addition, respondents at some of the institutions may not have provided accurate or truthful responses, out of fear that noncompliance might be reported to the NIH or NSF. For example, respondents at institutions receiving NIH funding may have been hesitant to admit that only online training was offered, because the NIH requires in-person training. Although the veracity of responses is a potential limitation, we do not think it is a significant one. First, we assured the individuals we contacted that their responses would be confidential and that we would not publish data pertaining to specific institutions. Second, many of the individuals we contacted indicated that they would be interested in the results of our study because they believed the information could be useful in further developing their RCR programs. This suggests that respondents were motivated to give accurate and truthful answers.
In conclusion, more than half of the research institutions responding to our survey have chosen to go beyond the federal mandates and require RCR training for individuals not specifically mandated by NIH or NSF to receive such instruction. We found that institutions with a medical school were more likely than those without to have training requirements that exceed federal mandates. Although these findings are encouraging, we believe that more research institutions should broaden their RCR training requirements to promote integrity in education and research.
Acknowledgments: The authors wish to thank Grace Kissling, Bill Schrader, and Bruce Androphy for helpful comments.
Funding/Support: This article is the work product of a group of employees of the National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH).
Other disclosures: None.
Ethical approval: Not applicable: This research did not involve human subjects, as determined by the NIH Office of Human Subjects Research Protections.
Disclaimer: The statements, opinions, or conclusions contained herein do not necessarily represent the statements, opinions, or conclusions of NIEHS, NIH, or the United States government.