The growth of the World Wide Web as a pedagogic tool has paralleled the expansion of interest in teaching the responsible conduct of research (RCR) to scientists, especially biomedical researchers.1,2 It is generally accepted that the Web allows many people to be exposed to a standard curriculum, that the curriculum can be as detailed or simple as necessary or appropriate, and that record-keeping and evaluation can be automated.
History and Overview of the CITI Program
The Collaborative Institutional Training Initiative (CITI) program was conceived in June 2000 as a response to the Department of Health and Human Services (DHHS) requirement that investigators be trained in human subjects protection.3 Representatives from 10 organizations* were recruited by CITI founders Karen Hansen, director, Institutional Review Office at the Fred Hutchinson Cancer Research Center, and one of us (P.B.) to develop, review, and implement a flexible, Web-based educational program consisting of 12 basic modules on various topics related to the protection of human research subjects. The content modules were developed and peer reviewed during the summer of 2000, and they were made available to the 10 institutions through the University of Miami’s distance learning software platform on September 3, 2000. Christiana Care, a regional health care system in northern Delaware, was admitted to the collaboration in December 2000 as the first subscriber organization. In 2001, generous financial support was provided to the program by the former Applied Research Ethics National Association (now a part of the organization Public Responsibility in Medicine and Research) and the Department of Veterans Affairs. During the next two years the human subjects research content was refined at semiannual CITI developer meetings, and a refresher course was added for those participating organizations that needed a vehicle for investigator recertification.
In the summer of 2003, additional modules were added to the program specifically to address the needs of investigators, staff, and students in the social and behavioral sciences. This effort, lead by Lorna Hicks, director, Program for the Protection of Human Subjects in Non-Medical Research at Duke University, involved representatives from seven institutions: The American Psychological Association, Columbia University, Duke University, Mississippi State University, the University of Virginia at Charlottesville, the North Carolina Department of Corrections, and the University of Chicago–Illinois.
Later that year, a new collaboration was formulated between CITI and Michael Fallon, DVM, PhD, chief veterinary medical officer, Office of Research and Development, Central Office, the U.S. Department of Veterans Affairs (VA), to develop and implement a new software package to deliver the CITI Program to both VA employees and CITI participating organizations. The platform was modeled after the software used in Dr. Fallon’s laboratory animal welfare course, developed for the VA. CITI licensed the software from the American Association for Laboratory Animal Science, modified the platform to fit the CITI presentation paradigm, and released CITI version 6.0 in May 2004. With the advent of this greatly improved technology, institutional subscriptions increased dramatically.
To address the needs of institutions with international collaborators, CITI developed a public access international course site at (www.irbtraining.org). Some of the CITI content was translated to Spanish, Chinese, Portuguese, French, and Arabic. The CITI program currently has 26 international participants, and the list is growing. New educational initiatives in Latin America and the Caribbean are designed to test the CITI program paradigm across cultures.
Through March 2007, more than 600,000 people, at more than 715 institutions (universities, freestanding research centers, hospitals, federal agencies, and commercial enterprises) around the world have earned a CITI course completion certificate documenting compliance with federal regulations and institutional policies regarding instruction in human subjects research protections. On average, three new organizations subscribe each week.
The PHS educational mandate of June 2000 has been well accepted by CITI subscribers. In an IRB-approved voluntary user evaluation survey provided as an option at the end of the course, we asked learners their opinions on the need for training in human subjects protections. On a scale of 1 to 10, where 1 was totally unnecessary and 10 was absolutely vital, 47% of the 6,465 responders indicated that such instruction is absolutely vital, whereas only 12% of the 6,465 responders provided a rating of 5 or less.
From November 3, 2005 to November 3, 2006, 10,308 basic course survey responders indicated their preference of course presentation. The Web-based approach is not preferred by all learners. Although voluntary survey responders overwhelmingly prefer the Web- based approach (8,143; 79%), other presentation vehicles such as audio tapes (515; 5%) and video (927; 9%) are preferred by some. Only 206 (2%) responders reported that they would prefer a classroom setting for this kind of course.
The CITI program has enjoyed substantial growth since version 6.0 was released in June 2004. The changes in monthly completion certificate awards are seen in Figure 1. We attribute increased use to a combination of adding, on average, three new participating organizations per week and adding new CITI content.
In large measure, the success of the CITI program reflects the efforts and dedication of members of the CITI developer group to ensure a quality program. The group, composed of ethicists, physicians, social scientists, biomedical scientists, IRB professionals, educators, and Web design experts, meets twice a year to review user feedback, assess data from the user evaluation survey, offer guidance for new CITI initiatives, enable editorial oversight of the course materials, and provide general programmatic oversight. The initial success of the CITI project suggested that the model could evolve into a comprehensive research ethics resource. To that end, four new initiatives have been added to the program:
▪ CITI Course in Good Clinical Practice (GCP) was added in 2006 to address the needs of institutions conducting industry-sponsored investigational drug and device studies. This initiative was overseen by the CITI GCP working group headed by Allison Handler, MS, CCRC, project leader, Duke Clinical Research Institute at Duke University. This course will soon be available in several languages, including Spanish and Portuguese, for member organizations in Latin America and Brazil.
▪ CITI Course in Health Information Privacy and Security (HIPS), developed by Reid Cushman, PhD, Department of Information Technology, University of Miami, Miller School of Medicine, and one of us (K.W.G.) at the University of Miami, was also added in 2006. This customizable course addresses the Health Insurance Portability and Accountability Act rules and many other “best practices” for maintaining data and protecting one’s identity from theft or misuse.
▪ CITI Course in Responsible Conduct of Research was developed with partial funding from the Office of Research Integrity and DHHS and with oversight from the newly created CITI RCR developer group (a part of the CITI developer group). The course was made available to the research community in January 2007. This totally customizable course of foundation text, case studies, and quizzes is available in discipline-specific versions (i.e., biomedical sciences, social and behavioral sciences, physical sciences, humanities).
▪ CITI Course in Laboratory Animal Welfare will be added to the CITI program in June 2007. The addition of this popular course developed by the VA further exemplifies the close collaboration between CITI and the VA. (See information about the course at www.researchtaining.org.)
One benefit of such a strategy is that it provides a convenient source of peer-reviewed, low-cost, online tools to institutions facing increasing—albeit not unreasonable—demands that they do a better job educating scientists and clinicians about the moral foundations of and regulatory requirements for research. CITI’s addition of a free RCR resource provides an opportunity to explore the difference between pedagogy for students and professionals on the one hand and mere compliance for compliance’s sake on the other.
Responsible Conduct of Research: Education or Training?
CITI began the development of a suite of resources for RCR instruction in 2004, and the new CITI RCR program became available in January 2007. Although there is no federal requirement for biomedical RCR training except for graduate students and NIH-sponsored fellows on institutional training grants, there is clearly something disagreeable about awaiting a federal law before providing such instruction. Even if the government had never contemplated mandatory RCR instruction, it should be uncontroversial that integrity is as important to the scientific enterprise as is knowledge of scientific methods and laboratory skills.
With partial support from the DHHS Office of Research Integrity, CITI developers have prepared an ensemble of RCR resources that reflects most of what that office calls “instructional areas”4:
▪ Data Acquisition, Management, Sharing and Ownership
▪ Conflict of Interest and Commitment
▪ Human Subjects
▪ Animal Welfare
▪ Research Misconduct
▪ Publication Practices and Responsible Authorship
▪ Mentor/Trainee Responsibilities
▪ Peer Review
▪ Collaborative Science
Human subjects and animal welfare courses were developed separately for the CITI project.
Beyond standards of practice, this list seems to call for instruction in scientific values, arguably universal values in the context of research. Young, and perhaps not-so-young, investigators should not simply be given a rule about the wrongness of fabrication, for example, but should be given the reasons that it is a rule not to make up data in the first place. There are many reasons from various moral theories. A comprehensive and robust RCR program ought not to consist in the recitation of rules and the telling of war stories. Rather, it should include a foundation in moral reasoning so that when trainees and others are told the rules they will understand why they ought to behave a certain way.
In our experience, students and trainees do not engage in frank misconduct because they are uncertain about its wrongness, or because they are torn by conflicting moral values. They typically cheat because they are in a tight spot and hope they will not be found out. This should compel those of us engaged in RCR instruction to be clear about motivations for the movement toward more RCR instruction. If the belief is that students will not fabricate, falsify, or plagiarize once they are told these things are unacceptable, then the belief is based on inadequate warrant. This is not to suggest that students ought not be told which actions are clearly right and wrong—they should. But they must also be instructed in how to make such judgments, with an emphasis on critical thinking rather than rote rule-following.
It follows, then, that the current thrust toward RCR training is misguided. One is trained when one learns a skill, but one is educated when one learns the conceptual foundations that support a belief, practice, or skill; this is an old pedagogic distinction. Despite the fact that we call research education grants “training grants” and their beneficiaries “trainees,” we should not lose sight of the goal of instructing young investigators in the critical-thinking skills that are the foundation of moral reasoning as well as scientific inquiry. To use a clinical analogy, it would be a sad day indeed if we counted it a success that a student learned how to run a gel or how to do x-ray crystallography or how to read a CBC report—without also learning about what these tasks or skills mean for the understanding of genetics or protein folding or the mechanisms of disease.
For these reasons, CITI’s developers have tried whenever possible to leaven CITI’s RCR content with some applied ethics. RCR modules were selected or developed in hopes of emphasizing something more than a recitation of the rule. This is a significant challenge, inasmuch as the Web, despite its extraordinary utility, does not clearly, always, or easily lend itself to the kind of give-and-take that characterizes seminars in which learner interaction drives the pedagogic process. We are concerned that by creating high-quality online resources in applied ethics, we make or support inferences that ethics can be relegated to cyberspace while the hard or serious stuff is reserved for classroom instruction. That said, if the RCR evaluations turn out to be anywhere near as positive as CITI’s human subject protection curricula, then CITI will have contributed to efforts to establish the Web as a productive venue for online education as well as training.
CITI Online: The Need for Evaluation
An advantage to Web-based pedagogy is that it can be easily and continually evaluated to determine whether the program is meeting its intended goals and objectives. The CITI program has used a series of voluntary, anonymous user evaluation surveys—reviewed and approved by the University of Miami’s IRB—to determine how learners use the course site and to elicit their views regarding software performance, presentation, quiz questions, and the value of the CITI program as an adjunct to their research activities. The CITI RCR developer group reviews survey responses semiannually to provide insights on how the course site could be modified and improved.
Responses to the latest evaluation surveys on the basic course and refresher course on human subjects protection (anonymously submitted electronically between November 3, 2005, and November 3, 2006) have been very positive. Most questions ask the learner to provide a score between 1 (strongly disagree) and 10 (strongly agree) to a series of statements. Some questions require a yes or no answer, whereas others require the responder to choose a response from a preset list. The learner also has the opportunity to provide an open-ended response to a question about how the course could be improved.
The basic course survey indicates that learners believe that the time spent in the course site is worthwhile (mean score = 7.7; 10,151 respondents) and that the course has increased their basic knowledge (mean score = 8.5; 10,215 respondents) in human subjects protection issues. In addition, learners expressed a rather high degree of confidence (mean score = 7.7; 10,269 respondents) that completing the course will enable them to help a colleague with a human subjects protection issue more effectively. (Further details and analyses of the survey are available on request.)
In the basic course evaluation survey conducted between November 3, 2005, and November 3, 2006, 7,224 of 10,215 responders (71%) answered yes to the question: “Would you like to have access to Web-based courses on other aspects of the Responsible Conduct of Research (RCR) such as Scientific Misconduct, Responsible Authorship, the Mentor–Student Relationship, Peer Review, and Responsible Data Management?” Because approximately 26% of CITI learners self-identify as “students” at the time of registration, the new CITI RCR program potentially has a large audience. CITI’s track record in human subjects protection suggests that the model is of potential use in educating investigators in the responsible conduct of research—even in the absence of a federal requirement. The use of the CITI RCR course site between the mid-January 2007 rollout and May 1, 2007 would seem to bear this out. During this time, more than 1,100 learners earned an RCR completion certificate for the 15-module course at the CITI public access RCR course site.
The Value of Online RCR Education
RCR education is the responsibility of all who are involved in the research enterprise. Although many will argue that Web-based courses or tutorials are not well suited to ethics education, the very fact that only 2% of survey responders reported that they preferred traditional classroom instruction to the Web-based approach that CITI provides clearly indicates that old paradigms must be reviewed and new ways to effectively deliver ethics education tested.
Research misconduct occurs because of underlying pathology, the highly competitive “publish or perish” academic environment, simple greed, thirst for fame, poor mentoring, and/or unpracticed ethical-decision-making skills. Courses such as those in the CITI program have proven useful in raising levels of awareness of the basic tenets of ethical behavior among all members of the research team, including the principal investigator. The growth of interest in RCR has been fed and watered by several public cases of irresponsible conduct. Although we exercise extreme caution in suggesting that CITI or any other four-hour curriculum will be causally related to any reduction in misconduct, we submit that when all members of the team are educated about common moral standards and appropriately mentored, the temptation for misbehavior by any one member is reduced, better science is promoted, and the public trust is preserved. At the very least, providing RCR education is a positive and useful means of reminding taxpayers and others that the same investigators who are spending their billions on research are simultaneously striving to ensure that integrity is woven into the fabric of the scientific enterprise. We can remain agnostic about best answers to Socrates’ question, “Can virtue be taught?” even as we hope that it can, at least a little, online.
The CITI program would not have been possible without the passion and dedication of the program’s authors and the CITI developer group members. Start-up funding in 2000 was provided by the Applied Research Ethics National Association, the University of Miami Office of Research. The program could not have been a success without generous support from the Department of Veterans Affairs and the Department of Energy.
CITI cofounders Karen Hansen and Paul Braunschweiger wish to acknowledge and thank for their dedication and contributions to the CITI program the original authors, CITI developer group members, and members of the CITI executive advisory committee.
Work on CITI’s RCR content was supported by two contracts from the Office of Research Integrity (combined under award SEA004651-01-MIA-001) and by the home institutions of CITI RCR developer group faculty members.
Translations of the human subject protection modules was funded in part by a National Institutes of Health, National Center for Research Resources grant (2 S07 RR018180), “FHCRC-Human Subjects Research Enhancements Program,” awarded to the Fred Hutchinson Cancer Research Center, Seattle, Wash; and by a National Institutes of Health, National Center for Research Resources grant (1 S07 RR18143-02), “South Florida Human Subjects Benchmarking Coalition,” awarded to the University of Miami.