Recently, both of us completed simulation-based, randomized control trials assessing the effect of an educational intervention on health care provider–related outcomes. We were so impressed with our work that we decided to submit the resulting manuscripts to a high-impact clinical journal. To our surprise, we received an e-mail from the editorial office of the journal requesting a clinical trial registration number before our manuscript could be processed for review. We vaguely knew that clinical trial registration had something to do with drug trials sponsored by “big pharma” and assumed that this was not applicable to simulation-based research. After digging a little deeper, we uncovered a number of interesting issues, which have triggered us to ask the question, “Is clinical trial registration for simulation-based research necessary?” In this commentary, we describe the background and reasoning behind clinical trial registration and discuss whether or not simulation-based studies should be registered as a prerequisite to publication.
Clinical trial registries are Web-based databases of clinical trials, providing researchers, journal editors, and reviewers detailed study information to help inform trial results.1–3 Clinical trial registries help granting agencies with more efficient allocation of research funds and provide a resource for investigators to search for ongoing or previous trials to help prevent unnecessary duplication of invested research resources. Scientifically, these registries help to improve trial recruitment, increase opportunities for collaboration, and ensure transparency of trial design and methods, which may lead to further improvement of trial quality for future studies.3 Furthermore, clinical trial registries fulfill an ethical obligation by ensuring that patients, family members, and the general public have easy access to information about publicly and privately supported clinical studies (ongoing and previously completed) for a spectrum of diseases and medical conditions.1–3 They enable monitoring of adherence to ethical principles and research processes and help to reduce the risk of abuse to trial subjects.3
Various trial registries exist and differ with respect to the focus of research, nature of required trial information, and sponsoring organization.1Clinicaltrials.gov is a United States–based registry that accepts detailed information about research involving human participants conducted around the world. It was established by the National Institutes of Health in 2000 in response to an act of the US Congress in 1997 (later updated in 2007) requiring public reporting of studies involving Food and Drug Administration–regulated medications, biologics, and devices.4 In 2005, the World Health Organization (WHO) advocated for clinical trial registration and developed a search portal that would allow searching across the various national sites via the WHO International Clinical Trials Registry Platform.5 Currently, 14 trial registry sites are included in the WHO Registry Network, based on a set of criteria developed to meet the standards of the International Clinical Trials Registry Platform.5
Importantly, in 2005, the International Committee of Medical Journal Editors (ICMJE) required that all clinical trials be prospectively registered before they would consider publication in their 11-member journals.6 Thus far, more than 1500 nonmember journals have voluntarily signed on to this requirement. The ICMJE, in alignment with WHO, defines a clinical trial as “any research project that prospectively assigns people or a group of people to an intervention, with or without concurrent comparison or control groups, to study the cause-and-effect relationship between a health-related intervention and a health outcome.”6 Health-related interventions are described as “those used to modify a biomedical or health-related outcome”6 (eg, drugs, surgical procedures, devices, behavioral treatments, dietary interventions, quality improvement interventions, and process-of-care changes), whereas health outcomes are defined as “any biomedical or health-related measures obtained in patients or participants, including pharmacokinetic measures and adverse events.”6
Does simulation-based research fit this definition? Simulation-based research has been broadly described as falling into 2 separate categories: (a) research about simulation as a training methodology, which examines the effectiveness of simulation as an educational intervention; and (b) research using simulation as an investigative methodology, which uses simulation as the environment to study factors affecting human and systems performance in health care.7 Independent of the type of simulation-based study, the answer to the question can be analyzed based on the 3 main components of the definition: the definitions of the research subject, intervention, and health outcome. First, although mannequins or part-task trainers are certainly not “people or groups of people,”6 health care providers being recruited as subjects of research certainly are. Second, if the intervention is a simulation-based educational session, it is less clear whether this would fit the ICMJE definition. Notably, behaviors, quality improvement interventions, and process-of-care changes are given as examples, but education, per se, is not. Perhaps, we could classify simulation-based education as a “quality improvement measure.”6 Finally, is learning a health outcome? One could argue that learning to put a central catheter in safely or doing chest compressions adequately during a simulated cardiac arrest is a health outcome—at least indirectly. Where do we draw the line? Does it become a relevant health outcome if the same provider can demonstrate those skills in the real clinical environment? The ICMJE Web site provides some guidance on this matter by recommending trial registration for studies assigning health care providers to intervention and comparison/control groups if the “purpose of the trial is to examine the effect of the provider intervention on the health outcome of the providers’ patients”… while going on to state that “if the purpose is to examine the effect only on the providers (for example provider knowledge or attitudes), then registration is not necessary.”8 If we regard simulation as translational science and categorize outcomes as T1 (results achieved in the simulated setting), T2 (improved health care delivery in the real clinical setting), and T3 (improved patient outcomes),9 then interpretation of the ICMJE definition would lead us to believe that T1-level studies would not require registration, whereas T2- and T3-level studies would require registration. If this is the case, then why was there an expectation for us to register our simulation-based studies, both of which assessed outcomes at the T1 level? Our experiences suggest that various journals may interpret the ICMJE definitions in different ways when applied to simulation-based research, and as such, researchers should strongly consider prospective registration in lieu of knowing exactly how different journals may handle the issue when the manuscript is submitted for publication.
Perhaps, the more important question is not whether studies using simulation each fit the definition, but rather, is registering simulation-based research studies a good thing that we should be encouraging? The majority of proposed benefits for registering trials seem to relate to simulation-based research in the same way they would apply to clinical studies. Registering simulation-based research trials would confer the same benefits related to allocation of research funding, enhanced collaborative opportunities, transparency in study methodology, and fulfilling the ethical obligation to the community as a whole. One of the main benefits of trial registries is the potential reduction of risk of abuse to trial subjects. As simulation studies typically recruit health care providers or students, as opposed to real patients for clinical trials, the risk although present, is markedly different considering the context of most simulation-based research studies.
Although potentially providing a myriad of benefits, the simulation research community will optimally benefit only if registration of simulation studies is completed on a consistent basis. We searched Clinicaltrials.gov for the terms education AND simulation and got 109 hits (0.06%) of the 170,537 trials registered. Some of the studies were about patient education, although many were the sort of reports we commonly see in Simulation in Healthcare. Perhaps, if the simulation community made a coherent effort to register simulation studies, we could substantially raise this number. If this were to occur, this would certainly enhance the benefit of clinical trial registration for simulation-based studies. Registration of simulation studies in clinical databases would potentially promote collaboration between clinical and simulation-based researchers and draw attention to the fact that powerful simulation-based educational interventions may have measurable effects on both provider- and patient-specific outcomes.9 This may ultimately improve opportunities for grant funding and subsequently enhance the impact of research conducted in the simulated setting. The development and integration of a uniform reporting convention for simulation-based research in clinical registries would be a method to improve the quality of simulation-based research.7 However, the adoption of the same convention across multiple clinical trial registries would require a coordinated effort from the international simulation research community, ICMJE, and other relevant stakeholders. Even without a uniform reporting convention specific to simulation research, there still remain significant benefits to registering simulation-based research in existing clinical trial databases. Lastly, in contrast to the scientific literature, where researchers have access only to completed studies, clinical trial registries provide a readily accessible database of trials reflecting the scope of both ongoing and previous work in a particular field.
An alternative to using Clinicaltrials.gov or one of the other existing registries would be to start a new database for trials specifically for simulation-based research. In theory, this would serve as a “one stop shop” for the simulation research community, provide an opportunity for the data elements to be customized to the unique nature of simulation-based research, and offer a more seamless integration of a uniform reporting convention for simulation-based research. For example, the database may require the entry of standardization methods for mitigation of threats to internal validity specific to simulation (eg, scenario design and delivery, simulator selection, standardized patient use),7 and in this fashion, serve to enhance the quality of simulation research being conducted and published. The downside to adopting this option is that it keeps simulation-based research in a separate silo, thus reducing the likelihood of cross-pollination between education and clinical research and potentially reaching a smaller audience of stakeholders. Furthermore, it is difficult to predict where the resources required to develop and sustain this database would come from. As such, we believe the more viable option would be to petition the appropriate organizations—ICMJE, WHO, and so on—to explicitly include “simulation” in their working definitions for the already existing database(s) and take steps as a simulation research community toward adopting a uniform reporting convention for simulation-based research.
In the end, we ended up registering our simulation-based research trials post hoc (with special consent obtained from the journal); the registration process took approximately 45 minutes to complete, and that it was simulation-based study did not prove to be an impediment in the registration process. Although the ICMJE definition suggests that simulation studies with T2 and/or T3 outcomes would qualify for clinical trial registration, there clearly exists some variability in way this definition is interpreted by different journals. Recognizing that journal editors may decide on a case-by-case basis whether to review unregistered trials,10 we would recommend that investigators err on the side of caution and prospectively register their simulation-based studies (with T1-, T2-, and/or T3-level outcomes) in one of the preexisting clinical trial registries. Taking this approach eliminates any chance of a manuscript being immediately rejected by a journal secondary to the lack of trial registration. Perhaps, Simulation in Healthcare, as the leading simulation-based research journal in the world, should consider sharing a formal recommendation on this matter, which could ultimately serve as a springboard for future discussion with ICMJE. In the future, we suspect there will be more clarity, but in the interim, registering simulation-based research studies is perhaps less a necessity but more a requirement for peace of mind.
1. Zarin DA, Keselman A. Registering a clinical trial
in clinicaltrials.gov. Chest
2007; 131: 909–912.
2. Zarin DA, Tse T, Ide NS. Trial registration
at ClinicalTrials.gov between May and October 2005. N Engl J Med
2005; 353: 2779–2787.
3. Krleza-Jeric K, Chan AW, Dickersin K, et al. Principles for international registration
of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ
2005; 330: 956–958.
7. Cheng A, Auerbach M, Hunt EA, et al. Designing and conducting simulation
2014; 133: 1091–1101.
9. McGaghie WC, Draycott TJ, Dunn WF, et al. Evaluating the Impact of simulation
on translational patient outcomes. Simul Healthc
2011; 6 (Suppl): S42–S47.
10. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration
: a statement from the International Committee of Medical Journal Editors. N Engl J Med
2004; 351: 1250–1251.