ClinicalTrials.gov is an online registry and results database of clinical trial information that is operated by the National Library of Medicine (NLM). Since 2005, journals adopting the recommendations of the International Committee of Medical Journal Editors will consider trial manuscripts for publication only if registration of the trial occurred before the first patient was enrolled. 1 The Food and Drug Administration Amendments Act of 2007 (FDAAA) requires that all applicable clinical trials be registered within 21 days of enrollment and that results be reported within 1 year of trial completion. 2 The FDAAA authorizes the Food and Drug Administration (FDA) to issue civil monetary penalties for noncompliance, which are currently $12,316 per trial per day. The Final Rule, which took effect January 18, 2017, clarified and expanded the FDAAA, with organizations expected to be in compliance by April 18, 2017. 3,4 At the same time, the National Institutes of Health (NIH) issued broader requirements that apply to all trials funded by the NIH. 5 Research groups that do not comply with the NIH policy can lose their funding. 4
In March 2014, several events prompted the Johns Hopkins University School of Medicine (JHUSOM) to initiate an assessment of institutional compliance with the requirements for clinical trial registration and results reporting. A principal investigator (PI) affiliated with the school had received a letter from the FDA citing noncompliance with the FDAAA, and other investigators were seeking assistance using ClinicalTrials.gov. Thus, JHUSOM vice dean of clinical investigation (D.E.F.) asked a project manager (PM; A.K.), to assess compliance at the institution.
At the time, little published literature described how to build a compliance program at an academic medical center. The PM attended an NLM workshop designed to help institutions understand and use ClinicalTrials.gov. 6 In June 2015, the PM obtained administrative control of all trials registered by JHUSOM in the Protocol Registration and Results System (PRS), the online system that allows the institution to register, update, and report results on ClinicalTrials.gov, and ran a baseline report of compliance.
In the PRS, registered trials are assigned a National Clinical Trial (NCT) number and are referred to as “records.” The baseline report in September 2015 identified 339 of 774 trials (44%) as being potentially noncompliant with the FDAAA (i.e., “problem records”); of these, 107 trials were identified as “Late results—per FDAAA” (late results), which are trials that have not been reported within 1 year of the date of last data collection for all primary outcome measures. At the time, we estimated that civil monetary penalties of approximately $10,000 per trial per day could result in a potential institutional liability of over $1,000,000 per day.
To ensure that the institution remained a responsible member of the research community, to alleviate potential financial liability, and because of the specialized knowledge required to understand the relevant clinical trial registration and results reporting requirements, we established a formal program to support investigators in compliance with these requirements. Here, we share the steps we took and the lessons we learned while establishing the ClinicalTrials.gov Program at JHUSOM for institutions that might develop similar programs.
The timeline shown in Figure 1 lists steps we took while establishing the ClinicalTrials.gov Program and the staffing levels entailed.
Evaluating processes at peer institutions
Before developing our program, the PM joined the Clinical Trials Registration and Results Reporting Taskforce. 7 Through this taskforce, the PM communicated with peer institutions to discuss their policies, procedures, and resources related to trial registration and results reporting. The taskforce continues to meet monthly and allows institutions to share information through its website.
Accessing trial records
Colleagues at peer institutions described the importance of internal collaborations with the institutional review board (IRB) and other relevant stakeholders. Before launching the program, the JHUSOM IRB director granted the PM read-only access to all studies, which allowed the PM to verify current study status, run reports, and contact investigators as needed.
Highlighting potential financial liability, the PM made a business case to leadership to secure institutional and financial support through the Clinical and Translational Science Award (CTSA). JHUSOM assigned the PM to work on trial registration and results reporting. JHUSOM hired a clinical research compliance specialist and launched the program in June 2016 with the following objectives:
- Develop institutional ClinicalTrials.gov policies
- Develop a process for communicating with investigators
- Identify and reduce problem records
- Develop and implement proactive compliance measures
- Train investigators to meet trial registration and results reporting requirements
- Expand the program across JHUSOM
Institutional policies and procedures
The program revised an existing JHUSOM Organization Policy on Registration of Clinical Trials to meet the requirements in the Final Rule. 4 After approval by the vice dean, we posted the updated policy to the institutional website.
In addition, we revised the IRB application to collect the details necessary to ensure that all clinical trials meeting the statutory requirements were appropriately identified. Program staff created standard operating procedures to ensure institutional uniformity of activities such as creating user accounts within the PRS, transferring records between institutions, and communicating requirements to PIs and departmental leadership. Today, our program regularly reviews JHUSOM policies to ensure they reflect current regulations and guidelines.
In 2016, the program staff began to communicate the requirements for registration and results reporting to investigators who might be noncompliant with the FDAAA. The vice dean also informed department directors of their financial responsibility for any monetary penalties levied against the institution.
For records with late results or other problems, program staff first notified investigators from the program’s email account. Program staff explained the problem and described steps to take to remedy it. If the investigator did not address the problem, the staff sent a second email and copied the department director. Occasionally, the program staff sent a third email copying the vice dean. Program staff also made phone calls and personal visits to meet investigators when necessary.
Eliminating late results
Our first steps in 2016 involved reporting late results. We found that some trials that were identified in the PRS as late results were still actively recruiting. For these trials, the program staff worked with the investigators to update the records to indicate that results were expected in the future.
We also discovered that several records identified as late results had not enrolled any participants; updating the trial status to “withdrawn” eliminated the requirement to report results.
We learned from peer institutions to start with protocols that had been late the longest. To collect the required information, we first reached out to PIs and then to anyone identified in the ClinicalTrials.gov record, IRB study team members, and manuscript coauthors. Some PIs and staff members had transferred departments, left the institution, or retired. We retrieved historical data for 10 trials from off-site storage, and we used the IRB termination report (i.e., a report submitted by the PI) to enter results into ClinicalTrials.gov for those trials.
Lastly, we communicated to investigators that the April 2017 Final Rule compliance date was the deadline for submitting all required information. We received a robust response before the deadline; 35% (31/88) of the trial results we submitted in 2017 were within ±2 weeks of the deadline.
Preventing problems prospectively
After hiring a second staff member in 2017, we began to use the PRS planning report to prospectively identify records requiring results reporting. We began to reach out to investigators 4 months before the results were due. If the anticipated completion date had changed, we assisted investigators in updating the record and reminded them to maintain the record’s accuracy.
Many investigators require support throughout the registration and results reporting process. We provide this support at no cost to the investigators. We also perform tasks on a fee-for-service basis. Since 2017, 6 investigators have asked us to enter results for 9 trials, which required 64 hours of effort, for which we charged $50 per hour.
In addition to late results, the PRS identifies problems such as “not recently updated,” “missing FDAAA information,” “incomplete results—per FDAAA,” and “PRS review comments.” A single record could have multiple problems. Today, the program reviews problem records daily, enabling us to quickly recognize new problems. In addition, we developed a comprehensive checklist that we use to help orient new program staff and to ensure consistency among administrators within our institution. The ClinicalTrials.gov PRS team at the NIH is helpful with challenging cases.
Although JHUSOM does not mandate training for investigators, we developed voluntary in-person training sessions to help investigators maintain compliance and avoid penalties. These sessions are available to departments upon request, and recordings are available through YouTube and on our website.
At JHUSOM, participant enrollment in clinical trials is recorded in the Clinical Research Management System (CRMS), an online reporting platform. In collaboration with the institution’s IRB, we automated the reports created to ensure that the information in ClinicalTrials.gov is accurate and up to date. These reports include one that identifies trials that began enrollment without citing the NCT number in either the IRB application or the CRMS. We ran a baseline report in August 2017 and identified 149 trials that we investigated until the NCT number was entered. Today, we reach out to investigators of new trials to ensure prompt registration.
Expanding the program
In addition to the JHUSOM PRS, other Johns Hopkins University entities have PRS accounts. In April 2018, the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (SKCCC) hired a clinical research compliance specialist to focus exclusively on SKCCC records. SKCCC already had procedures for clinical trial research operations, and we collaborated with their Clinical Research Office to facilitate communication and encourage cooperation. The staff within this office have established relationships with investigators and historical knowledge about trials and staff turnover. Just as JHUSOM began by addressing existing problem records, in its first year, the SKCCC program aimed toward retrospective compliance by addressing problem records and assigning appropriate individuals as record owners or access list members.
Compliance with the FDAAA and NIH policies increased for completed and ongoing trials from September 2015 (baseline) to September 2020 (4 years after we introduced the ClinicalTrials.gov Program at JHUSOM), as shown in Figure 2.
In the annual report run in September 2016 (only 3 months after program launch), we found that the total number of problem records had increased to 45% (383/852), with 742 total problems. Of these, 113 records had late results.
In September 2017, a year after the introduction of the program, the proportion of problem records decreased to 30% (282/955), including 479 total problems and 25 records with late results. Annual reports show continuing progress, so that by September 2020, the proportion of problem records decreased even further to 2% (32/1,304), with 63 total problems and 1 (0%) late result.
The program continues to focus on ensuring that records are maintained accurately and are up to date on ClinicalTrials.gov, including a zero-tolerance policy for late results. We review new registrations and record updates at regular intervals, conduct ongoing education in using the PRS, and update policies and standard operating procedures. We continue to evaluate tools (e.g., software) for managing the growing number of records.
As a CTSA hub, JHUSOM is driven to develop innovative solutions and increase efficiencies in clinical research. We have started monthly calls including other Johns Hopkins entities such as the Kennedy Krieger Institute and the School of Public Health to gain internal efficiencies. We plan to share the resources we have developed with our peer institutions through the taskforce, invited speaking engagements, and national meetings and publications, to enable us to contribute to national best practices. 8
Future efforts will be to identify or develop software solutions to more efficiently distinguish trials funded by the NIH, detect records before they become problems, and track metrics over time.
To comply with the scientific, ethical, and legal requirements for registering, updating, and reporting the results of clinical trials on ClinicalTrials.gov, institutions need experts who stay current with guidelines, detect problems in the PRS, and provide training and support for investigators. 9 Multiple offices, including IRBs 10 and institutional leadership, must be engaged to secure resources to support these activities, champion the importance of research transparency, and administer consequences for noncompliance with existing regulations.
Always strive for 100% compliance but realize there will inevitably be a few problems and occasional late results even for mature programs.
The authors would like to acknowledge Nidhi Atri, MD, for contributing to much of the initial infrastructure of the ClinicalTrials.gov Program.
1. De Angelis C, Drazen JM, Frizelle FA, et al. International Committee of Medical Journal EditorsClinical trial registration: A statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004; 351:1250–1251
2. Public Law No. 110-85-Sept. 27, 2007. 121 Stat 823. https://www.govinfo.gov/content/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf#page=82
. Accessed August 20, 2020
3. Annual civil monetary penalties inflation adjustmentFinal rule. Fed Regist. 2020; 85:2869–2887. https://www.govinfo.gov/content/pkg/FR-2020-01-17/pdf/2020-00738.pdf
. Accessed August 20, 2020
4. Clinical trials registration and results information submissionFinal rule. Fed Regist. 2016; 81:64981–65157. https://www.ncbi.nlm.nih.gov/pubmed/27658315
. Accessed August 20, 2020
5. National Institutes of Health. NIH Central Resource for Grants and Funding Information. Requirements for registering & reporting NIH-funded clinical trials in ClinicalTrials.gov. National Institutes of Health, U.S. Department of Health and Human Services.. https://grants.nih.gov/policy/clinical-trials/reporting/index.htm
. Updated October 24, 2017. Accessed August 20, 2020
6. National Institutes of Health. U.S. National Library of Medicine Results Database Train-the-Trainer Workshop. https://clinicaltrials.gov/ct2/manage-recs/present
. Accessed August 20, 2020
7. Clinical Trials Registration and Results Reporting Taskforce. https://ctrrtaskforce.org
. Accessed August 20, 2020
8. Keyes A, Mayo-Wilson E, Atri N, et al.Time from submission of Johns Hopkins University trial results to posting on ClinicalTrials.gov
. JAMA Intern Med. 2020; 180:317–319
9. Mayo-Wilson E, Heyward J, Keyes A, et al. National Clinical Trials Registration and Results Reporting Taskforce Survey SubcommitteeClinical trial registration and reporting: A survey of academic organizations in the United States. BMC Med. 2018; 16:60
10. Grant S, Bouskill KE.Why institutional review boards should have a role in the open science movement. PNAS. 2019; 116:21336–21338