In April 2020, the U.S. National Institutes of Health (NIH) established a public-private partnership (PPP), Accelerating Coronavirus Disease 2019 Therapeutic Interventions and Vaccines (ACTIV) (1), to catalyze development of therapeutics and vaccines in order to reduce morbidity and mortality due to coronavirus disease 2019 (COVID-19) and to accelerate ending the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic. ACTIV leverages the innovation and capacities of 18 pharmaceutical companies, the NIH, the Biomedical Advanced Research and Development Authority, the U.S. Centers for Disease Control and Prevention, the U.S. Department of Defense, and the U.S. Department of Veterans Affairs, with management support from the Foundation for the NIH.
ACTIV includes an Executive Committee and four working groups (WGs): Preclinical, Therapeutics-Clinical (TX-Clinical), Clinical Trial Capacity, and Vaccines. The TX-Clinical WG was charged with two specific tasks: 1) to identify agents and systematically and efficiently prioritize them for placement in master protocols (MPs) and 2) to design rigorous, randomized-controlled MPs to test those agents.
The TX-Clinical WG was established with 32 delegates representing 22 organizations. Delegates for this group were selected by their organizations based on expertise. ACTIV leadership encouraged communication with panels similarly charged in other nations, as well as recruitment of any additional experts necessary to meet the charge. Thus far, the TX-Clinical WG has benefited from the expertise of over 50 invited scientists, clinicians, funding agency executives, consultants, and manufacturing entities.
In order to capture the widest range of repurposed and novel agents for consideration and prioritization, the TX-Clinical WG surveyed lists generated by other groups worldwide and publicly available data, and established a public portal for candidate submissions from investigators stemming from industry, academia, and the general public. As of the end of May 2021, more than 750 agents from over 250 sources had been considered. To efficiently evaluate and refine the expansive list of candidate agents into shorter lists for inclusion into clinical trials, the TX-Clinical WG partitioned into a therapeutic agent prioritization (AP) subgroup and an MP development subgroup. This report describes the deliberative processes, outcomes, and lessons learned by the AP subgroup.
The work of the AP subgroup was divided into three waves. The first wave (from April 15, 2020, to May 1, 2020) aimed to expedite identification and prioritization of the most immediately available, previously Food and Drug Administration (FDA)–approved agents appropriate for clinical trials. The second wave (Investigational New Drug [IND]-enabled agents and agents needing only minimal additional preclinical or clinical pilot data; from May 2, 2020, to July 31, 2020) and third wave (agents needing more time to gather additional preclinical and clinical data; from August 1, 2020, to present) deepened the scope to include late-stage investigational agents, agents submitted for consideration through an ACTIV COVID-19 Clinical & Preclinical Candidate Compound Survey, and novel anti-SARS-CoV-2 specific agents, primarily antivirals, and neutralizing antibody (nAb) products.
The AP subgroup created and refined a general framework for evaluating and prioritizing agents.
First, the framework divided agents into therapeutic classes. Candidates were classified into agents with potential antiviral, immunomodulatory, or organ-supportive activity. A separate classification for nAb was added later (Fig. 1). Sorting of agents in this manner also helped enlist specific expertise in evaluating particular therapeutic class candidates.
Second, the framework classified agents by relevant populations that could potentially benefit based on route of delivery and by the severity of disease. Although IV medications are predominantly suitable for hospitalized patients, oral, topical (e.g., inhaled), or locally injected (subcutaneous and intramuscular) agents can be used in broader settings. Based on information available in April 2020, three target populations were identified: outpatients with COVID-19, patients hospitalized with COVID-19, and those not yet infected with SARS-CoV-2 (Supplemental Digital Content 1, https://links.lww.com/CCM/G647; legend, https://links.lww.com/CCM/G653).
Assembly of a List and Relevant Information for Drugs to Be Considered as Interventions for COVID-19
With the general framework established, the initial candidate list was assembled from public and solicited/contributed databases (Fig. 2) (2–6). A merged and deduplicated initial list yielded more than 450 candidate agents for initial evaluation and prioritization.
The AP subgroup identified five characteristics for the approach to the prioritization process (Fig. 3). The approach needed to be inclusive, systematic, unbiased, rapid, and adaptive to incorporate the evolving knowledge of the pandemic. It created a four-stage process consisting of: 1) initial screen for clinical readiness, 2) triage, 3) criteria-driven scoring, and 4) final selection.
Clinical Readiness Screen
The first wave reflected the imperative to establish clinical trials as quickly as possible. Thus, initial readiness criteria (Fig. 3) required the candidate product be: 1) FDA-approved or IND-enabled, 2) at a clinical development stage ready for testing in either COVID-19 or another relevant indication (Middle Eastern Respiratory Syndrome, acute respiratory distress syndrome, etc.), and 3) sufficiently available for at least one phase 3 trial. Notably, agents that failed this initial clinical readiness screen were not discarded; rather, they were set aside for future consideration. Of the more than 450 initial candidates assembled from the publicly available databases described earlier, 170 passed a triage/clinical readiness screen. Those agents that failed most commonly did so at steps 1, 2, or 4 of triage, as described in Figure 3.
Each of the 170 candidates surviving the clinical readiness screen was assigned to two nonconflicted, independent reviewers for initial evaluation and rapid presentation to the AP workgroup with the goal of reducing the candidate pool to a few dozen candidates. A real-time search was made to determine whether there were ongoing trials of each candidate: ACTIV specifically sought to avoid duplication with other national and international efforts. Thus, many agents with plausible mechanisms of action (MOA)—including some antiviral and immunomodulatory drugs—were deferred because those pathways were already being examined by other groups with robust well-designed trials.
Following a further review of screening criteria (Fig. 3), assigned reviewers responded to the question of whether the agent should be advanced to formal scoring or be deferred. In this review and discussion round, the list of 170 candidates was further reduced to 39: 10 antivirals, 14 host-targeted immunomodulatory drugs, and 15 organ-supportive therapies.
The initial scoring rubric is shown in Figure 4. Each candidate was assigned to at least three reviewers who independently sought information from sources such as PubMed, bioRxiv, medRxiv, clinicaltrials.gov, other clinical trial registries, official prescribing information, etc. For five of the seven categories, a “this agent not tenable-stop scoring” or “No Go” classification was established so as to maximize reviewer efficiency. Among the 117+ scorecards, only 15 included a final “No Go.”
Selection and Referral
Following scoring and discussion, the AP subgroup determined that six agents should be advanced for consideration in two separate trials: one trial for immune modulators and one trial for antithrombotic strategies.
None of the candidate antiviral agents scored well enough in this first wave of prioritization to be selected for clinical trial evaluation. This was largely due to inconsistency of results emerging from different in vitro SARS-CoV-2 assays or insufficient availability of COVID-19 treatment data.
Final results of wave 1 of Agent Prioritization (AP) were reviewed and approved by the ACTIV TX-Clinical WG (Fig. 5).
WAVE 2 AND BEYOND
The AP subgroup learned three lessons during wave 1 that informed refinements for wave 2. First, the breadth of agents to be considered should broaden beyond agents immediately available for repurposing. To this end, ACTIV commissioned a public portal (ACTIV COVID-19 Clinical & Preclinical Candidate Compound Survey) through which the global community (broadly defined to include individual researchers, academia, and industry) was encouraged to nominate agents derived from approved drugs and agents in development.
Second, because the AP subgroup benefited from ad hoc expertise in wave 1, they invited additional experts into the review groups for each therapy class to ensure the AP subgroup contained sufficiently broad expertise to evaluate all potential agents. For example, research scientists with close connections to clinical practice provided useful insights during assessment of organ-supportive therapies. Immunologists assessed whether a particular immunomodulator being reviewed may neglect or impede important immune response cascades, whereas virologists and pharmacologists more accurately assessed an antiviral agent’s ability to demonstrate in vitro potency and to achieve necessary tissue-specific drug concentrations. nAbs against SARS-CoV-2, a high-priority drug class, required a separate group of experts set for review. Ultimately, four class-specific review panels were established (Fig. 1).
The third lesson was that the initial scoring criteria were too general. For subsequent evaluations, the team amended these scoring criteria to enhance precision and also edited the “ACTIV COVID-19 Clinical & Preclinical Candidate Compound Survey” portal’s questions to align with division of agents by class and the new scoring criteria. The AP subgroup further recognized the scoring criteria needed to be tailored to each class of agents. For example, it was appropriate to emphasize in vitro activity for antivirals, whereas data from human studies should be more heavily weighted for immunomodulators. Bringing the survey portal questions into alignment with the scoring criteria proved essential to fast and objective evaluation of candidates (Supplemental Digital Content 2, https://links.lww.com/CCM/G648; legend, https://links.lww.com/CCM/G653).
Decisions frequently required further information to answer questions raised during the review process. Thus, initial scoring led to one of four outcomes: 1) Go, 2) Follow-up, 3) Defer, and 4) No Go (Supplemental Digital Content 3, https://links.lww.com/CCM/G649; legend, https://links.lww.com/CCM/G653). Agents in the “Go” category were advanced toward an ACTIV MP for a phase 3 or phase 2/3 progressive trial. If follow-up was required, agents were promptly reevaluated by the group as soon as additional data were supplied by the proposer. In some cases, interesting compounds were deferred for reconsideration pending outcomes of ongoing experiments. “No Go” indicated exclusion from further consideration.
For wave 2, a new subteam was recruited to evaluate and recommend nAbs to enter ACTIV-2 and ACTIV-3 phase 2 and/or 3 MPs. The nAb subteam was tasked with generating objective criteria and relevant supporting data for this agent class. The group responded with iteratively established minimal, supporting, and disqualifying (“No Go”) criteria (Fig. 6). Details of the criteria revision process are given in Supplemental Digital Content 4 (https://links.lww.com/CCM/G650; legend, https://links.lww.com/CCM/G653).
To date, for waves 2 and 3, over 300 agents have been assessed (or reassessed from an earlier wave based on additional data), and of those evaluated, 20 agents have been selected for placement into ACTIV MPs. Of the selected agents, a few were eventually withdrawn by the submitters to be developed in their own trials (Fig. 7).
FROM OFF-THE-SHELF TO DEVELOPING AGENTS: INTEGRATING WITH THE PRECLINICAL WG
As clinical AP processes progressed through the first two waves, groups of agents being evaluated shifted toward earlier stages of drug development. There are at least three explanations for this shift. First, the initial goal of rapidly identifying candidates for testing in the clinical trials had been met. Second, the public portal attracted nominations of agents that had not yet reached late-stage clinical trials. Third, a common reason for the delay in reaching human trials was gaps in available data that could most expeditiously be addressed by matching the agent with an appropriate preclinical model to confirm the proposed MOA in the context of SARS-CoV-2 infection. Furthermore, as greater insight into the SARS-CoV-2 pathobiology emerged, it was predictable that new nominees would require preclinical data to justify clinical trials and to better understand therapeutic potential for those agents.
The ACTIV Preclinical WG’s overall mission is to create an integrated preclinical framework to evaluate existing drugs and drug candidates that require preclinical evaluation for both repurposing and development of novel therapeutics (Supplemental Digital Content 5, https://links.lww.com/CCM/G651; legend, https://links.lww.com/CCM/G653). Given this mission, a team of government (NIH and FDA), academic, and industry scientists with expert knowledge of early-stage therapeutic development was assembled to assess how to progress efficient clinical entry. Nominations sent to the public portal without sufficient data to advance immediately to clinical trials were referred to this WG. The prioritization process was refined to include additional weighted criteria for assessing preclinical aspects of rationale for the proposed MOA, in vitro data supporting activity against SARS-CoV-2 itself or some aspect of COVID-19, and animal and human pharmacokinetics/pharmacodynamics data, in vivo data from preclinical models of SARS-CoV-2 infection or other potentially relevant in vivo disease models, toxicology and clinical safety data, drug-drug interactions, scalability, and a realistic estimated timeline for obtaining the necessary data that would permit assessment of readiness for inclusion in a clinical trial MP. Overall gaps in preclinical development for each compound were assessed, along with suggested approaches for addressing those gaps. The rubric evolved to include features such as the agent’s readiness for further prioritization, uniqueness of the compound’s MOA, and potential target populations for deployment (7,8).
This expert group has evaluated over 140 preclinical agents submitted through the survey. Many submissions were too early in the process to meet the timelines or would not be an appropriate candidate for one of ACTIV’s MPs; however, 25 were found to possess promising characteristics. In a related effort, the Preclinical WG assembled an inventory of research testing facilities that had the capacity and capability to carry out SARS-CoV-2 testing. In the final step of the preclinical process, the sponsors of these 25 agents have been connected to appropriate research facilities, and additional preclinical studies are being conducted to allow for clinical advancement (7,8).
The Preclinical and TX-Clinical WGs connected in order to expedite data accumulation on behalf of promising preclinical nominees. For those compounds assessed by the Preclinical WG as high priority based on the above criteria, a “matchmaking” process connected submitters with resources to address identified gaps and thereby prepare nominees for further evaluation by the TX-Clinical WG (Fig. 8).
CHALLENGES AND LESSONS LEARNED
Faced with a rapidly expanding pandemic caused by a novel pathogen with significant morbidity and mortality while lacking any known treatments of proven efficacy, rapid identification of potential therapeutic candidates was imperative. Drawing on broad scientific expertise, the AP subgroup developed a systematic, mechanism-based prioritization that accommodated evolving understanding of disease pathogenesis. Focusing on agents that could be repurposed, the first wave of the AP process aimed to identify candidate interventions for expedited testing in clinical trials. Immediate responses emerged from rapid consensus on triage and scoring criteria that enabled pragmatic prioritization of a few dozen candidate agents among the many hundreds of submissions.
However, the initial implementation of this AP process was not without its challenges. The first was the pace at which the review was needed, which in the beginning did not always allow for compilation and evaluation of all of the data the AP subgroup would have wished to have for each of the agents under consideration. This meant decisions had to be made on the best data that could be found rapidly from various sources, while taking into consideration that we were facing a new disease. The second was the speed at which the AP subgroup needed to get to know the task they were being asked to complete, as well as getting to know and trust each other. Given the time pressures, the team was often getting familiar with each other while working through critical decisions for the PPP. To help address this for future pandemic situations, it may be helpful for the infectious disease community to form a standing evaluation group that tracks important agents in development for these types of diseases.
The next challenge the AP subgroup faced was the progressively detailed understanding of COVID-19 disease pathogenesis and patterns; the changes in standard of care related to emerging clinical data on the effectiveness of remdesivir, corticosteroids, and anticoagulants in severely and critically ill patients; and the worldwide growth in clinical trials. It became progressively harder to successfully complete clinical trials in certain regions due to shifting epidemiology, making it harder for compounds to have collected preliminary evidence in COVID-19 patients. These factors understandably altered the prioritization process. Given this changing landscape, the group emphasized widening the candidate agent scope in the second wave of AP. The shift required three changes: triage and scoring criteria were refined, additional subject matter experts were recruited, and proposers were asked to submit greater detail not only about the candidate agent but also about the rationale and evidence supporting the candidate agent.
Another challenge that emerged as the pandemic progressed was the public pressure to test certain compounds, in particular widely available repurposed agents, that did not always have the rigorous early evidence to suggest effectiveness in treating or preventing COVID-19, the AP subgroup required of high-priority candidates. The group again adapted the triage and scoring criteria to evaluate these agents, adding criteria to account for broad public request for testing and nontraditional considerations, such as real-world evidence and case series data (Supplemental Digital Content 6, https://links.lww.com/CCM/G652; legend, https://links.lww.com/CCM/G653). In addition, additional subject matter experts were recruited to evaluate these new parameters and ensure that broad and scientifically robust perspectives on existing data were taken into account. For future pandemic efforts, if available, real-world evidence that suggests available agents already in use clinically with potential for treating a new disease could be considered earlier in the prioritization process.
There are at least three dynamics to consider in pandemic response AP. First, as knowledge was gained about the biology of the virus and host response, new agents needed to be nominated, and prior nominees were revisited. Second, the landscape of studies and study sites was (and remains) constantly shifting, requiring a level of global situational awareness unprecedented in drug development and trial design. Third, AP WGs such as this necessarily had to “learn by doing” and do so in a way that constantly attends to maintaining fair and objective evaluation standards and processes.
The AP process was enabled by the unique PPP—ACTIV—that brought together expertise spanning the federal government (including scientists, clinicians, administrators, and regulators), industry, and other funding entities. What made it work? First, none of these individuals were conscripted. Although membership was by nomination and invitation, all volunteered to serve ACTIV in addition to their “day jobs.” Second, all members were empowered as representatives of—not for—respective constituencies to make the best recommendation irrespective of whether it might “help” or “hurt” those who granted them leave to serve. Third, all members were encouraged to use their expertise however they thought it might best help the common effort. The pandemic united all entities toward a common goal—to defeat the global threat to human health—and it ceased to matter which federal agency, which regulatory body, and which industrial concern or individual investigator came forward first or finally with the best insight. Each contributor worked as if their life—and their families’ lives and their neighbors’ lives—depended on it.
PPPs in science are not new. In 1944, President Roosevelt asked, “What can the Government do now and in the future to aid research activities by public and private organizations (9)? The proper roles of public and of private research, and their relationship, should be carefully considered.” ACTIV shows the importance of these partnerships in addressing urgent national health security priorities. Development of the AP process described here leaves us better prepared for the next pandemic. Sustaining the PPP will be critical to our national response to this pandemic and to our future preparedness.
The authors thank for the contribution and support from members of the Accelerating Coronavirus Disease 2019 Therapeutic Interventions and Vaccines (ACTIV) Therapeutics-Clinical Working Group: John Beigel (National Institutes of Health), Sam Bozzette (National Institutes of Health), Timothy Burgess (Uniformed Services University of the Health Sciences), Joan Butterton (Merck & Co), Sylva Collins (Food and Drug Administration), Christine Colvis (National Institutes of Health), Michelle Culp (National Institutes of Health), Judy Currier (University of California, Los Angeles), Angelo R. De Claro (Food and Drug Administration), Mark Eisner (Genentech), Ellen Gadbois (National Institutes of Health), Carl Garner (Eli Lilly and Company), Keith Gottesdiener (Prime Medicine), Elizabeth Higgs (National Institutes of Health), David M. Hone (Defense Threat Reduction Agency), Keith W. Hoots (National Institutes of Health), Andrei Kindzelski (National Institutes of Health), Jacqueline Kirchner (Bill and Melinda Gates Foundation), Walter Koroshetz (National Institutes of Health), Lisa LaVange (University of North Carolina at Chapel Hill), Elliot Levy (Amgen), Joanne Lumsden (National Institutes of Health), Hilary Marston (National Institutes of Health), John W. Mellors (University of Pittsburgh), Sandeep Menon (Pfizer), Ashley Parker (National Institutes of Health), Naimish Patel (Sanofi/AbbVie), Amy Patterson (National Institutes of Health), Amanda Peppercorn (GlaxoSmithKline), Martha Petrovick (Massachusetts Institute of Technology), Revell L. Phillips (Defense Threat Reduction Agency), Mike Poole (Bill and Melinda Gates Foundation), Michael Proschan (National Institutes of Health), Lora A. Reineck (National Institutes of Health), Ronald B. Reisler (Defense Threat Reduction Agency), Daniel Rubin (Food and Drug Administration), Yuan Li Shen (Food and Drug Administration), Christine Sizemore (National Institutes of Health), Peter Stein (Food and Drug Administration), David Wholley (Foundation for the National Institutes of Health), Peter Wung (Sanofi/AbbVie), John Young (Roche), Helen Chen (Deloitte Consulting, LLP), and Hana Nasr (Deloitte Consulting, LLP).