Re: “Commentary” by Rothrock et al. “Meta-Analyses Did Not Establish Improved Mortality With Ivermectin Use in COVID-19”

The “Commentary” by Rothrock et al¹ includes a critique of our Systematic Review² and Letter.³ They also introduce a misleading account of ivermectin safety and proposals for individual participant data (IPD) in meta-analysis that are impractical in a health emergency. We respond to the various matters in turn.

METHODOLOGY

Saying we “include[ed] many PRISMA items” is inaccurate. It would be appropriate to say we adhered fully to the PRISMA guidelines.⁴ The leading authors² have extensive experience in both process and methodologies for evidence synthesis; our structure and layout are consistent with many previous publications applying standard systematic review methodologies. These include Cochrane reviews, WHO and HTA monographs, and papers from completed RCTs. The work by Bryant et al² is an extensively peer-reviewed research paper and uses the same approaches found in most systematic reviews.

It is untrue that there was no a priori protocol. Citation 26 in our paper (2) references the protocol lodged with the Cochrane organisation without objection. Cochrane later rejected the work on January 25, 2021, on the ground that another group (now known to be Popp et al⁵) had registered an interest. The protocol was, therefore, published elsewhere and remains available.⁶ T.A.L. approached A.B. to lead the review as an independent academic researcher without prior awareness of ivermectin. The suggestion of biases from the lack of an a priori protocol is, therefore, rejected. We lodged it under assurances and cited it,² and it remains available.⁶

We did add, and document² (e438, Methods), 2 post hoc outcomes: improvement and deterioration. They were included as meaningful data were available, and both outcomes demonstrated a clear signal of clinical efficacy.

Contrary to claims,¹ inclusion criteria were stated in “Methods” section,² the a priori protocol,⁵ and implicitly in the “Design” column of Table 1,² indicating the quasi-randomized trials.

Experts were contacted for any ongoing or unpublished trials not identified in the electronic databases. This was a normal part of a wider search to ensure that all eligible trials were identified and does not represent selection bias.

STATISTICAL POWER, RANDOMIZATION, AND IMBALANCE OF COVARIATES

Including small and/or underpowered trials is not a bias in a meta-analysis. On the contrary, exclusion would be a strong source of publication bias. Direction of estimates can widely vary in small trials. Review authors are encouraged to identify and include small trials to reduce small trial biases such as publication bias.⁷ Excluding small trials would likely show asymmetry on a funnel plot and be an indicator that the results were biased.⁷ Through our conscious effort to include all available eligible evidence,² this bias was minimized.

Rothrock et al¹ made a similarly fallacious argument regarding inclusion of trials making multiple comparisons. The review team² only extracted data for outcomes for our comparison of “ivermectin” versus “no ivermectin.” Whether the primary sources overanalyze their data or not is immaterial. Their primary data were the input to our meta-analysis. “Cherry picking” of trials and selective reporting of outcomes would only have created the biases already alluded to.

These criticisms¹ would condemn the entire library of systematic reviews or evidence syntheses. This is at odds with evidence synthesis being at the top of the hierarchical pyramid of research evidence.⁸

Similarly, in small trials, statistical adjustment to effect estimates on grounds of unbalanced covariates would not be sensible. In a meta-analysis that includes a number of small trials, it is expected that any imbalances between treatment arms would tend to balance out across the meta-analysis. This is a common issue across all meta-analyses.

BLINDING

We contend that blinding is unlikely to matter for objective outcomes, such as all-cause mortality.⁹ Disputing this, Rothrock et al¹ listed some generic concerns but failed to make any compelling argument or add supporting references.

The implication that clinicians might alter management tactics is improbable when there were few available interventions of known efficacy. It seems unlikely that there would have been a lack of equipoise, especially when other potentially active interventions were comparators. We do not dismiss all concerns¹ regarding blinding but cannot view them as a serious threat to the integrity of the evidence for objective outcomes.

PICO QUESTIONS AND CHILD PARTICIPANTS

The conjecture that active controls might worsen outcomes, thereby falsely skewing results in favour of ivermectin, is imaginative, but defies logic, unless the “active” control is in reality “exacerbating.” No such example is given. The reverse trend would be expected, that is, “active” controls might result in understating ivermectin efficacy, a possibility we acknowledge.

We have addressed elsewhere¹⁰ the rationale and need for a pragmatic review in this area, in response to the selective and minimalistic Cochrane review of Popp et al⁵ and specifically the effect of potentially active controls. We based our review² on the comparison of “ivermectin” versus “no ivermectin” on pragmatic grounds. Ivermectin is an experimental drug in an efficacy sense for COVID-19, and at the time many of the trials were conducted, very little was known about treatment. In these circumstances, lack of a placebo control or use of standard of care or best practice is perfectly reasonable, both on ethical grounds for the practicing clinicians and for subsequent systematic reviews of the outcomes reported.

Although interpretation of a meta-analysis based on such a PICO question may require thought and cannot easily address heterogeneous matters such as dose or adjuncts, we reject the idea that it “limits the ability to draw summary conclusions.” Our findings showed a clear signal of efficacy in all major outcomes without the need to trade-off with morbidity or safety concerns.

Rothrock et al¹ complained that 2 trials included children. One¹¹ reports 9 of 69 participants aged 18 years or younger, and the other¹² reports ages 16–86 years over 70 patients. Nonadult participants would be 3.5 standard deviations from the mean. There were thus around 10 “children” in 2438 participants (0.4%). Children and young adults in general are well-known to not have significant mortality risk¹³; there is no abrupt transition at age 18 years. Because few trials were specifically age-stratified, age-dependent mortality was not evaluated, and an arbitrary lower age would have been another unwarranted bias.

EFFECT OF DISPUTED STUDIES AND UPDATING POLICY

Regarding updates, several strategies are available: Main options being a living review¹⁴ continuously updated when new trial results become available, updating after a specified period (The Cochrane Collaboration policy is that reviews should either be updated within 2 years or include a commentary to explain why this is not the case)¹⁵ or a trial sequential analysis¹⁶ to ascertain the likelihood of further trials actually being required. We opted for the latter policy. We have previously described³ a sensitivity analysis excluding 1 disputed study.¹⁷ We are, of course, monitoring the situation and will publish an updated systematic review and meta-analyses when we next update the searches. This will include all new eligible trials, as well as exclude resolved disputes. The Niaee trial¹⁸ is criticized for unbalanced controls, but the Bayesian analysis of Neil and Fenton¹⁹ already shows a robust conclusion of ivermectin efficacy in mortality, after the exclusion of both trials (in the case of Niaee because unstratified by severity). However, we take the allegations made by Rockrock et al extremely seriously. If the matter has not been resolved by time of submission of our updated evidence synthesis and meta-analyses, that will additional studies from April 2020 to present, we will of course omit the Niaee trial18 from the main analyses or exclude as appropriate.

IVERMECTIN SAFETY

We did not undertake to review the safety comprehensively because this is not best served by restricting to RCTs. The safety documentation for ivermectin is already comprehensive. As it is FDA-licensed for human use and a WHO Essential Medicine,²⁰ it has now accumulated literally billions of doses,²¹ many in WHO-sponsored Mass Drug Administration (MDA) campaigns.²²

The WHO pharmacovigilance database²³ reports 20 deaths (at June 24, 2021) associated with ivermectin over the past 25 years. These were most likely due to severe reactions to dying parasites²⁴ irrelevant to antiviral usage. Several previous reviews^24–26 all affirm the sound safety record of ivermectin; continuous high-dose ivermectin has been reported without incident in pediatric leukemia.²⁷ A recent expert toxicology report assessing over 200 studies concluded²⁸ as follows:

“… no severe adverse reactions have seemingly so far been described in relation to off-label studies or clinical trials of ivermectin as a potential prophylactic or curative treatment of COVID-19.

… ivermectin human toxicity cannot be claimed to be a serious cause for concern.”

Finally, Professor Chris Whitty (Chief Medical Officer for England) has written²⁹: “The drug has proven to be safe. Doses up to 10 times²⁵the approved limit are well tolerated by healthy volunteers. Adverse reactions are few and usually mild.”

However, Rothrock et al¹ ignored the previous literature, introducing media reports of self-medication with veterinary formulations. We did not endorse self-medication, certainly not with veterinary products. The insinuation that these cases resulted from our meta-analysis is completely unfounded. It is more plausible that they result from Health Regulatory Authorities (HRAs) obstructing access to regulatory compliant human formulations. Under proper clinical supervision of genuine contraindications, these are unlikely to do harm, even if efficacy is doubted.

Only one material source is given,³⁰ providing only sketchy data and identifying no clear association to ivermectin, rather than other factors such as the veterinary formulations. Against the extensive previous experience over billions of doses, this makes no serious contribution. Drug interactions, beyond our scope here, are a legitimate concern but are similarly well-documented.

Thus, Rothrock et al¹ rehearsed a propaganda narrative that is already overworked in media reports.³¹ Other such misleading reports include a claim³² that 70% of calls to a poison control center arose from veterinary formulations. In actuality, 2% of calls were for ivermectin, with only 4 for veterinary products,³³ leading to corrections in the international press. E-mails released by the FDA to journalists showed that the alert led directly to the FDA's notorious “Tweet”³⁴ about livestock formulations, prompting further exaggerated reports.³⁵ Such propaganda provides no forensic analysis.

The closure,¹ “It is possible that inconclusive ivermectin studies and meta-analyses contributed to this activity,” is ambiguous regarding “this activity.” If in reference to “off-label use,” prescribing outside the original licensing label is routine and unremarkable. If the insinuation is that our meta-analysis has contributed to self-medication with veterinary formulations, this is without basis and defamatory.

We fully endorse continued pharmacovigilance but contend that confused media reports provide minimal evidence of harm against the extensive previous safety literature.

IPD META-ANALYSIS

IPD meta-analyses^36–39 [IPDMA] are recognized as providing high-quality clinical evidence. They are also recognized as exceptionally time-consuming. An IPD can typically take many months to retrieve from trial administrators and can take in excess of 5 years.³⁹In addition, it may not be possible to complete an IPD review in a suitable time frame for the question of interest, and in some situations, the additional resource required may be prohibitive.³⁸

A call for IPDMA amounts to procrastination in a pandemic situation requiring rapid decision making. It defies the WHO policy regarding data-sharing during emergencies,⁴⁰ developed in the 2015 Ebola epidemic. Our meta-analysis was performed precisely to alert readers to the emerging results on the use of ivermectin in COVID-19, in a timely manner. Although the pandemic continues, no IPD analysis can be timely and only wastes resources while risking further loss of life.

We can only applaud data transparency. However, demanding it exclusively for reviews of negligible cost generics creates a conspicuous double standard. IPD is not disclosed for licensing of proprietary therapeutics^41,42 of much higher cost.

The estimated cost of the 480,000 courses of molnupiravir purchased⁴³ by the British Government is $336 million, at $700 per course. The same quantity of a 5-day ivermectin course could be as low as $288,000, at $0.60 per course.⁴⁴

Full-IPD disclosure is implausible for proprietary therapeutics seeking licensing. In this environment, suggestions that ivermectin data are invalid without IPD seem only as another tool to suppress information on an affordable and safe drug that would have global reach.