The “Commentary” by Rothrock et al1 includes a critique of our Systematic Review2 and Letter.3 They also introduce a misleading account of ivermectin safety and proposals for individual participant data (IPD) in meta-analysis that are impractical in a health emergency. We respond to the various matters in turn.
Saying we “include[ed] many PRISMA items” is inaccurate. It would be appropriate to say we adhered fully to the PRISMA guidelines.4 The leading authors2 have extensive experience in both process and methodologies for evidence synthesis; our structure and layout are consistent with many previous publications applying standard systematic review methodologies. These include Cochrane reviews, WHO and HTA monographs, and papers from completed RCTs. The work by Bryant et al2 is an extensively peer-reviewed research paper and uses the same approaches found in most systematic reviews.
It is untrue that there was no a priori protocol. Citation 26 in our paper (2) references the protocol lodged with the Cochrane organisation without objection. Cochrane later rejected the work on January 25, 2021, on the ground that another group (now known to be Popp et al5) had registered an interest. The protocol was, therefore, published elsewhere and remains available.6 T.A.L. approached A.B. to lead the review as an independent academic researcher without prior awareness of ivermectin. The suggestion of biases from the lack of an a priori protocol is, therefore, rejected. We lodged it under assurances and cited it,2 and it remains available.6
We did add, and document2 (e438, Methods), 2 post hoc outcomes: improvement and deterioration. They were included as meaningful data were available, and both outcomes demonstrated a clear signal of clinical efficacy.
Contrary to claims,1 inclusion criteria were stated in “Methods” section,2 the a priori protocol,5 and implicitly in the “Design” column of Table 1,2 indicating the quasi-randomized trials.
Experts were contacted for any ongoing or unpublished trials not identified in the electronic databases. This was a normal part of a wider search to ensure that all eligible trials were identified and does not represent selection bias.
STATISTICAL POWER, RANDOMIZATION, AND IMBALANCE OF COVARIATES
Including small and/or underpowered trials is not a bias in a meta-analysis. On the contrary, exclusion would be a strong source of publication bias. Direction of estimates can widely vary in small trials. Review authors are encouraged to identify and include small trials to reduce small trial biases such as publication bias.7 Excluding small trials would likely show asymmetry on a funnel plot and be an indicator that the results were biased.7 Through our conscious effort to include all available eligible evidence,2 this bias was minimized.
Rothrock et al1 made a similarly fallacious argument regarding inclusion of trials making multiple comparisons. The review team2 only extracted data for outcomes for our comparison of “ivermectin” versus “no ivermectin.” Whether the primary sources overanalyze their data or not is immaterial. Their primary data were the input to our meta-analysis. “Cherry picking” of trials and selective reporting of outcomes would only have created the biases already alluded to.
These criticisms1 would condemn the entire library of systematic reviews or evidence syntheses. This is at odds with evidence synthesis being at the top of the hierarchical pyramid of research evidence.8
Similarly, in small trials, statistical adjustment to effect estimates on grounds of unbalanced covariates would not be sensible. In a meta-analysis that includes a number of small trials, it is expected that any imbalances between treatment arms would tend to balance out across the meta-analysis. This is a common issue across all meta-analyses.
We contend that blinding is unlikely to matter for objective outcomes, such as all-cause mortality.9 Disputing this, Rothrock et al1 listed some generic concerns but failed to make any compelling argument or add supporting references.
The implication that clinicians might alter management tactics is improbable when there were few available interventions of known efficacy. It seems unlikely that there would have been a lack of equipoise, especially when other potentially active interventions were comparators. We do not dismiss all concerns1 regarding blinding but cannot view them as a serious threat to the integrity of the evidence for objective outcomes.
PICO QUESTIONS AND CHILD PARTICIPANTS
The conjecture that active controls might worsen outcomes, thereby falsely skewing results in favour of ivermectin, is imaginative, but defies logic, unless the “active” control is in reality “exacerbating.” No such example is given. The reverse trend would be expected, that is, “active” controls might result in understating ivermectin efficacy, a possibility we acknowledge.
We have addressed elsewhere10 the rationale and need for a pragmatic review in this area, in response to the selective and minimalistic Cochrane review of Popp et al5 and specifically the effect of potentially active controls. We based our review2 on the comparison of “ivermectin” versus “no ivermectin” on pragmatic grounds. Ivermectin is an experimental drug in an efficacy sense for COVID-19, and at the time many of the trials were conducted, very little was known about treatment. In these circumstances, lack of a placebo control or use of standard of care or best practice is perfectly reasonable, both on ethical grounds for the practicing clinicians and for subsequent systematic reviews of the outcomes reported.
Although interpretation of a meta-analysis based on such a PICO question may require thought and cannot easily address heterogeneous matters such as dose or adjuncts, we reject the idea that it “limits the ability to draw summary conclusions.” Our findings showed a clear signal of efficacy in all major outcomes without the need to trade-off with morbidity or safety concerns.
Rothrock et al1 complained that 2 trials included children. One11 reports 9 of 69 participants aged 18 years or younger, and the other12 reports ages 16–86 years over 70 patients. Nonadult participants would be 3.5 standard deviations from the mean. There were thus around 10 “children” in 2438 participants (0.4%). Children and young adults in general are well-known to not have significant mortality risk13; there is no abrupt transition at age 18 years. Because few trials were specifically age-stratified, age-dependent mortality was not evaluated, and an arbitrary lower age would have been another unwarranted bias.
EFFECT OF DISPUTED STUDIES AND UPDATING POLICY
Regarding updates, several strategies are available: Main options being a living review14 continuously updated when new trial results become available, updating after a specified period (The Cochrane Collaboration policy is that reviews should either be updated within 2 years or include a commentary to explain why this is not the case)15 or a trial sequential analysis16 to ascertain the likelihood of further trials actually being required. We opted for the latter policy. We have previously described3 a sensitivity analysis excluding 1 disputed study.17 We are, of course, monitoring the situation and will publish an updated systematic review and meta-analyses when we next update the searches. This will include all new eligible trials, as well as exclude resolved disputes. The Niaee trial18 is criticized for unbalanced controls, but the Bayesian analysis of Neil and Fenton19 already shows a robust conclusion of ivermectin efficacy in mortality, after the exclusion of both trials (in the case of Niaee because unstratified by severity). However, we take the allegations made by Rockrock et al extremely seriously. If the matter has not been resolved by time of submission of our updated evidence synthesis and meta-analyses, that will additional studies from April 2020 to present, we will of course omit the Niaee trial18 from the main analyses or exclude as appropriate.
We did not undertake to review the safety comprehensively because this is not best served by restricting to RCTs. The safety documentation for ivermectin is already comprehensive. As it is FDA-licensed for human use and a WHO Essential Medicine,20 it has now accumulated literally billions of doses,21 many in WHO-sponsored Mass Drug Administration (MDA) campaigns.22
The WHO pharmacovigilance database23 reports 20 deaths (at June 24, 2021) associated with ivermectin over the past 25 years. These were most likely due to severe reactions to dying parasites24 irrelevant to antiviral usage. Several previous reviews24–26 all affirm the sound safety record of ivermectin; continuous high-dose ivermectin has been reported without incident in pediatric leukemia.27 A recent expert toxicology report assessing over 200 studies concluded28 as follows:
“… no severe adverse reactions have seemingly so far been described in relation to off-label studies or clinical trials of ivermectin as a potential prophylactic or curative treatment of COVID-19.
… ivermectin human toxicity cannot be claimed to be a serious cause for concern.”
Finally, Professor Chris Whitty (Chief Medical Officer for England) has written29: “The drug has proven to be safe. Doses up to 10 times25the approved limit are well tolerated by healthy volunteers. Adverse reactions are few and usually mild.”
However, Rothrock et al1 ignored the previous literature, introducing media reports of self-medication with veterinary formulations. We did not endorse self-medication, certainly not with veterinary products. The insinuation that these cases resulted from our meta-analysis is completely unfounded. It is more plausible that they result from Health Regulatory Authorities (HRAs) obstructing access to regulatory compliant human formulations. Under proper clinical supervision of genuine contraindications, these are unlikely to do harm, even if efficacy is doubted.
Only one material source is given,30 providing only sketchy data and identifying no clear association to ivermectin, rather than other factors such as the veterinary formulations. Against the extensive previous experience over billions of doses, this makes no serious contribution. Drug interactions, beyond our scope here, are a legitimate concern but are similarly well-documented.
Thus, Rothrock et al1 rehearsed a propaganda narrative that is already overworked in media reports.31 Other such misleading reports include a claim32 that 70% of calls to a poison control center arose from veterinary formulations. In actuality, 2% of calls were for ivermectin, with only 4 for veterinary products,33 leading to corrections in the international press. E-mails released by the FDA to journalists showed that the alert led directly to the FDA's notorious “Tweet”34 about livestock formulations, prompting further exaggerated reports.35 Such propaganda provides no forensic analysis.
The closure,1 “It is possible that inconclusive ivermectin studies and meta-analyses contributed to this activity,” is ambiguous regarding “this activity.” If in reference to “off-label use,” prescribing outside the original licensing label is routine and unremarkable. If the insinuation is that our meta-analysis has contributed to self-medication with veterinary formulations, this is without basis and defamatory.
We fully endorse continued pharmacovigilance but contend that confused media reports provide minimal evidence of harm against the extensive previous safety literature.
IPD meta-analyses36–39 [IPDMA] are recognized as providing high-quality clinical evidence. They are also recognized as exceptionally time-consuming. An IPD can typically take many months to retrieve from trial administrators and can take in excess of 5 years.39In addition, it may not be possible to complete an IPD review in a suitable time frame for the question of interest, and in some situations, the additional resource required may be prohibitive.38
A call for IPDMA amounts to procrastination in a pandemic situation requiring rapid decision making. It defies the WHO policy regarding data-sharing during emergencies,40 developed in the 2015 Ebola epidemic. Our meta-analysis was performed precisely to alert readers to the emerging results on the use of ivermectin in COVID-19, in a timely manner. Although the pandemic continues, no IPD analysis can be timely and only wastes resources while risking further loss of life.
We can only applaud data transparency. However, demanding it exclusively for reviews of negligible cost generics creates a conspicuous double standard. IPD is not disclosed for licensing of proprietary therapeutics41,42 of much higher cost.
The estimated cost of the 480,000 courses of molnupiravir purchased43 by the British Government is $336 million, at $700 per course. The same quantity of a 5-day ivermectin course could be as low as $288,000, at $0.60 per course.44
Full-IPD disclosure is implausible for proprietary therapeutics seeking licensing. In this environment, suggestions that ivermectin data are invalid without IPD seem only as another tool to suppress information on an affordable and safe drug that would have global reach.
1. Rothrock SG, Weber KD, Giordano PA, et al. Meta-analyses do not establish improved mortality with ivermectin use in COVID-19. “Commentary”. Am J Ther. 2021.
2. Bryant A, Lawrie TA, Dowswell T, et al. Ivermectin for prevention and treatment of covid-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Therap. 2021;28:e434–460.
3. Bryant A, Lawrie TA, Fordham EJ. Ivermectin for prevention and treatment of cOVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Ther. 2021;2828:e434–e460.
4. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
5. Popp M, Stegemann M, Metzendorf M-I, et al. Ivermectin for preventing and treating covid-19. Cochrane Database Syst Rev. 2021;7:CD015017.
6. Bryant A, Lawrie T, Dowsell T, et al. Ivermectin for Prevention and Treatment of Covid-19 (Protocol). Bath, England: The Evidence-Based Medical Consultancy Ltd; 2021. Available at: https://tinyurl.com/cx7pnaxa
. Accessed November 11, 2021.
7. Page MJ, Higgins JPT, Sterne JAC. Chapter 13: assessing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 6.0 (Updated July 2019). London, England: Cochrane Organisation; 2019. Available at: http://training.cochrane.org/handbook/current/chapter-13
8. Murad MH, Asi N, Alsawas M, et al. New evidence pyramid. Evid Based Med. 2016;21:125–127.
9. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.
10. Fordham E, Lawrie TA, MacGilchrist K, et al. The uses and abuses of systematic reviews, OSF preprint, 2021.
11. Shahbaznejad L, Davoudi A, Eslami G, et al. Effects of ivermectin in patients with COVID-19: a multicenter, double-blind, randomized, controlled clinical trial. Clin Ther. 2021;43:1007–1019.
12. Hashim HA, Maulood MF, Ali CL, et al. Controlled randomized clinical trial on using ivermectin with xycycline for treating COVID-19 patients in Baghdad, Iraq. Iraqi J Med Sci. 2021;19:117–115.
13. Spiegelhalter D. Use of normal risk to improve understanding of dangers of covid-19. BMJ. 2020;370:m3259.
14. Elliott JH, Synnot A, Turner T, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91:23–30.
15. Cumpston M, Chandler J. Chapter IV: updating a review. In: Higgins JPT, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021). London, England: Cochrane Organisation; 2021. Available at: http://training.cochrane.org/handbook/current/chapter-iv
16. Thorlund K, Engstrøm J, Wetterslev J, et al. User manual for trial sequential analysis (TSA); 2011. Available at: www.ctu.dk/tsa
. Accessed April 8, 2021.
17. Elgazzar A, Hany B, Youssef SA, et al. Efficacy and safety of ivermectin for treatment and prophylaxis of COVID-19 pandemic, 2020. Research Square preprint (withdrawn).
18. Niaee MS, Namdar P, Allami A, et al. Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: a randomized multi-center clinical trial. Asian Pac J Trop Med. 2021;14:266.
19. Neil M, Fenton N. Bayesian hypothesis testing and hierarchical modeling of ivermectin effectiveness. Am J Ther. 2021;28:e576–e579.
20. World Health Organisation. Model list of essential medicines, 21st list, 2019. Available at: https://apps.who.int/iris/bitstream/handle/10665/325771/WHO-MVP-EMP-IAU-2019.06-eng.pdf
21. Nicolas P, Maia MF, Bassat Q, et al. Safety of oral ivermectin during pregnancy: a systematic review and meta-analysis. Lancet Glob Health. 2020;8:E92–E100.
22. WHO. Preventive chemotherapy in human helminthiasis: coordinated use of anthelminthic drugs in control interventions: a manual for health professionals and programme managers. In: Crompton DWT, ed. World Health Organization Manual. World Health Organisation; 2006, ISBN 978 92 4 154710 9.
23. WHO. VigiAccess, WHO collaborating centre for international drug monitoring (Uppsala); 2021.
24. Navarro M, Camprubí D, Requena-Méndez A, et al. Safety of high-dose ivermectin: a systematic review and meta-analysis. J Antimicrob Chemother. 2020;75:827–834.
25. Guzzo C, Furtek C, Porras A, et al. Safety, tolerability, and pharmacokinetics of escalating high doses of ivermectin in healthy adult subjects. J Clin Pharmacol. 2002;42:1122–1133.
26. Kircik LH, Del Rosso JQ, Layton AM, et al. Over 25 years of clinical experience with ivermectin: an overview of safety for an increasing number of indications. J Drugs Dermatol. 2016;15:325–332.
27. de Castro CGJ, Gregianin LJ, Burger JA. Continuous high-dose ivermectin appears to be safe in patients with acute myelogenous leukemia and could inform clinical repurposing for COVID-19 infection. Leuk Lymphoma. 2020;61:2536–2537.
28. Descotes J. Expert review report: Medical safety of ivermectin; 2021. Report to MedinCell SA by ImmunoSafe consultancy, February 2021, 47pp. Available at: https://www.medincell.com/ivermectin/
29. Chaccour C, Lines J, Whitty CJM. Effect of ivermectin on Anopheles gambiae mosquitoes fed on humans: the potential of oral insecticides in malaria control. J Infect Dis. 2010;202:113–116.
30. Temple C, Hoang R, Hendrickson RG. Toxic effects from ivermectin use associated with prevention and treatment of covid-19. New Engl J Med. 2021;385:2197–2198.
31. Capuzzo M. A day after our investigative reporter Mary Beth pfeiffer proved that the FDA lied (and thousands likely died) with the ivermectin horse-dewormer hoax, she was locked out of twitter; 2021. Available at: https://rescue.substack.com/p/a-day-after-our-investigative-reporter
. Accessed October 18, 2021.
32. Mississippi State Dept of Health. Increased poison control calls due to ivermectin ingestion and potential toxicity; 2021. MS Health Alert Network Alert. Available at: https://msdh.ms.gov/msdhsite/_static/resources/15400.pdf
. August 20, 2021.
33. Pfeiffer MB, Bonvie L. Horse-bleep: how 4 calls on animal ivermectin launched a false FDA-media attack on a life-saving human medicine; 2021. Available at: https://rescue.substack.com/p/a2520b80-bcd1-4905-a913-68f6f6809779
34. U.S. FDA. You are not a horse, 2021. “Tweet”, 21 August 2021.
35. Mole B. Ivermectin overdose death probed in NM: “I don't want more people to die”, 2021. Ars Technica, 9/10/2021. Available at: https://arstechnica.com/science/2021/09/ivermectin-overdose-death-probed-in-nm-i-dont-want-more-people-to-die/
36. Cochrane Organisation. About IPD meta-analysis, 2020. Available at: https://methods.cochrane.org/ipdma/about-ipd-meta-analyses
37. Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Professions. 2002;25:76–97.
38. Tierney JF, Stewart LA, Clarke M. Chapter 26: individual participant data. In: Higgins JPT, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021). London, England: Cochrane Organisation; 2021. Available at: www.training.cochrane.org/handbook
39. Ventresca M, Schünemann HJ, Macbeth F. Obtaining and managing data sets for individual participant data meta-analysis: scoping review and practical guide. BMC Med Res Methodol. 2021;20:13.
40. Modjarrad K, Moorthy VS, Millett P, et al. Developing global norms for sharing data and results during public health emergencies. PLoS Med. 2016;13:1–5.
42. Medical and Healthcare Products Regulatory Agency. Summary of product characteristics for Lagevrio, 2021. Available at: https://www.gov.uk/government/publications/regulatory-approval-of-lagevrio-molnupiravir/summary-of-product-characteristics-for-lagevrio
. Accessed November 4, 2021.
43. Reed J. Molnupiravir: first pill to treat Covid gets approval in UK, 2021. BBC News report. Available at: https://www.bbc.co.uk/news/health-59163899
. Accessed November 4, 2021.
44. Ahmed S, Karim MM, Ross AG, et al. A five-day course of ivermectin for the treatment of COVID-19 may reduce the duration of illness. Int J Infect Dis. 2021;103:214–216.