Secondary Logo

Journal Logo

Commentary

Meta-Analyses Do Not Establish Improved Mortality With Ivermectin Use in COVID-19

Rothrock, Steven G. MD1,2,*; Weber, Kurt D. MD3; Giordano, Philip A. MD3; Barneck, Mitchell D. MD3

Author Information
doi: 10.1097/MJT.0000000000001461
  • Free

Ivermectin has been identified as an inexpensive, readily available drug with the potential to be repurposed as a treatment for COVID-19, especially in countries with limited access to vaccines. Although multiple studies have been published in an attempt to evaluate its usefulness in COVID-19, many are small and not constructed appropriately to detect differences in important clinical outcomes (ie, death). For this reason, researchers have turned to meta-analyses to combine study results and draw summary conclusions regarding ivermectin's effectiveness. Two such meta-analyses recently published in the American Journal of Therapeutics concluded that ivermectin decreased mortality and improved other surrogate end points in COVID-19.1–4 A recently withdrawn article caused both authors to rework their meta-analyses without altering their main conclusions.1–5 We feel that shortcomings within both sets of meta-analyses and limitations in the component studies are significant enough to invalidate their main finding that ivermectin reduces mortality. A review of other meta-analyses on the same subject, containing many of the same individual studies, were similarly limited by poor design.

KORY AND MARIK META-ANALYSES

In their updated meta-analysis, the authors missed an opportunity to improve methodologic weaknesses within their original study.1,2 In both the original and follow-up meta-analyses, the authors did not list standard methodological items (eg, Preferred Reporting Items for Systematic Reviews-Metanalysis) and had no description of statistical techniques.6 Absence of these descriptors makes meta-analyses nonreproducible and nonreplicable, thereby limiting external validity.7 Their absence automatically classifies this meta-analysis as having critically low quality.8,9

The authors implied their follow-up was performed to exclude a retracted study. However, without explanation, the authors deleted a second study while adding 2 others.10 One newly added study was unblinded and a second enrolled patients with negative PCR tests and did not control for underling comorbid risks.11,12 In a larger omission, the authors did not include 3 double-blind placebo-controlled trials showing no effect on mortality—biasing results in favor of ivermectin.13–15

In their follow-up, the authors did not correct errors from the original meta-analysis that overstated ivermectin's mortality benefit. In their description of the study by Ravikirti et al, mortality was described as “improved” with ivermectin compared with controls “0% versus 6.9%, P = 0.019.”1,16 However, there was no statistical difference in mortality (0/55 vs. 4/57, Fisher exact P value = 0.12). A mortality benefit with ivermectin in severely ill patients was stated to be “of borderline statistical significance, 0% (0/11) versus 27.3% (6/22), P = 0.052” in the description of Hashim's study.1,17 This was incorrect, as the Fisher exact test, P value is 0.08. The phrases “borderline significant,” “approached” and “nearly statistically significant” used in this meta-analysis have been described as inappropriate and misleading as “results do not include movement and cannot approach significance because of the dichotomous definition” of statistically significant.1,18,19

Inclusion of unblinded, single-blinded, and nonrandomized studies limited the authors' ability to calculate reliable, reproducible, pooled estimates of treatment effects. The largest study within the reworked meta-analysis comprising 39% of all cases and 48% of all deaths was a chart review where only 3.5% of patients were given ivermectin.20 Patients also received multiple other treatments in an uncontrolled manner (eg, steroids, tocilizumab) with no comparison of disease severity or comorbidity between the groups. In the study by Khan et al, nonrandomization resulted in a sicker control population (46% control vs. 10% study oxygen requirement).21 Nonrandomization led to different treatment (39.8% dexamethasone in ivermectin vs. 19.6% in control group) in the retrospective study by Rajter et al.22 The authors inappropriately adjusted for baseline risk between the groups by constructing a prediction model using binary logistic regression that overfitted their data using 13 variables to create a model that predicted 53 outcomes.22 The ratio of studied prediction variables to outcome should be, at most, 1:10 or 1:20.23

In other studies, described as double-blind, randomized controlled trials (RCTs), study and control groups differed. In the study by Niaee et al,24 control patients had fewer positive PCR tests, lower oxygen saturations, higher body mass indices (BMI), and more initial CT scans compared with ivermectin patients. The authors miscalculated P values when comparing PCR positivity and initial CT performance between the groups. The values should be <0.001 for both comparisons and not 0.421, 0.527 as reported.24 Vital sign data appear to be in error as a diastolic blood pressure median and interquartile range of 80 mm Hg (80–80) was reported in 4 of 6 groups in the study. It is unlikely that half of all patients (60/120 patients) in the first 4 groups had the exact same diastolic blood pressure. In the study by Mahmud et al, 32 cases were lost to follow-up (LTFU).25 Study validity is affected when LTFU is >5% (9% in this study), LTFUs exceed the outcome (32 LTFUs vs. 3 deaths) or a worst-case scenario imputation alters results (ie, all study LTFU and no control LTFUs have adverse outcome).26–28 This imputation would result in 8.2% mortality in ivermectin cases versus 1.7% in controls (P < 0.01) and render the overall mortality difference between the groups nonsignificant [odds ratio, 0.48; 95% confidence interval (CI) 0.21–1.07, RevMan 5.4; MedCalc 19.7, Osteen Belgium] within this meta-analysis.

BRYANT META-ANALYSES

Bryant's meta-analyses corrected many methodological shortcomings within the first set of meta-analyses by including many Preferred Reporting Items for Systematic Reviews-Metanalysis items. The authors, however, did not appear to register their meta-analysis, a critical item that categorizes a meta-analysis as low quality.8,9 Study registration improves transparency and minimizes the risk of selective outcome reporting bias. Selection bias may have been introduced when authors of 2 previous pro-ivermectin (or pro ivermectin) meta-analyses were contacted to aid in identifying additional articles for this analysis. The authors did not list inclusion or exclusion criteria within their methods only stating that they searched for RCTs. Their final included articles were described as double blind, open label, and quasi-RCTs. Some experts believe quasiexperimental studies are not RCTs, thus, requiring exclusion because true randomization is absent.29,30

Many studies were undersized with no sample size calculations for any outcome in 6 studies with other studies calculating samples sizes to detect differences in surrogate end points: clinical recovery (n = 5), viral clearance (2), and CIs around combined group mortality (1).3,4 No study was constructed to detect mortality differences between ivermectin and control groups. Eleven studies were small (<100 patients) increasing the likelihood that findings are false and inflating the effects of treatments.31,32 Further misinterpretation of results can occur in undersized studies that make multiple comparisons without any statistical corrections. For example, Okumus et al and Shahbaznejad et al conducted 69 and 58 statistical comparisons in studies with only 69 and 60 total patients, respectively.11,12

Nine of 14 studies were described as having adequate random sequence generation and allocation concealment.3 Failed randomization/allocation occurred in 2 of these “adequate” studies with a different viral cycle threshold plus different polymerase chain reaction (PCR) positivity, BMI, and oxygen saturation between ivermectin and control groups.24,33 In 5 studies described as not having adequate allocation, one had different baseline SOFA scores and 4 did not analyze comorbidity or other treatments between the groups.11,13,14,17,34 These confounding biases can alter the results of individual studies, thus, altering results of the meta-analysis.

The authors describe 6 of 14 studies as having an unclear or high risk for inadequate blinding.3 They state blinding is “less important” for evaluating evidence related to mortality.3 We disagree. Inadequate blinding can alter treatments, aggressiveness of care, and other actions taken to manage patients potentially resulting in different outcomes. Five studies lacked placebo controls,11,12,17,33,35 4 contained nonplacebo controls (ritonavir/lopinavir, doxycycline, chloroquine, hydroxychloroquine),12,13,33,35 and 3 added doxycycline to ivermectin treatment groups.17,25,36 Studies with active or nonplacebo controls, or that add drugs to treatment groups, do not allow ivermectin's effects to be isolated from those of the other drugs. Moreover, it is possible that active controls might worsen outcomes, falsely skewing results toward ivermectin's effectiveness.

Several other aspects of studies within each set of meta-analysis limits the ability to draw summary conclusions. Variable ivermectin doses, duration of ivermectin use, admission status, variable testing or no COVID-19 testing, and timing of drug administration during the disease course obscure the interpretation of results. Mixing studies with varying disease severity definitions further limits the ability to make conclusions. Moreover, inclusion of 2 studies enrolling children was inappropriate because mortality differs dramatically between adults and children.12,17

In their original meta-analysis, Bryant et al3 performed sensitivity analyses to assess robustness of their results by performing trial sequential analysis to confirm that ivermectin reduced mortality. Reanalysis of the study by Niaee et al showed that it should have been recategorized as having a high risk of bias and excluded during sensitivity analysis.24 This study had data and statistical errors previously described and included PCR-negative patients with different between-group PCR positivity, BMI, oxygen saturation, and initial CT performance, indicating randomization failure. In our opinion, these issues require removal of this article from any meta-analysis. Importantly, this was the only article within the Bryant meta-analysis showing that ivermectin reduced mortality.4 Removal of this single article alters their results so that no significant difference now exists between ivermectin and control mortality (risk ratio, 0.73; 95% CI, 0.45–1.18).

OTHER META-ANALYSES

We performed a literature search (EMBASE, PubMed, Web of Science, bioRxiv, medRxiv, Google Scholar, January 12,020 to October 15, 2021) and found 18 additional meta-analyses (now 20 total) that evaluated ivermectin's effect on COVID-19 mortality (Table 1).37–54 There was substantial overlap of studies within meta-analyses. Fourteen of 18 had the majority of their included studies contained within the Bryant or Marik meta-analyses (Table 1). Eleven concluded that ivermectin prevented mortality (Table 1). Removing the withdrawn study by Elgazzar et al left 9 and removing the study by Niaee et al left only 5 meta-analyses concluding that ivermectin is effective (Table 1). These 5 include Marik's meta-analysis (previously critiqued), meta-analyses by Kow and Nardlli whose individual studies overlap completely with Marik's and Bryant's meta-analyses, a meta-analysis by Karale et al that found no mortality difference in subsets with RCTs or without active controls, and a 3-study meta-analysis by Padhy et al with the quality of evidence, Cochrane GRADE, rated as very low.2,4,5,24,44,46–48

Table 1. - Meta-analyses evaluating Ivermectin's association with mortality in patients with COVID-19.
Author* publication date Mortality studies (without Elgazzar) Number (%) overlap with revised Bryant Number (%) overlap with revised Marik Ivermectin and overall mortality Ivermectin and mortality excluding Elgazzar§ Ivermectin and mortality excluding Elgazzar/Niaee§ AMSTAR 2 meta-analysis quality
Siemeniuk51
July 30, 2020
Living meta-analysis, updates periodically
8 (7) 7/7 (100%) 3/7 (43%) Network relative estimate
0.31 (0.14 to 0.72)
RR, 0.33 (0.14 to 0.77) RR, 0.54 (0.21 to 1.36) Mod/high
Padhy48
November 23, 2020
3 0 2/3 (67%) OR, 0.53 (0.29 to 0.96) NA NA Critically low
Kim45
December 30, 2020
2 0 2/2 (100%) Moderate/severe
OR, 0.76 (0.25 to 2.33)
Critical (one study)
OR, 0.15 (0.04 to 0.57)
NA NA Mod/high
Castenada-Sabogal#,37
January 27, 2021
4 0 3 (75%) RR, 0.7 (0.31 to 2.28) NA NA Critically low
Kow46
March 29, 2021
6 (5) 5/5 (100%) 5/5 (100%) OR, 0.21 (0.11 to 0.42) OR, 0.27 (0.14 to 0.55) OR, 0.37 (0.16 to 0.89) Critically low
Nardelli47
May 8, 2021
7 (6) 6/6 (100%) 5/6 (83%) OR, 0.19 (0.1 to 0.34) OR, 0.27 (0.14 to 0.55) OR, 0.37 (0.16 to 0.89) Critically low
Hariyanto41
June 6, 2021
8 (7) 7/7 (100%) 5/7 (71%) RR, 0.31 (0.15 to 0.62)** RR, 0.42 (0.24 to 0.74) RR, 0.55 (0.3 to 1) Mod/high
Zein53
June 27, 2021
9 (8) 7/8 (88%) 5/8 (63%) RR, 0.39 (0.2 to 0.74) RR, 0.56 (0.34 to 0.93) RR, 0.6 (0.34 to 1.05) Low
Roman50
June 28, 2021
5 5/5 (100%) 2/5 (40%) RR, 0.37 (0.12 to 1.13) NA RR, 0.62 (0.24 to 1.65) Critically low
Hill42
July 6, 2021
11 (10) 9/10 (90%) 6/10 (60%) RR, 0.44 (0.25 to 0.77) RR, 0.57 (0.35 to 0.95) RR, 0.77 (0.51 to 1.16) Critically low
Popp49
Jul 28, 2021
4 4/4 (100%) 1/4 (25%) Moderate–severe RR, 0.6 (0.14 to 2.51) mild to RR, 0.33 (0.01 to 8.05) NA NA Mod/high
Izcovich43
August 21, 2021
12 (11) 9/11 (82%) 6/11 (55%) RR, 0.5 (0.28 to 0.88); RR, 0.96†† (0.58 to 1.96) RR, 0.64 (0.4 to 1.05) RR, 0.84 (0.57 to 1.24) Mod/high
Bryant4
August 27, 2021
14 14/14 (100%) 6/14 (43%) RR, 0.51 (0.27 to 0.95) RR, 0.73 (0.45 to 1.18) Low
Marik2
September 2, 2021
10 6/10 (60%) 10/10 (100%) OR, 0.39 (0.25 to 0.6) NA OR, 0.44 (0.28 to 0.71) Critically low
Deng40
September 2, 2021
13 9/13 (69%) 4/13 (31%) OR 0.77 (0.5 to 1.19) NA NA Low
Cruciani39
September 8, 2021
8 (7) 7/7 (100%) 4/7 (57%) RD to 0.02 (−0.05 to 0.01) RD −0.01 (−0.04 to 0.01) NA Low
Cheng38
September 16, 2021
3 2/3 (67%) 1/3 (33%) RR, 0.53 (0.11 to 2.61) NA NA Critically low
Karale44
September 17, 2021
29 11/29 (38%) 10/29 (34%) OR, 0.54 (0.34 to 0.86) NA OR, 0.58 (0.36 to 0.92) Critically low
Zhang‡‡,54
September 28, 2021
10 6/10 (60%) 4/10 (40%) OR, 0.61 (0.37 to 1) NA OR, 0.87 (0.51 to 1.49) Critically low
Singh52
October 10, 2021
2 2/2 (100%) 2/2 (100%) OR, 0.45 (0.17 to 0.18) NA NA Low
*Dates may be inaccurate when publication on preprint server changes to online or print journal publication. Within meta-analyses, listed first authors for included individual studies may have differed between preprint and final published articles (eg, Cepelowicz = Rajter, Fonseca = Bermijo Galan = Galan, Shahbazneiad = Rezai, Ravikirti = Kirti, Beltran-Gonzales = Gonzales).
Number (%) of individual studies within these meta-analyses that are also contained within Marik and Bryant revised meta-analyses.
RD, RR, OR with 95% CIs in parenthesis.
§Recalculated RD, OR, RR with deletion of Elgazzar study and with deletion of both Elgazzar/Niaee studies,5,24 Multiple meta-analyses have different total mortality and total cases for the same individual studies. When available, the mortality and total numbers used by each meta-analysis were used for each re-calculation. NA within columns indicatesnot applicable because Elgazzar or Niaee article is not contained within these meta-analyses. RevMan 5.4 and MedCalc 19.7, osteen Belgium were used to calculate values repeating the same statistical technique used in each study.
Two reviewers (by consensus, disagreements to be settled by third reviewer–unnecessary) independently assessed each meta-analysis for the presence or absence of 7 critical items to classify its quality as critically low, low, or moderate/high.8,9 If all critical items are present, the meta-analysis is moderate or high quality. The absence of one item categorizes a meta-analysis as low quality and 2 or more items categorize an article as having critically low quality. (kappa = 0.72, 95% confidence interval, 0.49–0.94 for initial 2 reviewer agreement).
Cruciani left off 4 deaths in control group, adding these back in changes initial RD to −0.03 (−0.06 to 0.01).
#These authors treated the same article by Cepelowicz-Rajter [preprint (Cepelowicz) and final published article (Rajter)] as 2 separate studies when they were the same study. Thus, there were only 4 total studies and their initial calculations appear incorrect. Deleting the preprint version of the article alters their calculated OR to 0.68 (0.24–1.97).
**The authors mislabeled and transposed their mortality and PCR Forest plots within their article. Figure 2 (a) and 2 (b) descriptions are transposed.
††Excluding studies with a high risk of bias.
‡‡The 10 included articles within this ivermectin mortality analysis were identified by comparing the ivermectin studies within Table S3 (Detailed Trial Characteristics) with those within Table S5 [Evaluation of risk of bias (mortality)].
OR, odds ratio; RD, risk difference; RR, risk ratio (relative risk).

A quality appraisal of all 20 meta-analyses using the AMSTAR-2 tool found that 15 had low or critically low quality.8,9 The 5 meta-analyses concluding that ivermectin was effective after article exclusions all had critically low quality2,44,46–48 (Table 1). Others have noted similar low quality in a large number of meta-analyses and recommended remediation by following best-practice guidelines for meta-analyses and using anonymized individual patient data (IPD) obtained directly from study authors.55–57 Using IPD would increase transparency, improve data quality, offset inadequate reporting within studies, allow for better subgroup analysis, allow for standardizing common measures, and avoid inclusion of studies with a high risk of bias.

In-depth assessment of “other” meta-analyses indicates that many were not transparent, had inadequate inclusion/exclusion criteria, and lacked proper risk of bias assessments, across study quality assessments and sensitivity analyses. The same failures occurred in many of these meta-analyses. The combination of including flawed studies and using inadequate meta-analysis techniques rendered many of their conclusions unreliable and useless for clinical decision making.

Unfortunately, inadequate and improperly conducted studies of this drug have added to confusion both inside and outside of the medical community. In an August 26, 2021 health advisory, the Centers for Disease Control and Prevention reported a 24-fold increase in ivermectin prescriptions per week since the prepandemic period, indicating that a substantial number of physicians are prescribing this drug for COVID-19.58 In July 2021, the Centers for Disease Control and Prevention reported a 5-fold increase in calls to US poison control centers related to ivermectin with increased emergency department visits for adverse effects.58 A more recent report from Oregon detailed 21 poison center calls with patients developing seizures, hypotension, confusion, and ataxia and 4 requiring intensive care unit admission after ivermectin ingestion. Seventeen of 21 patients had purchased veterinary formulations, only 3 had received prescriptions and 1 had an unknown source for their drug.59 These reports indicate that despite its absence from NIH COVID-19 treatment guidelines and lack of United States Food and Drug Administration approval, off-label use of ivermectin by lay people and physicians has continued throughout the pandemic.58 It is possible that inconclusive ivermectin studies and meta-analyses contributed to this activity.

NEXT STEPS

Ultimately, imperfect studies need to be replaced by larger, adequately powered, double-blind, placebo-controlled trials, with meta-analyses, ideally using IPD, based on these studies. Assuming an overall COVID-19 case fatality rate of 2%, a study with 4638 patients (2319 each arm) would be needed to have 80% power (alpha 0.05, 1:1 enrollment) to detect a 1% or 50% relative drop in mortality from any intervention (eg, ivermectin).60 Assuming a sicker population (7% mortality, Bryant meta-analysis), a study with 1274 patients (637 each study arm) would be needed to have 80% power to detect a 3.5% (50% relative) drop in mortality. We await results of such studies before concluding that treatment with ivermectin can decrease mortality in all or a subset of patients with COVID-19.

REFERENCES

1. Kory P, GU Meduri, Varon J, et al. Review of the emerging evidence demonstrating the efficacy of ivermectin in the prophylaxis and treatment of COVID-9. Am J Ther. 2021;28:e299–e318.
2. Marik PE, Kory P. Ivermectin, a reanalysis of the data. Am J Ther. 2021;28:e579–e580.
3. Bryant A, Lawrie TA, Dowswell T, et al. Ivermectin for prevention and treatment of COVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Ther. 2021:e434–e460.
4. Bryant A, Lawrie TA, Fordham EJ. Ivermectin for prevention and treatment of COVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Ther. 2021;28:e573–e576.
5. Elgazzar A, Eltaweel A, Youssef SA, et al. Efficacy and safety of ivermectin for treatment and prophylaxis of covid-19 pandemic. Res Square. 2020. Preprint. doi: 10.21203/rs.3.rs-100956/v2. withdrawn
6. Page MJ, Cumpston M, Chandler J, et al. Chapter III: reporting the review. In: Higgins JP, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Review of Interventions Version 6.2. Available at: https://training.cochrane.org/handbook/current/chapter-iii. Accessed October 20, 2021.
7. National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in Science. Washington, DC: The National Academies Press; 2019.
8. Pieper D, Lorenz RC, Rombey T, et al. Authors should clearly report how they derived the overall rating when applying AMSTAR 2–a cross sectional study. J Clin Epidemiol. 2021;129:97–103.
9. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomized or non-randomised studies of healthcare interventions or both. BMJ. 2017;358:j4008.
10. Cadegani FA, Goren A, Wambier CG, et al. Early COVID-19 therapy with azithromycin plus nitazoxanide, ivermectin or hydroxychloroquine in outpatient settings significantly improved COVID-19 outcomes compared to known outcomes in untreated patients. New Microbe New Infect. 2021;43:100915.
11. Okumus N, Demirturk N, Cetinkaya RA, et al. Evaluation of the effectiveness and safety of adding ivermectin to treatment in severe COVID-19 patients. BMC Infect Dis. 2021;21:411.
12. Shahbaznejad L, Davoudi A, Eslami G, et al. Effects of ivermectin in patients with COVID-19: a multicenter, double-blind, randomized, controlled clinical trial. Clin Ther. 2021;43:1007–1019.
13. Beltran-Gonzalez JL, Gamez MG, Enciso EAM, et al. Efficacy and Safety of Ivermectin and Hydroxychloroquine in patients with severe covid-19. a randomized controlled trial. medRxiv. Preprint. Available at: https://www.medrxiv.org/content/10.1101/2021.02.18.21252037v1. Accessed February 23, 2021.
14. Chaccour C, Casellas A, Blanco-Di Matteo A, et al. The effect of early treatment with ivermectin on viral load, symptoms and humoral response in patients with non-severe COVID-19: a pilot, double-blind, placebo-controlled, randomized clinical trial. EClinicalMedicine. 2021;32:100720.
15. Lopez-Medina C, ELopez P, Hurtado IC, et al. Effect of ivermectin on time to resolution of symptoms among adults with mild COVID-19: a randomized clinical trial. JAMA. 2021;325:1426–1435.
16. Ravikirti RR, Roy R, Pattadar C, et al. Evaluation of ivermectin as a potential treatment for mild to moderate COVID-19: a double-blind randomized placebo controlled trial in Eastern India. J Pharm Pharm Sci. 2021;24:343–350.
17. Hashim HA, Maulood MF, Ali CL, et al. Controlled randomized clinical trial on using ivermectin with doxycycline for treating COVID-19 patients in Baghdad, Iraq. Iraqi JMS. 2021;19:107–115.
18. AMAStyleInsider. Bucking the “trend” and approaching “approaching significance.” JAMA Network. Available at: https://amastyleinsider.com/2012/06/18/bucking-the-trend-and-approaching-approaching-significance/. Accessed October 14, 2021.
19. Wood J, Freemantle N, King M, et al. Trap of trends to statistical significance: likelihood of near significant p value become more significant with extra data. BMJ. 2014;348:2215.
20. Budhiraja S, Soni A, Jha V, et al. Clinical profile of first 1000 COVID-19 cases admitted at tertiary care hospitals and the correlates of their mortality: an Indian experience. medRxiv. 2020. Preprint. Available at https://www.medrxiv.org/content/10.1101/2020.11.16.20232223v1. Accessed November 18, 2020.
21. Khan SI, Debnath CR, Al Mahtab M, et al. Ivermectin treatment may improve prognosis of patients with COVID-19. Arch Bronconeumol. 2020;56:813–830.
22. Rajter JC, Sherman MS, Fatteh N, et al. Use of ivermectin is associated with lower mortality in hospitalized patients with coronavirus disease 2019: the ivermectin in COVID nineteen study. Chest. 2021;159:85–92.
23. van der Ploeg T, Austin PC, Steyerbery EW. Modern modeling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Med Res Methodol. 2014;14:137.
24. Niaee MS, Namdar P, Allami A, et al. Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: a randomized multi-center clinical trial. Asian Pac J Trop Med. 2021;14:266–273.
25. Mahmud R, Rahman MM, Alam I, et al. Ivermectin in combination with doxycycline for treating COVID-19 symptoms: a randomized trial. J Int Med Res. 2021;49:1–14.
26. Bhandari M, Guyatt GH, Swiontkowski MF. User's guide to the orthopaedic literature: how to use an article about surgical therapy. J Bone Joint Surg Am. 2002;84:1672–1682.
27. Dettori JR. Loss to follow-up. Evid Based Spine Care J. 2011;2:7–10.
28. Shulz KF, Grimes DA. Sample size slippages in randomized trials: exclusions and the lost and wayward. Lancet. 2002;359:781–785.
29. Krass I. Quasi experimental designs in pharmacist intervention research. Int J Clin Pharm. 2016;38:647–654.
30. Miller CJ, Smith SN, Pugatch M. Experimental and quasi-experimental designs in implementation research. Psychiatry Res. 2020;283:112452.
31. Ioannidis JP. Why most published research findings are false. Plos Med. 2005;2:e124.
32. Pereira TV, Ioannidis JP. Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol. 2011;64:1060–1069.
33. Babalola OE, Bode CO, Ajayi AA, et al. Ivermectin shows clinical benefits in mild to moderate COVID19: a randomized controlled double-blind, dose response trial in Lagos. QJM. 2021:hcab035. doi: 10.1093/qjmed/hcab035.
34. Petkov S. Multicenter, randomized, double-blind, placebo-controlled study investigating efficacy, safety and tolerability of ivermectin HUVE-19 in patients with proven SARS-CoV-2 infection (Covid-19) and manifested clinical symptoms; 2021. Available at: https://clinicaltrialsregister.eu/ctr-search/trial/2020-002091-12/BG. Accessed October 20, 2021.
35. Galan LEB, Santos NMD, Asato MS, et al. Phase 2 randomized study on chloroquine, hydroxychloroquine or ivermectin in hospitalized patients with severe manifestations of SARS-CoV-2 infection. Pathog Glob Health. 2021;115:235–242.
36. Ahmed S, Karim MM, Ross AG, et al. A five day course of ivermectin for the treatment of covid-19 may reduce the duration of illness. Int J Infect Dis. 2020;103:214–216.
37. Castenada-Sabogal A, Chambergo-Michilot D, Toro-Huamanchumo CJ, et al. Outcomes of ivermectin in the treatment of COVID-19: a systematic review and meta-analysis. medRxiv. 2021. Preprint. doi: 10.1101/2021.01.26.21250420. Available at: https://www.medrxiv.org/content/10.1101/2021.01.26.21250420v1. Accessed January 27, 2021.
38. Cheng Q, Chen J, Jia Q, et al. Efficacy and safety of current medications for treating severe and non-severe COVID-19 patients: an updated network meta-analysis of randomized placebo-controlled trials. Aging (Albany NY). 2021;13:21866–21902.
39. Cruciani M, Pati I, Masiello F, et al. Ivermectin for prophylaxis and treatment of COVID-19: a systematic review and meta-analysis. Diagnostics (Basel). 2021;11:1645.
40. Deng J, Zhou F, Ali S, et al. Efficacy and safety of ivermectin for the treatment of COVID-19: a systematic review and meta-analysis. QJM. 2021. doi: 10.1093/qjmed/hcab247.
41. Hariyanto TI, Halim DA, Rosalind J, et al. Ivermectin and outcomes from COVID-19 pneumonia: a systematic review and meta-analysis of randomized clinical trials. Rev Med Virol. 2021:e2265. doi: 10.1002/rmv.2265. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8209939/.
42. Hill A, Garratt A, Levi J, et al. Meta-analysis of randomized trials of ivermectin to treat SARS-CoV-2 infection. Open Forum Infect Dis. 2021;8:ofab358.
43. Izcovich A, Peiris S, Ragusa M, et al. Bias as a source of inconsistency in ivermectin trials for COVID-19: a systematic review. medRxiv. 2021. Preprint. doi: 10.1101/2021.08.19.2126230. Available at: https://www.medrxiv.org/content/10.1101/2021.08.19.21262304v1. Accessed August 21, 2021.
44. Karale S, Bansal V, Makadia J, et al. An updated systematic review and meta-analysis, need for ICU admission, use of mechanical ventilation, adverse effects and other clinical outcomes of ivermectin treatment in COVID-19 patients. medRxiv. 2021. Preprint. doi: 10.1101/2021.04.30.21256415. Available at: https://www.medrxiv.org/content/10.1101/2021.04.30.21256415v2. Accessed September 17, 2021.
45. Kim MS, An MH, Kim WJ, et al. Comparative efficacy and safety of pharmacological interventions for the treatment of COVID-19: a systematic review and network meta-analysis. PLoS Med. 2020;17:e1003501.
46. Kow CS, Merchant HA, Mustafa ZU, et al. The association between the use of ivermectin and mortality in patients with COVID-19: a meta-analysis. Pharmacol Rep. 2021;73:1473–1479.
47. Nardelli P, Zangrillo A, Sanchini G, et al. Crying wolf in times of Corona: the strange case of ivermectin and hydroxychloroquine. Is the fear of failure withholding life-saving treatment from clinical use. Signa Vitae. 2021;17:3–4.
48. Padhy BM, Mohanty RR, Das S, et al. Therapeutic potential of ivermectin as add on treatment in COVID 19: a systematic review and meta-analysis. J Pharm Pharm Sci. 2020;23:462–469.
49. Popp M, Stegemann M, Metzendorf MI, et al. Ivermectin for preventing and treating COVID-19. Cochrane Database Syst Rev. 2021;7:CD015017.
50. Roman YM, Burela PA, Pasupuleti V, et al. Ivermectin for the treatment of COVID-19: a systematic review and meta-analysis of randomized controlled trials. Clin Infect Dis. 2021:ciab591. doi: 10.1093/cid/ciab591. Online ahead of print.
51. Siemeniuk RA, Bartoszko JJ, Ge L, et al. Drug treatments for COVID-19: living systematic review and network meta-analysis. BMJ. 2020;370:m2980.
52. Singh A, Sheth PG, Dhaneria S, et al. Efficacy and safety of ivermectin for COVID-19: a systematic review and meta-analysis. Asian Pac J Trop Med. 2021;14:440–450.
53. Zein AFMZ, Sulistiyana CS, Raffaelo WM, et al. Ivermectin and mortality in patients with COVID-19: a systematic review, meta-analysis, and meta-regression of randomized controlled trials. Diabetes Metab Synd. 2021;15:102186.
54. Zhang C, Jin H, Wen YF, et al. Efficacy of COVID-19 treatments: a Bayesian network meta-analysis of randomized controlled trials. Front. Public Health. 2021;9:729559.
55. Deeks JJ, Higgins JPT, Altman DG. Cochrane statistical methods group. Chapter 10: analyzing data and undertaking meta-analyses. In: Higgins JP, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Review of Interventions Version 6.2. Available at: https://training.cochrane.org/handbook/current/chapter-10. Accessed October 20, 2021.
56. Lawrence JM, Meyerowitz-Katz G, Healthers JAJ, et al. The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable. Nat Med. 2021;27:1853–1854.
57. Tierney JF, Steward LA, Clarke M. Cochrane individual participant data meta-analysis methods group. Chapter 26: individual participant data. In: Higgins JP, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Review of Interventions Version 6.2. Available at: https://training.cochrane.org/handbook/current/chapter-26. Accessed October 20, 2021.
58. CDC Health Alert Network. Rapid Increase in Ivermectin Prescriptions and Reports of Severe Illness Associatd with Use of Products Containing Ivermectin to Prevent or Treat COVID-19. Emergency Preparedness and Response. Centers for Disease Control and Prevention. Available at: https://emergency.cdc.gov/han/2021/han00449.asp. Accessed November 1, 2021.
59. Temple C, Hoang R, Hendrickson RG. Toxic effects from ivermectin use associated with prevention and treatment of COVID-19. N Engl J Med. 2021. 10.1056/NEJMc2114907 Online ahead of print.
60. World Health Organization. Weekly Epidemiological Update on COVID-19–26 October 2021. Available at: https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19---26-october-2021. Accessed October 31, 2021.
Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.