The Non-Inferiority Complex: What Do Non-Inferiority Trials Tell Us? : Journal of the American Society of Nephrology

Journal Logo

Perspective

The Non-Inferiority Complex: What Do Non-Inferiority Trials Tell Us?

Assimon, Magdalene M.1; Cutter, Gary R.2; Bargman, Joanne M.3

Author Information
JASN 33(4):p 674-676, April 2022. | DOI: 10.1681/ASN.2021050681
  • Open
  • SDC

In this issue of the JASN, Fishbane et al. add to the ever-growing noninferiority trial literature in the field of nephrology. They report the results of the ROCKIES trial, an international, phase 3, randomized, open-label, active comparator trial of 2133 dialysis patients evaluating if the hypoxia-inducible factor prolyl hydroxylase inhibitor roxadusat is noninferior to epoetin alfa for the treatment of anemia in dialysis-dependent ESKD.1 Despite the recent publication of other high-impact randomized controlled trials that used a noninferiority design (e.g., PIVOTAL, MENTOR, and INNO2VATE), some clinicians may not be familiar with noninferiority trial methodology. Therefore, in this commentary we briefly highlight important aspects of noninferiority trial design and provide an overview of noninferiority trial interpretation using the ROCKIES trial as an example.

Unlike “traditional” superiority trials, randomized controlled trials designed to show that a new treatment is better than a standard treatment or placebo, noninferiority trials are designed to show that a new treatment is not inferior to the standard treatment by a prespecified clinically acceptable amount called the noninferiority margin. This trial design is often used when a new treatment is expected to have similar efficacy or a clinically acceptable level of reduced efficacy compared to the current standard treatment, but offers other potential benefits such as more convenient administration, increased tolerability, or a more favorable safety profile.2 If noninferiority is established, the potential clinical utility of the new treatment is based upon the ancillary benefits it offers.2

The “null hypothesis” of a noninferiority trial is that the new treatment is worse than the standard treatment by more than the noninferiority margin. Rejecting this null hypothesis supports the claim that the new treatment is noninferior to the standard treatment. Therefore, one of the most important aspects of noninferiority trial design is margin selection. Selecting a noninferiority margin is often in the hands of the investigator and ideally involves sound statistical reasoning and good clinical judgment.3,4 In efficacy trials, such as the ROCKIES trial, the noninferiority margin is the maximum clinically acceptable extent that a new treatment can be less effective than a standard treatment.2,5 Like any other decision-making process, determining what constitutes a clinically acceptable loss of efficacy is inherently subjective. What one investigator thinks is reasonable may be considered unreasonable by another.

Consideration also needs to be given to the magnitude of the selected noninferiority margin. Noninferiority trials that use margins that are unreasonably large may conclude that the new treatment is statistically noninferior to the standard treatment when the new treatment is actually inferior. Notably, drug regulatory authorities, such as the U.S. Food and Drug Administration, provide clear guidance on this matter, stipulating that a noninferiority margin cannot be larger than the entire presumed effect of the comparator (i.e., standard) treatment.5 Thus, the noninferiority margin must be narrow enough to preserve a sufficient amount of the effect of the standard treatment. Choosing a noninferiority margin that preserves at least 50% of the standard treatment’s effect has become usual practice when evaluating the efficacy of new treatments.6 However, as previously mentioned, clinical context must also be considered.

Interpreting the results of a noninferiority trial depends on where the confidence interval (CI) of the treatment effect lies relative to both the noninferiority margin and the null effect. Figure 1 depicts the range of possible outcomes for noninferiority trials assessing a continuous outcome, such as the average change in a biomarker or laboratory parameter from baseline. For a new treatment to be considered noninferior to a standard treatment, the lower bound of the two-sided CI of the treatment effect must be greater (i.e., a higher number) than the prespecified noninferiority margin (–Δ).

F1
Figure 1.:
Possible conclusions from noninferiority trials. A conceptual illustration of the possible outcomes of a noninferiority trial assessing a continuous outcome, such as the average change in a biomarker or laboratory parameter from baseline. The solid vertical line at zero represents the null, that is, no difference between the new and standard treatments. The dashed vertical line represents the noninferiority margin, –Δ. Scenario A: the 95% CI of the treatment effect does not include zero and its lower bound lies to the right of (i.e., is greater than) both –Δ and zero. Therefore, the new treatment is both noninferior and superior to the standard treatment. Scenario B: the 95% CI of the treatment effect includes zero, but its lower bound lies to the right of (i.e., is greater than) –Δ. Therefore, the new treatment is noninferior to the standard treatment. Scenario C: the 95% CI of the treatment effect lies between –Δ and zero. Its lower bound lies to right of (i.e., is greater than) –Δ. Therefore, the new treatment is noninferior to the standard treatment. Its upper bound lies to the left of (i.e., is less than) zero. Therefore, the new treatment is also inferior to the standard treatment. Scenario D: the 95% CI of the treatment straddles both –Δ and zero. Its lower bound lies to the left of (i.e., is less than) –Δ and its upper bound lies to the right (i.e., is greater than) zero). Therefore, the results are inconclusive. Scenario E: the entire 95% CI of the treatment effect, including its lower and upper bounds, lies to the left of (i.e., is less than) zero and –Δ. Therefore, the new treatment is inferior to the standard treatment.

In the ROCKIES trial, roxadustat was compared with an active control, epoetin alpha. The primary efficacy outcome of interest was the mean change in hemoglobin from baseline to the average hemoglobin level during the evaluation period, weeks 28–52, regardless of rescue therapy use (e.g., red blood cell transfusion).1 Roxadustat was compared with epoetin alpha and the prespecified noninferiority margin for the primary efficacy outcome was –0.75 g/dL.1 While not explicitly stated, it is likely that this margin was selected in consultation with regulatory agencies, similar to other hypoxia-inducible factor prolyl hydroxylase inhibitor trials.7

Fishbane et al. found that the least squares mean (95% CI) change in hemoglobin concentration from baseline to the average level during weeks 28–52 was 0.77 (0.69 to 0.85) g/dl in the roxadustat group versus 0.68 (0.60 to 0.75) g/dl in the epoetin alpha group.1 The corresponding least means squares difference (95% CI) in hemoglobin change from baseline comparing the roxadustat group to the epoetin alpha group was 0.09 (0.01 to 0.18) g/dl.1 Because the lower bound of the 95% CI of this effect estimate, 0.01 g/dl, was greater than the noninferiority margin of –0.75 g/dl (i.e., 0.01 g/dl > –0.75 g/dl), the authors were able to conclude that roxadustat was noninferior to epoetin alpha with regards to the primary efficacy outcome. Analogous assessments evaluating where the 95% CI of the treatment effects lie relative to other prespecified outcome-specific noninferiority margins can be performed to interpret the results of secondary and exploratory endpoint analyses.

In addition to understanding how to interpret the results of noninferiority trials, it is important to be cognizant of common vulnerabilities that may impact their validity. The “noise” of high levels of drop-out, nonadherence, and treatment crossover may increase the likelihood that the new treatment “appears” similar to the standard treatment, potentially resulting in an erroneous claim of noninferiority.8,9

Noninferiority trials are being used more frequently to study the efficacy and safety of new kidney disease therapies. The design of noninferiority trials is complex, and it can be challenging to fully understand the results of these studies and their clinical implications. We hope that our commentary is a resource that clinicians can look to for guidance. When interpreting the results of a noninferiority trial, remember it is all about the margin.

Disclosures

M.M. Assimon is supported by R01 HL152034 awarded by the National Heart, Lung, and Blood Institute of the National Institutes of Health and also reports prior research funding from the Renal Research Institute (a subsidiary of Fresenius Medical Care, North America). In addition she has received honoraria from the International Society of Nephrology (KI Reports Statisitical Reviewer) and the American Society of Nephrology (JASN Editorial Fellow). G.R. Cutter reports consultancy agreements with the data and safety monitoring boards of Astra-Zeneca, Avexis Pharmaceuticals, Biolinerx, Brainstorm Cell Therapeutics, Bristol Meyers Squibb/Celgene, CSL Behring, Galmed Pharmaceuticals, Green Valley Pharma, Mapi Pharmaceuticals Ltd, Merck, Merck/Pfizer, Mitsubishi Tanabe Pharma Holdings, Opko Biologics, Neurim, NHLBI (Protocol Review Committee), NICHD (OPRU oversight committee), Novartis, Ophazyme, Sanofi-Aventis, Reata Pharmaceuticals, Teva pharmaceuticals, and VielaBio Inc; has a consulting or advisory ownership interest in Pythagoras, Inc. a private consulting company; is a JASN statistical editor and Neurology Clinical Practice contributing statistical editor, and is a member of the editorial boards for Multiple Sclerosis Journal and Multiple Sclerosis and Related Diseases. J.M. Bargman has served as a consultant for Akebia, Bayer, Davita Healthcare Partners, Glaxo Smith Kline, Novartis, and Otsuka; has received honoraria from Akebia, Amgen, Baxter Healthcare, Davita Healthcare Partners, and Glaxo Smith Kline; is a member of the editorial boards for JASN (Associate Editor), Peritoneal Dialysis International and CJASN; and has participated in a speakers’ bureau for Baxter Canada, Baxter Global, DaVita Healthcare Partners, and Glaxo Smith Kline.

Funding

None.

Published online ahead of print. Publication date available at www.jasn.org.

See related article, “Roxadustat versus Epoetin Alfa for Treating Anemia in Patients with Chronic Kidney Disease on Dialysis: Results from the Randomized Phase 3 ROCKIES Study,” on pages .

Acknowledgments

None.

Author Contributions

M. Assimon, G. Cutter, and J. Bargman conceptualized the study, were responsible for formal analysis and reviewed and edited the manuscript; M. Assimon wrote the original draft; and G. Cutter and J. Bargman provided supervision.

References

1. Fishbane S, Pollock C, El-Shahawy M, et al.: Roxadustat versus epoetin alfa for treating anemia in patients with chronic kidney disease on dialysis: Results from the randomized phase 3 ROCKIES study. J Am Soc Nephrol 33: 850–866, 2022
2. Kaul S, Diamond GA: Good enough: A primer on the analysis and interpretation of noninferiority trials. Ann Intern Med 145: 62–69, 2006
3. Fleming TR: Current issues in non-inferiority trials. Stat Med 27: 317–332, 2008
4. Mulla SM, Scott IA, Jackevicius CA, You JJ, Guyatt GH: How to use a noninferiority trial: Users’ guides to the medical literature. JAMA 308: 2605–2611, 2012
5. US Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER). Non-inferiority clinical trials to establish effectiveness-guidance for industry. 2016. Available at https://www.fda.gov/downloads/Drugs/Guidances/UCM202140.pdf. Accessed May 21, 2021
6. Althunian TA, de Boer A, Groenwold RHH, Klungel OH: Defining the noninferiority margin and analysing noninferiority: An overview. Br J Clin Pharmacol 83: 1636–1642, 2017
7. Eckardt KU, Agarwal R, Aswad A, Awad A, Block GA, Bacci MR, et al.: Safety and efficacy of Vadadustat for anemia in patients undergoing dialysis. N Engl J Med 384: 1601–1612, 2021
8. Mauri L, D’Agostino RB Sr: Challenges in the design and interpretation of noninferiority trials. N Engl J Med 377: 1357–1367, 2017
9. Mo Y, Lim C, Watson JA, White NJ, Cooper BS: Non-adherence in non-inferiority trials: Pitfalls and recommendations. BMJ 370: m2215, 2020
Keywords:

noninferiority trials; study design; nephrology

Copyright © 2022 by the American Society of Nephrology