Objectives: Recent literature has drawn attention to the potential inadequacy of frequentist analysis and threshold p values as tools for reporting outcomes in clinical trials. The fragility index, which is a measure of how many events the statistical significance of a result depends on, has been suggested as a means to aid the interpretation of trial results. This study aimed to calculate the fragility index of clinical trials in critical care medicine reporting a statistically significant effect on mortality (increasing or decreasing mortality).
Data Sources: Literature search (PubMed/MEDLINE) to identify all multicenter randomized controlled trials in critical care medicine.
Study Selection: We identified 862 trials; of which 56 fulfilled eligibility criteria and were included in our analysis.
Data Extraction: Calculation of fragility index for trials reporting a statistically significant effect on mortality, and analysis of the relationship between trial characteristics and fragility index.
Data Synthesis: The median fragility index was 2 (interquartile range, 1–3.5), and greater than 40% of trials had a fragility index of less than or equal to 1. 12.5% of trials reported loss to follow-up greater than their fragility index. Trial sample size was positively correlated, and reported p value was negatively correlated, with fragility index.
Conclusions: In critical care trials reporting statistically significant effects on mortality, the findings often depend on a small number of events. Critical care clinicians should be wary of basing decisions on trials with a low fragility index. We advocate the reporting of fragility index for future trials in critical care to aid interpretation and decision making by clinicians.
1Intensive Care Unit, Wellington Regional Hospital, Wellington, New Zealand.
2Medical Research Institute of New Zealand, Wellington, New Zealand.
3Department of Intensive Care, Austin Hospital, Faculty of Medicine, the University of Melbourne and Australian and New Zealand Intensive Care Research Centre, Monash University School of Public Health and Preventive Medicine, Melbourne, VIC, Australia.
4Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy.
5Vita-Salute San Raffaele University, Milan, Italy.
*See also p. 1423.
Dr. Ridgeon, Dr. Young, Prof. Bellomo, and Dr. Landoni contributed to study design. Dr. Ridgeon and Dr. Landoni contributed to data collection and analysis. Dr. Ridgeon, Dr. Young, Prof. Bellomo, Dr. Mucchetti, Dr. Lembo, and Dr. Landoni contributed to drafting and approval of the article.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal).
The Medical Research Institute of New Zealand was supported by Independent Research Organization funding from the Health Research Council of New Zealand.
The authors have disclosed that they do not have any potential conflicts of interest.
For information regarding this article, E-mail: firstname.lastname@example.org