Comparative trials evaluating categorical outcomes have important implications on surgical decision making. The purpose of this study was to examine the statistical stability of sports medicine research.
Comparative clinical sports medicine research studies involving anterior cruciate ligament, meniscus, and knee instability were reviewed in two journals between 2006 and 2016. The statistical stability for each study outcome was determined by the number of event reversals required to change the P value to either greater or less than 0.05. The number of patients lost to follow-up was also determined.
Of the 1,505 studies screened, 102 studies were included for analysis, 40 of which were randomized controlled trials. There were 339 total outcome events, with 98 significant and 241 not significant. The Fragility Index, or the median number of events required to change the statistical significance of the overall study, was five (interquartile range, 3 to 8) or 5.4% of the total study population. In addition, the average number of patients lost to follow-up was 7.9, which is greater than the number needed to change the significance of each study arm and the entire study population.
Results in the comparative sports medicine literature may not be as stable as previously thought, with only a small percentage of outcome events needed to change study significance. Outcomes research based on a single discreet P value cutoff may be misleading.
From the Department of Orthopaedic Surgery, Boston Medical Center, Boston, MA (Dr. Parisien, Dr. Dashe, Dr. Curry, and Dr. Li), the Department of Orthopaedic Surgery, Columbia University Medical Center, New York, NY (Dr. Trofa), the Department of Orthopaedic Surgery, Massachusetts General Hospital & Harvard Medical School, Boston, MA (Dr. Cronin), and the Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA (Dr. Fu).
Correspondence to Dr. Li: firstname.lastname@example.org
Dr. Fu or an immediate family member is an employee of Medicrea and serves as a board member, owner, officer, or committee member of the American Academy of Orthopaedic Surgeons, the International Society of Arthroscopy, Knee Surgery, Orthopaedic Sports Medicine, and the World Endoscopy Doctors Association. None of the following authors or any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this article: Dr. Parisien, Dr. Trofa, Dr. Dashe, Dr. Cronin, Dr. Curry, and Dr. Li.