Akshal S. Patel, MD
Department of Neurosurgery, Penn State Hershey Medical Center
Journal Club Article: Siddiq F, Chaudhry SA, Tummala RP, Suri MFK and Qureshi AI. Factors and Outcomes Associated With Early and Delayed Aneurysm Treatment in Subarachnoid Hemorrhage Patients in the United States. Neurosurgery. 2012; 71(3): 670-8.
Significance/context and importance of the study
Siddiq et al. (2012) utilize data from the Nationwide Inpatient Sample (NIS) to conduct a retrospective cohort study comparing early (< 48 hours) vs. late (> 48 hours) treatment of aneurysmal subarachnoid hemorrhage (aSAH). U.S. patients 18 years and older who presented with aSAH between 2005 and 2008 and were treated with either microsurgery or endovascular embolization were included. Primary outcome measures included length of hospital stay (LOS), discharge destination, hospital charges and mortality.
An increasing number of aSAH patients are undergoing endovascular therapy, and the percentage of patients receiving early therapy may also have increased. This study attempts to determine what variables are associated with early and delayed treatment and whether outcomes vary between these two treatment groups.
Originality of the work
The study by Siddiq et al. examines the timing of aneurysm treatment following aSAH, an issue that has not been thoroughly evaluated since the International Cooperative Study (ICS). The authors pose the hypothesis that because endovascular treatment can be delivered at the time of diagnostic angiography and because endovascular therapy may have a lower treatment risk than open surgery in the early post-SAH period, the interval from SAH to definitive treatment may have decreased substantially over the last couple decades.
Appropriateness of the study design or experimental approach
Though the information contained in the database is extensive and the NIS’ large sample size permits analysis of relatively rare diseases, it lacks the variables that a well-constructed, prospective registry would possess. The NIS contains no conventional identifiable SAH or stroke severity scales. Severity of the bleed is the most powerful predictor of outcome in patients with SAH. To estimate disease severity on presentation the authors make use of the 3M APR-DRG, which aggregates factors that represent level of illness and risk of mortality to construct a four point ordinal system. By contrast, the ICS utilized level of consciousness on admission to stratify patients. The distribution of patients within each bracket of the 3M APR-DRG scale is strikingly dissimilar to the distribution of level of consciousness reported in the ICT (p<0.001), suggesting that the 3M-derived scale is a poor surrogate for disease severity (compare figures 1 and 2).
The authors rely on ICD-9 codes to isolate the correct patient population for study, patient comorbidities, in-hospital procedures and periprocedural complications. Siddiq et al. recognize that ICD-9 coding fidelity is at best 86%. This reliance exposes the study to considerable risks of both selection and misclassification bias, and, consequently, the risk of both type I and type II errors. The ICD-9 coding system is also used to assign patients to the early and delayed treatment groups based on the time between diagnosis of SAH and treatment as documented in the dataset. The actual number of days post hemorrhage is not obtainable from the NIS. This may distort treatment timing in a non-random way, since the day of diagnosis may not be the day of SAH.
There is no information in the NIS database explicitly addressing functional independence. Siddiq et al. attempt to remedy this deficiency by using discharge destination as a surrogate for independence. This strategy is fraught with potential confounders, including the presence and competence of caretakers at home, insurance coverage, patient and family preference, and availability of rehabilitation beds. Discharge to home is not synonymous with minimal disability, and discharge to a rehabilitation facility for a brief period of structured support is common even for patients with minimal disability. The North American cohort of patients from the ICS does document functional capacity scores at discharge. There is a discrepancy (p<0.001) in the distribution of long term outcomes between these two studies, again suggesting the inadequacy of discharge destination as a surrogate measure (compare figure 3 and 4).
Finally, the adoption of 48 hours as the definition of “early treatment” in the current study complicates comparison with previous studies which consistently use 72 hours. The 48 hour criterion also increases the number of patients categorized as having “delayed treatment”.
Adequacy of experimental techniques
This study is a retrospective cohort comparison between two predefined groups of patients. The authors use the NIS database to detect statistically significant differences between the early and delayed treatment groups with regard to both patient and hospital characteristics. A probability value of 0.01 was considered statistically significant and a Bonferroni correction was used to minimize the risk of type I error. Despite these precautions, the risk of accepting an association between a patient characteristic and the timing of surgery when one does not really exist (a type I error) remains substantial. The authors report the results of 36 separate comparisons (Table 1). The NIS database would support potentially even more tests of association which are not reported. In addition, the very large size of the NIS database produces a substantial risk of detecting a statistically significant association of little clinical relevance.
Although the authors do not explicitly discuss the distribution of patients into early and late treatment groups, information included in the manuscript coupled with data from other sources does allow construction of a CONSORT-like patient flow diagram (figure 5). The Northern Manhattan Study estimates the annual incidence of aSAH in the United States to be 9.7 per 100,000 people. The NIS provides an estimate of 14.5 discharges for aSAH per 100,000 adults. Using these numbers, and assuming a U.S. population of 310 million, 30,070 to 44,950 patients experience aSAH per year. Thus Siddiq et al. have included for study only one quarter to one third of all patients with aSAH estimated to have developed this condition during the study period. This results in an important reduction in generalizability of their results. In addition, since one cannot predict, at the time of SAH, which patients will eventually undergo surgical treatment, the associations detected in this study become less useful. If the exclusion of patients from this study was non-random, a profound risk of selection bias is incurred.
Soundness of conclusions and interpretation (including analysis)
Siddiq et al. identify three associations related to early treatment in aSAH. First, patients treated within 2 days of aneurysmal SAH diagnosis were almost twice as likely to have undergone endovascular coil embolization. The authors speculate that admission to teaching hospitals may play a role in this difference, but treatment at a teaching hospital was one of the clinical characteristics analyzed between groups and did not predict early treatment (p=0.06).
The authors also report that female sex predicts early treatment and weekend admission predicts delayed treatment. Although the p-values for these associations achieve statistical significance, the confidence intervals for the odds ratios for these associations (1.03 – 1.28 for female sex, 0.64 – 0.85 for weekend admission) are relatively wide, and approach unity on their lower and uppers ends, respectively. In addition, the association with weekend admission may largely result from the authors’ definition of “early treatment” as treatment occurring within 48, rather than within 72 hours. Consequently, the clinical significance of these findings is less compelling.
Siddiq et al. have also constructed two models (1 and 2), using logistic regression to adjust for age, sex, medical comorbidities and APR DRG severity scale, to examine the effect of early vs. delayed treatment on functional outcome. The first model reports the likelihood of death with respect to timing of surgery. The second model compares the functional outcome of early vs. late treatment in patients who survived their hospitalization and were discharged. In-hospital mortality was higher in the early treatment group (maximally adjusted OR 1.36 (1.12 – 1.66), p = 0.002, model 1). On-the-other-hand, functional independence was greater in the early treatment group (maximally adjusted OR 1.30 (1.14 – 1.47, < 0.001, model 2). Although these results appear to be antagonistic to each other, the authors do hypothesize possible plausible explanations for the findings. Using the authors’ aggregate data (from their Table 1), a more intuitive estimation of the clinical relevance of these findings – the number needed to harm (NNH) and number needed to treat (NNT) – can be calculated. These calculations show that 33 patients (95%CI 26-45) would need to undergo early surgery to incur one additional in-hospital death, and 25 patients (95%CI 19-36) would need to undergo early surgery to achieve one additional patient discharged with none to minimal disability. However, these associations do not translate into robust predictors of outcome. At the time of presentation with aneurysmal SAH it is not known which patients will ultimately undergo a neurosurgical procedure, and whether that procedure will occur early or late. In part this is because the authors’ study is not prospective and randomized (and thus treatment “assignment” can be affected by bias as noted above, and by a host of unknown confounders not accounted for in the adjusted analysis).
The major medical comorbidities examined in the present study include hypertension, diabetes mellitus, congestive heart failure and renal failure. Three of these features occurred significantly more frequently (p = 0.005, 0.02, 0.74 and 0.02 respectively) in the delayed treatment group. Although only hypertension achieves the authors’ criterion level of statistical significance, these potential confounders of clinical outcome do reduce the reliability of any inferences made about differences in outcome between the early and late treatment groups.
Relevance of discussion
The authors’ discussion succinctly highlights the most interesting and clinically relevant findings of the paper, and the authors clearly appreciate the limitations of a retrospective cohort study using a large non-specific database. They outline a number of threats to reliability embedded in the study design, including use of the ICD-9 coding system, 3M APR-DRG severity score and the lack of neurologic assessments housed within the NIS. The authors do make inferences that suffer from the logical error of cum hoc ergo propter hoc – mistaking correlation with causation. They state that early treatment is associated with better outcomes, and that this association is largely driven by the increased application of endovascular interventions. The association with improved outcomes suffers from numerous threats to reliability, and, based on the authors’ data, can only be considered a hypothesis, not a basis for practice recommendations.
Clarity of writing, strength and organization of the paper
Data is presented clearly and commentary is lucid. The statistical tools utilized in this retrospective analysis are stated. Results are displayed with precision and do not overwhelm the reader.
Economy of words
The paper is focused and concise.
Relevance, accuracy and completeness of bibliography
The bibliography is thorough but excludes two important studies regarding the timing of surgery for aSAH. The Cochrane Database systematic review by Whitfield et al. in 2001 and the meta-analysis by de Gans et al. in 2002 both concluded that there is insufficient high-quality evidence in the literature to warrant concrete recommendations regarding early vs. delayed treatment.
Number and quality of the figures, tables and illustrations
Table 3 is confusing and would benefit from more explanation in the body of the paper.
Siddiq et al. have provided a comprehensive overview of the current state of management of aSAH in the United States. Based on their retrospective cohort study, the authors have generated a number of intriguing hypotheses regarding the predictors and consequences of early vs. delayed treatment which should be included among the critical questions driving prospective analyses. The current study also emphasizes the limitations of using large non-specific administrative database without neurologic assessment scales and anatomic and radiographic details to attempt to definitively answer questions of neurosurgical prognosis and management. A well-constructed prospective registry could address the inadequacies of this study. Such a registry could add considerably to our knowledge without the costs and ethical constraints inherent in a randomized, prospective trial, and seems to be the logical next investigative step.
The author has no personal financial or institutional interest in any of the drugs, materials, or devices described in this article. This work is not supported by or affiliated with any funding source.
I gratefully acknowledge the residents who participated in formulating this journal club report including: Brian Anderson, Einar Bogason, Nicholas Brandmeir, Ephraim Church, Jonathan Cooke, Gareth Davies, Koijan Kainth, John P. Kelleher, Russell Payne, Pratik Rohatgi, Emily Sieg and Omar Zalatimo. I would also like to thank the faculty who provided superb guidance during the writing: Dr. Michael J. Glantz, Dr. Robert E. Harbaugh and Dr. Kevin M. Cockroft.
1. Kassell, N.F. and J.C. Torner, The International Cooperative study on timing of aneurysm surgery. Acta Neurochir (Wien), 1982. 63(1-4): p. 119-23.
2. Haley, E.C., Jr., N.F. Kassell, and J.C. Torner, The International Cooperative Study on the Timing of Aneurysm Surgery. The North American experience. Stroke, 1992. 23(2): p. 205-14.
3. Labovitz, D.L., et al., Subarachnoid hemorrhage incidence among Whites, Blacks and Caribbean Hispanics: the Northern Manhattan Study. Neuroepidemiology, 2006. 26(3): p. 147-50.
4. Whitfield, P.C. and P.J. Kirkpatrick, Timing of surgery for aneurysmal subarachnoid haemorrhage. Cochrane Database Syst Rev, 2001(2): p. CD001697.
5. de Gans, K., et al., Timing of aneurysm surgery in subarachnoid hemorrhage: a systematic review of the literature. Neurosurgery, 2002. 50(2): p. 336-40; discussion 340-2.