The PICU management of severe traumatic brain injury (sTBI) is challenging. There are no “level I Brain Trauma Foundation” (BTF) recommendations (1). For “temperature control,” the 2012 BTF recommendations were “weak” based on “moderate” quality evidence from contradictory class II/III studies. The “level II” recommendations included “moderate hypothermia (32–33°C) beginning early after sTBI for only 24-hour duration should be avoided; and, moderate hypothermia beginning within 8 hours after sTBI for up to 48-hour duration should be considered to reduce intracranial hypertension.”
Since 2012, there has been new randomized controlled trial (RCT) evidence of hypothermia versus normothermia after sTBI in children (2, 3). Recent meta-analyses (4–7) conclude that in pediatric patients there is no benefit of hypothermia, and it may even increase risk of mortality. In conventional “frequentist” interpretation of RCTs, the null hypothesis assesses significance using effect size, p value, and the CI. The p value is the probability of observing the effect (or more extreme one) given the null hypothesis (8); it is not the probability that the null is true (9). (The 2016 American Statistical Association statement is “P-values do not measure the probability that the studied hypothesis is true, or that the data were produced by random chance alone.” ). The problem with decision-making based on p value is that it is misleading (11): statistically significant difference is not necessarily clinically important; an insignificant difference does not indicate no difference; and studies with the similar p values do not provide the same level of evidence to reject the null hypothesis (9, 12). Therefore, decision-making is based on effect size and the CI (9), but these measures do not help at the bedside. For example, the 95% CI indicates that if the RCT is repeated, then 95% of the CIs obtained will contain the true effect size. We want to know the interval that the effect size lies within, with the probability greater than or equal to 95%. Another vulnerability of this approach is the potential that patients of differing risk stratification, such as Glasgow Coma Scale (GCS) score, are lumped together.
Frequentist analysis starts with a hypothesis and uses data to judge plausibility. In contrast, Bayesian analysis is a directed decision-making approach starting with a prior belief and using outcome of an analysis to generate a “posterior” or “current” probability, which characterizes a renewed belief after observing data. That is, using both our prior belief (which is discounted in the frequentist framework) and observed data to evaluate a hypothesis (13). These posterior and prior beliefs are represented by probability distributions and the observed data described by a likelihood distribution (i.e., probability of observed data conditional on the phenomenon). In hypothermia for sTBI, for example, the posterior probability of an effect of treatment could incorporate the prior probability generated from previous RCTs. Furthermore, aggregate statistical assessment can be translated into clinically relevant information using what the clinician believes from experience or interpretation of a patient’s condition. In the Bayesian framework, the “subjective” prior may be enthusiastic, noninformative, or skeptical (14, 15). In the past, it was impractical to implement Bayesian approaches to meta-analyses because calculating the posterior distribution requires major computation. However, using the Markov Chain Monte Carlo algorithm and computer simulation means that Bayesian analyses are now used frequently in medicine (16–20).
We have used frequentist and Bayesian approaches to assess data from pediatric RCTs of hypothermia after sTBI in order to better understand current clinical recommendations (1). We show that Bayesian analysis can add clinically useful information to the interpretation of clinical trials that cannot be provided by the frequentist decision framework.
A systematic search of English language literature using PubMed (2000 to 31 August 2016) search terms “hypothermia,” “traumatic brain injury,” “head injury,” and “intracranial pressure.” RCTs in pediatric patients were identified. We hand-searched reference lists of four meta-analyses of therapeutic hypothermia in pediatric sTBI (4–7). We looked at the catalog of RCTs in the PICU population at www.picutrials.net. Last, we checked a federated literature search engine (ACCESSSS, see: http://plus.mcmaster.ca/ACCESSSS/). The abstracts were analyzed for relevance and eligible articles were assessed. Reference lists from RCTs were also checked for further resources.
Study Selection Criteria, Quality Assessment and Data Extraction
The minimum inclusion for meta-analysis was use of induced systemic hypothermia for greater than or equal to 12 hours and survival at PICU discharge. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation system (21). Data extraction included trial design, patient number, GCS score; hypothermia and control normothermia therapy; and outcomes of mortality and poor/unfavorable outcome. The division between poor/unfavorable and favorable outcome was defined using the Glasgow Outcome Scale (GOS) or the Pediatric Cerebral Performance Category (PCPC) (22, 23). In reports that used GOS, patients with scores 3 (severe outcome), 4 (vegetative state), or 5 (death) were grouped as poor/unfavorable outcome. In reports that used the PCPC, patients with scores 4 (severe disability), 5 (vegetative state), or 6 (death) were grouped as poor/unfavorable outcome.
In the conventional meta-analysis, we used risk ratio (RR) of death (or poor outcomes) as the effect measure to compare therapy (24). “R-Studio software” (R Studio, Inc., Boston, MA; https://www.rstudio.com/) was used to model data and perform meta-analysis with “meta-package” (https://cran.r-project.org/web/packages/meta/index.html). The random-effects model was used to account for heterogeneity of populations, where the number of deaths and sample size of each trial was used to compute effect size and CI using the inverse variance method and the DerSimonian-Laird heterogeneity estimator (25). A Forest plot was performed using meta-package. Heterogeneity was investigated by comparing relative risks of subgroups defined by hypothermia duration.
In the cumulative Bayesian random-effects meta-analysis, we used log (RR) in R-Studio software, where we assumed Gaussian distribution (26) (rjags package, https://cran.r-project.org/web/packages/rjags/index.html). JAGS software (Just Another Gibbs Sample v4.2.0; http://mcmc-jags.sourceforge.net/) was used to sample the posterior density. A noninformative prior was used for both mean and variance of the log (RR) (27). At each time point, this prior and the studies up to that time were used in the meta-analysis (14, 15, 28). Then, the posterior probability of relative risk reduction (RRR) being greater than 0% and 20% was calculated after each study.
Last, since Bayesian analysis can incorporate prior belief, we have investigated the impact of prior skeptical and optimistic beliefs with two informative priors for mean log (RR), that is, probabilities of 0.01 or 0.90, respectively. These probabilities are commonly used in Bayesian clinical decision-making studies (28–33). We have also calculated the posterior probability of relative risk increase (RRI) in mortality greater than 20%.
The literature search is illustrated in Figure S1 (Supplemental Digital Content 1, http://links.lww.com/PCC/A389). Thirty-five clinical trials were hand-searched to assess suitability for inclusion. At first, nine RCTs were reviewed (2, 3, 34–39), and these studies were compared with RCTs in recent meta-analyses (4–7). None of the studies in previous meta-analyses were missed. However, we found that two reports used subsets of previously published RCTs: Bourdages et al (34) reported cases from Hutchison et al (38) and Salonia et al (35) reported cases from Adelson et al (37). The authors reached consensus on the quality of the remaining RCTs (Table 1). The overall quality was low: two trials were high (3, 38), and in the other trials one was moderate (36), and four were low (2, 37, 39).
There were outlier studies (Table 2). Mean time to starting cooling after injury was 4.2–6.8 hours in six studies; in contrast, in the study by Adelson et al (37) mean time was 15 hours. Site of temperature measurement differed across studies: three used esophageal temperature (2, 36, 38); two, rectal temperature (37); one, intracranial temperature (39); and one, either intracranial or rectal temperature (3). Intended temperature in the normothermia group was 36–37.5°C, but in one study that used exclusively intracranial temperature, the range was 37.5–38.5°C (39), which is equivalent to pyrexia. Hypothermia ranged 32–34°C but the duration of temperature management varied: 24 hours (38), 48 hours (3, 36, 37), or 72 hours (2, 39). Rewarming after hypothermia was given in six studies, and varied: short, 12–18 hours (36–38); intermediate duration, 16–35 hours (2); or prolonged, 42–54 hours (3).
Statistical heterogeneity was low (p = 0.27; I2 = 20.4%). None of the trials found benefit of hypothermia (Fig. 1). The funnel plot of effect of hypothermia versus normothermia on mortality did not indicate publication bias (Fig. 2). Using RR of death in hypothermia versus normothermia, the pooled estimate with a random-effects model is 1.42 (95% CI, 0.77–2.61; p = 0.26). Table 3 shows the changes in the overall estimates when conducting the sensitivity analysis. That is, after removing one study at a time and rerunning the model, the estimates changed little—which implies the results are statistically reliable.
Figure 3 shows the Forest plot of hypothermia versus normothermia with studies grouped by duration of hypothermia (24, 48, and 72 hr). The statistical heterogeneity of these subgroups was low (24 hr, no analysis since only one study; 48 hr, p = 0.16, I2 = 42.2%; 72 hr, p = 0.20, I2 = 39.1%). The results showed no difference in hypothermia versus normothermia. Effect sizes and 95% CIs were 24 hours, RR = 1.78 (95% CI, 0.97–3.28; p = 0.06); 48 hours, RR = 1.29 (95% CI, 0.42–3.95; p = 0.65); and 72 hours, RR = 1.18 (95% CI, 0.16–8.84; p = 0.87). Duration of hypothermia did not show a difference in the magnitude of the association with mortality (p = 0.84). These same analyses were repeated using poor outcome and were no different to the results of mortality (Fig. S2, Supplemental Digital Content 1, http://links.lww.com/PCC/A389).
After ordering the studies by month/yr of recruitment closure, we carried out a Bayesian cumulative meta-analysis using log (RR) with a noninformative prior (Fig. 4). The figure shows that after the seventh RCT (2), the current probability of reducing mortality with hypothermia compared with normothermia is 0.40 (with RR < 1 or RRR > 0%). The current probability of RRR of death greater than 20%, with hypothermia rather than normothermia, is 0.28 (close to 1-in-3, see RRR > 20% curve in Fig. 4).
Figures 5 and S3 (Supplemental Digital Content 1, http://links.lww.com/PCC/A389) show the effect of an a priori belief on current probability of RRR of death greater than 20%. Based on the seven RCTs, a skeptical belief (0.01 probability that RRR of death > 20% when using hypothermia rather than normothermia) gives a current probability of 0.16 (1-in-6) of RRR of death greater than 20% on theoretically using hypothermia rather than normothermia. However, the probability of RRI of death greater than 20% is 0.61 (3-in-5) if selecting hypothermia rather than normothermia. An optimistic belief (0.90 probability that RRR of death > 20%) gives a current probability of 0.50 (1-in-2) of RRR of death greater than 20% on theoretically using hypothermia rather than normothermia. The probability of RRI of death greater than 20% is 0.28 (1-in-3) if choosing hypothermia rather than normothermia. The Bayesian analysis for poor outcome provides similar findings and is given in Figure S4 (Supplemental Digital Content 1, http://links.lww.com/PCC/A389).
We have evaluated data from seven pediatric RCTs of hypothermia versus normothermia after sTBI using both conventional and Bayesian meta-analyses in order to better understand the basis for clinical recommendations.
The updated conventional meta-analysis of hypothermia in sTBI in pediatric patients confirms that the null hypothesis—no difference between hypothermia versus normothermia on mortality and poor/unfavorable outcome—cannot be rejected. However, we should be circumspect about the use of hypothermia for two main reasons. First, the overall quality of evidence is low. Second, lumping together these RCTs is questionable. There is heterogeneity with patients of differing risk stratification (range in GCS score) across the studies. Studies also used different body sites for temperature measurement, as well as different durations of hypothermia and rewarming. And, there were two obvious outliers: one study had considerably later timing (mean 15 hr vs mean 4.2–6.8 hr in the other studies) for starting cooling (37) and one study used a pyrexial target temperature in the normothermia group (39).
In regard to clinical decision-making, a previous conventional meta-analysis of five RCTs went so far as to say “because of safety concerns with hypothermia, we do not recommend further RCTs of the intervention in children with sTBI” (5). This conclusion was based on an overall estimate showing hypothermia could increase mortality and poor/unfavorable outcome with sTBI but without statistical significance. We too have a similar finding with two more pediatric RCTs: the RR for death is 1.42 (95% CI, 0.77–2.61; p = 0.26). So, would, should, or could we use hypothermia in our sTBI patients? Our response to this quandary was to reframe the meta-analysis using a Bayesian approach (11, 13–20).
The Bayesian approach considers all sources of preexisting knowledge admissible for analysis. Both previous RCT results and different opinions are used to facilitate informed decisions likely to be meaningful to clinicians and guideline developers. First, in regard to information from RCTs, we have calculated the probability of a hypothesis (i.e., RRR of death > 20% with hypothermia rather than normothermia) given information from each trial in historical sequence. After the seventh pediatric RCT, the current probability of RRR of death greater than 20% is close to 1-in-3. The 20% threshold in RRR for mortality is used since a new therapy for critical illness is likely to be accepted by clinicians and regulators at this level (15). Second, in regard to the more subjective aspect of clinician’s prior skepticism or optimism (i.e., beliefs), we have used the Bayesian approach to incorporate a range of views in assessing the effect of hypothermia in a particular clinical setting. For example, given a current probability for RRR of death greater than 20% of 1-in-3, a skeptical (0.01 probability that RRR of death > 20%) belief in the effect of hypothermia treatment on outcome changes the probability from 1-in-3 to 1-in-6, that is, less chance of RRR death greater than 20%. If we also consider this change in probability along with a current probability of RRI in death greater than 20% of 3-in-5 by being skeptical about effectiveness of hypothermia, you would have to conclude that hypothermia should not be used. That is, in comparison with normothermia, hypothermia has less chance of RRR death greater than 20% and more than even chance of RRI in death greater than 20%. In contrast, an optimistic (0.90 probability that RRR of death > 20%) belief in the effect of hypothermia treatment on outcome changes the probability of 1-in-3 to 1-in-2, that is, even chance in RRR death greater than 20%. In addition, the current probability of RRI in death greater than 20% of 1-in-3 means a less than even chance of RRI in death greater than 20%. Under these circumstances, the clinician with an optimistic belief about the effect of hypothermia in a particular setting may consider that the choice of hypothermia is a valid therapeutic option. In fact, this observation adds support to the recent proposal by Lazaridis and Robertson (40) that “the benefit-risk ratio of HT has not been shown to be unfavorable.”
The different degrees of clinical belief that are formally translated into mathematical calculations in Bayesian meta-analysis warrant further discussion. We do not fully understand what clinical features lead treating physicians to have skeptical or optimistic beliefs about effectiveness of hypothermia in a particular pediatric patient with sTBI. Our specialty has not fully codified “optimism” and “doubt” about particular outcomes in sTBI in relation to use of hypothermia management. In contrast, investigators of hypothermia for out-of-hospital cardiac arrest (41) and neonatal hypoxic-ischemic encephalopathy (33) have better characterized subgroups that are less likely to respond to the intervention. Now, too, there is a systematic review of methods to elicit clinical beliefs from expert and nonexpert clinicians for Bayesian priors (18) that could also be applied to the design of future pediatric critical care studies (14, 42, 43).
In our updated conventional meta-analysis of seven RCTs, we show that the null hypothesis—no difference between hypothermia versus normothermia on mortality and poor/unfavorable outcome in pediatric severe TBI—cannot be rejected. We also found no influence on outcome when comparing duration of hypothermia (24, 48, and 72 hr). These findings have an impact on the BTF 2012 recommendations: we cannot conclude that hypothermia for only 24 hours “should be avoided,” and we cannot conclude that hypothermia for 48 hours “should be considered.” This lack of guidance means that clinicians are left making the decision, and we know that some are choosing to use hypothermia (40, 44).
The Bayesian meta-analysis provides a framework for decision-making in circumstances of uncertainty (45). By taking the cumulative probability of RRR of death greater than 20% and chance of RRI of death greater than 20% from seven RCTs and including one’s prior degree of skepticism or optimism about treatment effectiveness, one may be dissuaded or persuaded about treatment in a particular clinical setting. Therefore, more work is needed to understand clinical skepticism and optimism about treatment effect. Only in this way we will know how to deal with the results of seven RCTs on hypothermia in sTBI and be able to decide when hypothermia should be “avoided” and when it should be “considered” in pediatric management.
1. Kochanek PM, Carney N, Adelson PD, et al. Guidelines for the acute medical management of severe traumatic brain injury in infants, children, and adolescents—second edition. Pediatr Crit Care Med 2012; 13 (Suppl 1):S1–S82.
2. Beca J, McSharry B, Erickson S, et al; Pediatric Study Group of the Australia and New Zealand Intensive Care Society Clinical Trials Group: Hypothermia for traumatic brain injury in children-A phase ii randomized controlled trial. Crit Care Med 2015; 43:1458–1466.
3. Adelson PD, Wisniewski SR, Beca J, et al; Paediatric Traumatic Brain Injury Consortium: Comparison of hypothermia and normothermia after severe traumatic brain injury in children (Cool Kids): A phase 3, randomised controlled trial. Lancet Neurol 2013; 12:546–553.
4. Georgiou AP, Manara AR. Role of therapeutic hypothermia in improving outcome after traumatic brain injury: A systematic review. Br J Anaesth 2013; 110:357–367.
5. Hutchison JS, Guerguerian AM. Cooling of children with severe traumatic brain injury. Lancet Neurol 2013; 12:527–529.
6. Ma C, He X, Wang L, et al. Is therapeutic hypothermia beneficial for pediatric patients with traumatic brain injury? A meta-analysis. Childs Nerv Syst 2013; 29:979–984.
7. Zhang BF, Wang J, Liu ZW, et al. Meta-analysis of the efficacy and safety of therapeutic hypothermia in children with acute traumatic brain injury. World Neurosurg 2015; 83:567–573.
8. Korner-Nievergelt F, Roth T, von Felten S, et al. Bayesian Data Analysis in Ecology Using Linear Models With R, BUGS, and Stan. 2015First Edition. Waltham, MA, Academic Press.
9. Cumming G. Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis. 2013New York, NY, Routledge, Taylor & Francis Group.
11. Boylan JF, Kavanagh BP, Armitage J. Randomised controlled trials: Important but overrated? J R Coll Physicians Edinb 2011; 41:126–131.
12. Goodman S. A dirty dozen: Twelve p-value misconceptions. Semin Hematol 2008; 45:135–140.
13. Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian Approaches to Clinical Trials and Health-Care Evaluation. 2004Chichester, West Sussex, United Kingdom, John Wiley & Sons.
14. Kalil AC, Sun J. Bayesian methodology for the design and interpretation of clinical trials in critical care medicine: A primer for clinicians. Crit Care Med 2014; 42:2267–2277.
15. Kalil AC, Sun J. Why are clinicians not embracing the results from pivotal clinical trials in severe sepsis? A Bayesian analysis. PLoS One 2008; 3:e2291
16. Chaloner K, Church T, Louis TA, et al. Graphical elicitation of a prior distribution for a clinical trial. The Statistician 1993; 42:341–353.
17. Hiance A, Chevret S, Lévy V. A practical approach for eliciting expert prior beliefs about cancer survival in phase III randomized trial. J Clin Epidemiol 2009; 62:431–437.e2.
18. Johnson SR, Tomlinson GA, Hawker GA, et al. Methods to elicit beliefs for Bayesian priors: A systematic review. J Clin Epidemiol 2010; 63:355–369.
19. See CW, Srinivasan M, Saravanan S, et al. Prior elicitation and Bayesian analysis of the Steroids for Corneal Ulcers Trial. Ophthalmic Epidemiol 2012; 19:407–413.
20. Sun CQ, Prajna NV, Krishnan T, et al. Expert prior elicitation and Bayesian analysis of the Mycotic Ulcer Treatment Trial I. Invest Ophthalmol Vis Sci 2013; 54:4167–4173.
21. Morrison LJ, Gent LM, Lang E, et al. Part 2: Evidence evaluation and management of conflicts of interest: 2015 American Heart Association Guidelines Update for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation 2015; 132 (Suppl 2):S368–S382.
22. Jennett B, Bond M. Assessment of outcome after severe brain damage. Lancet 1975; 1:480–484.
23. Fiser DH. Assessing the outcome of pediatric intensive care. J Pediatr 1992; 121:68–74.
24. Stangl D, Berry DA. Meta-Analysis in Medicine and Health Policy. 2000New York, NY, CRC Press.
25. Hedges LV, Vevea JL. Fixed- and random-effects models in meta-analysis. Psychol Methods 1998; 3:486–504.
26. Sutton AAJ, Abrams KR, Jones DR, et al. Methods for Meta-Analysis in Medical Research. 2000Chichester, West Sussex, UK, John Wiley & Sons.
27. Smith TC, Spiegelhalter DJ, Thomas A. Bayesian approaches to random-effects meta-analysis: A comparative study. Stat Med 1995; 14:2685–2699.
28. Kalil AC, Sun J. Low-dose steroids for septic shock and severe sepsis: The use of Bayesian statistics to resolve clinical trial controversies. Intensive Care Med 2011; 37:420–429.
29. Harrell FE Jr, Shih YC. Using full probability models to compute probabilities of actual interest to decision makers. Int J Technol Assess Health Care 2001; 17:17–26.
30. Higgins JP, Spiegelhalter DJ. Being sceptical about meta-analyses: A Bayesian perspective on magnesium trials in myocardial infarction. Int J Epidemiol 2002; 31:96–104.
31. Diamond GA, Kaul S. Prior convictions: Bayesian approaches to the analysis and interpretation of clinical megatrials. J Am Coll Cardiol 2004; 43:1929–1939.
32. Salpeter SR, Cheng J, Thabane L, et al. Bayesian meta-analysis of hormone therapy and mortality in younger postmenopausal women. Am J Med 2009; 122:1016–1022.e1.
33. Pedroza C, Tyson JE, Das A, et al; Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network: Advantages of Bayesian monitoring methods in deciding whether and when to stop a clinical trial: An example of a neonatal cooling trial. Trials 2016; 17:335
34. Bourdages M, Bigras JL, Farrell CA, et al; Canadian Critical Care Trials Group: Cardiac arrhythmias associated with severe traumatic brain injury and hypothermia therapy. Pediatr Crit Care Med 2010; 11:408–414.
35. Salonia R, Empey PE, Poloyac SM, et al. Endothelin-1 is increased in cerebrospinal fluid and associated with unfavorable outcomes in children after severe traumatic brain injury. J Neurotrauma 2010; 27:1819–1825.
36. Biswas AK, Bruce DA, Sklar FH, et al. Treatment of acute traumatic brain injury in children with moderate hypothermia improves intracranial hypertension. Crit Care Med 2002; 30:2742–2751.
37. Adelson PD, Ragheb J, Kanev P, et al. Phase II clinical trial of moderate hypothermia after severe traumatic brain injury in children. Neurosurgery 2005; 56:740–754; discussion 740.
38. Hutchison JS, Ward RE, Lacroix J, et al; Hypothermia Pediatric Head Injury Trial Investigators and the Canadian Critical Care Trials Group: Hypothermia therapy after traumatic brain injury in children. N Engl J Med 2008; 358:2447–2456.
39. Li H, Lu G, Shi W, et al. Protective effect of moderate hypothermia on severe traumatic brain injury in children. J Neurotrauma 2009; 26:1905–1909.
40. Lazaridis C, Robertson CS. Hypothermia for increased intracranial pressure: Is it dead? Curr Neurol Neurosci Rep 2016; 16:78
41. Meert KL, Telford R, Holubkov R, et al; Therapeutic Hypothermia after Pediatric Cardiac Arrest (THAPCA) Trial Investigators: Pediatric out-of-hospital cardiac arrest characteristics and their association with survival and neurobehavioral outcome. Pediatr Crit Care Med 2016; 17:e543–e550.
42. Schoenfeld DA, Hui Zheng, Finkelstein DM. Bayesian design using adult data to augment pediatric trials. Clin Trials 2009; 6:297–304.
43. Moatti M, Zohar S, Facon T, et al. Modeling of experts’ divergent prior beliefs for a sequential phase III clinical trial. Clin Trials 2013; 10:505–514.
44. Fulkerson DH, White IK, Rees JM, et al. Analysis of long-term (median 10.5 years) outcomes in children presenting with traumatic brain injury and an initial Glasgow Coma Scale score of 3 or 4. J Neurosurg Pediatr 2015; 16:410–419.
45. Tasker RC, Akhondi-Asl A. Targeted temperature management after cardiac arrest due to drowning: “Frequentist” and “Bayesian” decision making. Pediatr Crit Care Med 2016; 17:789–791.
Bayesian analysis; meta-analysis; therapeutic hypothermia; traumatic brain injury