CHALLENGES FACED BY TRAUMA TRIALS
Randomized controlled trials are considered the criterion standard for proving the effectiveness of new treatments. However, clinical trials in trauma care pose a number of challenges. One of the most important is sample size. Clinical trials aim to compare two or more groups of patients, which should be as similar as possible, other than for the treatment under investigation. Trauma patients, however, represent a heterogeneous population. Patients with traumatic brain injuries differ from those who suffer from exsanguinating hemorrhage, and even patients with more closely related diagnoses—for example, those with pelvic fractures—may have very different presentations, needs, and outcomes. As a result, the number of patients with comparable injuries, both in terms of anatomy and physiology, is often small. This can be problematic, because to detect meaningful clinical differences using classic (also known as “frequentist”) statistical approaches to clinical trial design, power calculations often require sample sizes that are not easily achievable given the size of the eligible patient population. This applies particularly when the treatment under investigation is complex, as is the case with many medical device trials, when they are being introduced to highly specific subgroups of patients.
Resuscitative endovascular balloon occlusion of the aorta (REBOA) falls into this category. Resuscitative endovascular balloon occlusion of the aorta is a promising and conceptually attractive new intervention, designed to reduce further blood loss and improve myocardial and cerebral perfusion, albeit at the cost of an ischemic debt. However, to date, the effectiveness of REBOA has only been evaluated using case series and nonrandomized comparisons.1–5 Importantly, one large study, from Japan, has shown that the technique may in fact be harmful,6 although the application of REBOA in the Japanese setting may differ from how it is used in Europe and North America.
Given this uncertainty, a randomized controlled trial would be helpful as any potential confounding variables should be equalized. We have recently designed such a trial, to evaluate the effectiveness, and cost-effectiveness, of REBOA. Given the very specific pool of patients for whom REBOA would be considered appropriate, it became clear that the numbers required for a traditional, frequentist trial design would not be feasible. This prompted us to explore other randomized trial design options, and we eventually adopted a Bayesian group-sequential design. The aim of this special commentary is to describe the deliberations that took place and the design eventually chosen, in order to illustrate how such methods may be used to overcome the problems posed by small populations such as those seen in trauma care.
THE UK-REBOA TRIAL
The UK-REBOA Trial will compare standard major trauma center treatment plus REBOA with standard major trauma center treatment alone, for trauma patients with confirmed or suspected life-threatening torso hemorrhage. Patient recruitment is expected to commence in October 2017. The total duration of the study will be 4 years, including an initial 9-month feasibility assessment phase (Fig. 1). The trial’s entry criteria include adult trauma patients with confirmed or suspected life-threatening torso hemorrhage, which is thought to be amenable to adjunctive treatment with REBOA (zone I or III), presenting at major trauma centers. The primary outcome is 90-day mortality, defined as death within 90 days of injury, before or after discharge from hospital. This outcome is intended to capture any late harmful effects. Cost-effectiveness is also being addressed.
An analysis of national trauma registry data revealed that, per year, an estimated 200 trauma patients in England might benefit from the use of REBOA. Approximately 125 of these present to major trauma centers.7 Ninety-day mortality was 33.5%. A standard sample size calculation for comparing two proportions showed that in order to demonstrate a 5% reduction in mortality (from 33.5% to 28.5%), based on a two-sided α of 5% and 80% power, a total of 2,764 patients would be required (1,382 in each arm). This equates to more than 20 years’ recruitment of all available patients presenting to major trauma centers in England. Given the number of patients who could, even in theory, be recruited, it was clear that a traditional, frequentist design was simply not feasible.
TRADITIONAL APPROACHES TO REDUCING SAMPLE SIZE
We therefore considered means of reducing the sample size to a more realistic level, while ensuring the scientific integrity of the trial. Parmar et al.8 have recently described a useful framework for a systematic and constructive examination of the options. Step 1, which is largely intuitive, considers “increasing what is feasible,” including lengthening the accrual time, broadening the eligibility criteria, and extending the collaboration nationally and internationally. Step 2 considers common approaches to reducing sample size, such as using a primary outcome, which is more “information heavy” (usually a continuous outcome), defining a realistic and worthwhile (but larger) target difference, and reducing power by small amounts. Step 3 explores less common approaches to reducing sample size, such as relaxing α, moving to one-sided significance tests, and including covariates in the design. Although this article provides a very useful framework, and one that we would recommend is reviewed, when applied to REBOA none of these approaches resulted in a feasible trial design. Instead we decided to use a combination of two other strategies that have occasionally been suggested, but rarely implemented in trauma trials: adaptive (and specifically group-sequential) methods9,10 and a shift to the Bayesian paradigm. We discuss both of these in the following sections.
A group-sequential clinical trial design is a type of adaptive design that allows for one or several interim analyses of the accruing data while recruitment is still ongoing, with the option to terminate the trial early if the intervention under study is found to be ineffective (“stopping for futility”) or even unsafe.11,12 Some group-sequential designs also permit early stopping for benefit when a clear and convincing treatment effect manifests early on. These methods have the potential to speed up trials and increase the efficiency of clinical research. When using classic (frequentist) statistics, however, repeated interim analyses of accruing data pose a multiplicity problem: the trial’s overall Type I error rate α is inflated, unless statistical adjustments are made, but these again reduce the efficiency of the design.
Bayesian statistics, in contrast, provide a very natural framework for repeated data analyses, without the need for any separate adjustments. Bayesian designs are thus particularly attractive when there are safety concerns, which demand regular analysis of the data. If an intervention is indeed harmful, then the study should be stopped as soon as possible, to minimize the number of patients exposed. Given the findings of the Japanese study, this was an area of particular concern.
BAYESIAN CLINICAL TRIAL DESIGNS
The Bayesian approach to trial design is fundamentally different.13 When a clinical trial is being planned, there is usually some existing prior knowledge regarding the effect of the intervention under investigation, for example, from a pilot study or case series. In a Bayesian trial, this existing prior knowledge can be utilized and is represented by the prior probability distribution (or, for short, the prior). New data are then gathered from the trial itself (and also summarized in the form of a probability distribution, often referred to as likelihood), and combined with the prior, using Bayes’ theorem, yielding the posterior probability distribution (or posterior for short), which represents the updated knowledge after seeing the trial data. The definitions of these terms are summarized in Table 1, and the combining of probability distributions is shown in Figure 2. In effect, the prior information and the trial results, as they emerge, are viewed as a continuous stream of information, in which inferences can be updated as new data become available. This concept is shown in Figure 3.
Although widely used in other fields, the uptake of Bayesian statistics by the medical sciences, and particularly in clinical trial design, has been slow. This may be related to the mathematical complexity of some Bayesian techniques, potentially increased computational cost, and also the concept of a prior. The prior distribution plays a key role in Bayesian analysis, but is often misunderstood. Conceptually, it is easy to comprehend how previous knowledge, when combined into what are known as strongly informative priors, could be used to contextualize new data. When good prior information exists, the Bayesian approach allows for such information to be incorporated into the design and analysis of a trial, justifying a smaller or shorter-duration trial, and thus increasing trial efficiency. The major issues lie in how to decide what “good” prior information is, how to determine an informative prior that is acceptable for trialists and stakeholders, and how to “weight” the prior knowledge in comparison to the new trial data. All this is far from straightforward in practice, and even more so when the information available is contradictory, and comes from heterogeneous patient populations. Informative priors, although conceptually attractive, are therefore sometimes regarded as inherently subjective and viewed with a degree of suspicion. Given the conflicting prior evidence relating to REBOA, and the confirmatory nature of the intended trial, we therefore chose to use noninformative priors.
Noninformative priors (which are also known as flat or weak priors, on account of the shape of their probability distributions) avoid these issues, but cannot make use of existing knowledge. Some regard the term “noninformative” as an unhelpful misnomer (because a completely flat prior suggests that all values of the effect of the intervention are considered equally likely, which is also information) and refer to these priors as “reference” priors instead. They are primarily used when there is no prior information available, when the evidence is conflicting, or for confirmatory trials, where the aim is to “let the data speak for themselves.”
These priors carry very little information and thus have only minimal influence on the posterior (Fig. 2A). Despite not incorporating (much) prior knowledge, these types of trials are often more efficient and appealing than frequentist designs. The reason is that the Bayesian concept can be used for iterative updating of the state of knowledge: starting with a completely flat prior and adding data from the first patient (or cohort of patients), the posterior is the same as the likelihood. This posterior—which is now “informative”—then becomes the prior for the next updating step when data from the next patient (or cohort of patients) are added, and so on (Fig. 3).
Another major advantage of Bayesian methods is that they provide intuitively comprehensible answers. One of the problems with frequentist statistics is that p values and confidence intervals are easy to misinterpret14–16 and when correctly interpreted often do not give a useful answer to the question asked: a Bayesian analysis yields the probability of a specific treatment effect given the data, whereas a frequentist p value is the probability of a specific or more extreme treatment effect given the assumption of a null hypothesis of no effect being true, which is convoluted and hard to appreciate. And unlike frequentist confidence intervals, Bayesian credible intervals are also straightforward to understand: there is a 95% probability that the true value of the treatment effect falls within the 95% credible interval, given the observed data. Moreover, Bayesian designs can offer great flexibility when it comes to formulating meaningful decision criteria, for example, for declaring the success of a trial or for stopping a sequential trial early.
In a Bayesian trial, we often present how a Bayesian analysis will perform given a set sample size. We therefore estimated the number of patients whom we felt we would be able to recruit for the UK-REBOA Trial, in a reasonable number of centers, in a reasonable time. We calculated this number as 120, based on a pilot phase comprising five major trauma centers, recruiting over a period of 9 months, and a full trial phase comprising 10 major trauma centers, recruiting for 24 months (Fig. 1), based on our registry analysis. One hundred twenty, therefore, became our set sample size on which the expected performance of the Bayesian design was assessed.
We adopted a group-sequential Bayesian design17 with three stages, of 40 patients each, and two interim analyses after 40 and 80 randomized participants, and a final analysis after a maximum of 120 randomized participants. We decided that the trial should be stopped early if there is a high (posterior) probability (90% or greater) that the 90-day survival odds ratio (OR) falls below 1 (i.e., REBOA is harmful) at the first or second interim analysis. Resuscitative endovascular balloon occlusion of the aorta will be declared “successful” if the probability that the 90-day survival OR exceeds 1 at the final analysis is 95% or greater.
We explored the trial’s operating characteristics based on the data obtained from our registry analysis and assumed a control group (standard major trauma center treatment alone) 90-day survival rate of 66.5%.7 The design’s properties in terms of the probabilities of stopping for futility, at each interim look, and declaring success, for potential effect sizes from an OR of 0.7 (equating to a reduction in 90-day survival from 66.5% to 58.2%, i.e., REBOA causing harm) through to 1.3 (equating to an increase in 90-day survival from 66.5% to 72.1%), and the expected sample size (the number of patients expected to be randomized under this scenario), are shown in Table 2. The probabilities were computed using the R package gsbDesign.18
The probabilities of early stopping are high if REBOA results in markedly decreased 90-day survival, roughly 19% if there is no difference to standard care (Table 2, highlighted in blue) and less than 10% if REBOA is a success with OR 1.05 or greater (highlighted in yellow). The probability that success is declared is less than 1.5% if REBOA is harmful with OR of 0.95 or less, exactly 5% if both treatments are equal (as specified in our success criterion), greater than 60% if REBOA does well (OR ≥1.2, highlighted in light green), and greater than 90% if it does exceptionally well (OR ≥1.3, highlighted in dark green). The expected sample size is directly related to the probability of early stopping.
Conducting clinical trials in trauma care is challenging. As new treatments become available, we are faced with the dilemma of how to confirm their effectiveness and strengthen the evidence base. Target groups in trauma care are often small and specialized, making the classic approach to trial design difficult. As a result, studies may not happen at all, often because of infeasible sample sizes, or may be underpowered.
One possible solution is to combine the results of such studies, if available, in meta-analyses. However, meta-analyses have their own limitations—they require a sufficient number of primary studies to be available, and heterogeneity may be problematic. An alternative solution is a complete shift in statistical paradigm, from traditional frequentist to Bayesian designs, using innovative methods such as the Bayesian group-sequential design that we implemented for UK-REBOA.
Conceptually, meta-analysis and Bayesian updating share similarities: Both techniques combine information in order to obtain the current best estimate about the treatment effect of interest. Under a Bayesian framework, one could—at least in theory—construct a prior that captures and represents all available information about the treatment effect (such as estimates from previous trials) and update it with new trial data so that the posterior would be broadly equivalent to an updated meta-analysis, but of course it would pose challenges such as how to weight the prior information in relation to the emerging trial data. Bayesian updating does not make meta-analysis obsolete, as the goal of the latter is not only to aggregate evidence and estimate a treatment effect but also to assess consistency or heterogeneity between single trials. Furthermore, meta-analysis can be conducted with Bayesian methods as they are well suited to flexibly modeling complex hierarchical relationships.19
When considering the design of primary research, Bayesian clinical trial designs represent an innovative means of increasing trial efficiency and conducting trials with more realistic sample sizes, especially when the conduct of a trial would otherwise be infeasible, along with other benefit such as straightforward interpretability. The use of these designs is increasing, and both the Food and Drug Administration and the European Medicines Agency have released guidance relating to the use of Bayesian clinical trials,20,21 but despite such endorsements, Bayesian designs are often still viewed as too unconventional or difficult. Medical devices are unusual in that regulatory approval does not require proof of effectiveness, resulting in their rapid proliferation. Buxton’s law of the evaluation of new technologies and devices—which states that “it is always too early [for rigorous evaluation] until, unfortunately, it’s suddenly too late”22—unfortunately often applies in trauma care. Bayesian clinical trial designs are an attractive alternative to case series and poorly controlled retrospective comparisons.
THE UK-REBOA TRIAL INVESTIGATORS
The UK-REBOA Trial Investigators are J.O.J., Aberdeen Royal Infirmary/University of Aberdeen, United Kingdom; M.K.C., University of Aberdeen; Dwayne Boyers, University of Aberdeen, United Kingdom; Karim Brohi, Queen Mary University of London, United Kingdom; Katie Gillies, University of Aberdeen, United Kingdom; Tim Harris, Royal London Hospital, United Kingdom; Fiona Lecky, University of Sheffield/University of Manchester, United Kingdom; Robbie Lendrum, Royal Infirmary of Edinburgh, United Kingdom; G.M.L., University of Aberdeen, United Kingdom; Mark Midwinter, University of Queensland, Australia; Chris Moran, Queens Medical Centre, Nottingham, United Kingdom; Jonathan Morrison, Queen Elizabeth University Hospital, Glasgow, United Kingdom; John Norrie, University of Aberdeen, United Kingdom; Alan Paterson, Strathclyde University, Glasgow, United Kingdom; Graham Scotland, University of Aberdeen, United Kingdom; Nigel Tai, Royal London Hospital, United Kingdom; Nick Welch, Dronfield, United Kingdom; P.P., Lancaster University, United Kingdom; Thomas Jaki, Lancaster University, United Kingdom.
The idea for this article was conceived by J.O.J., G.M., and M.K.C. The first draft was written by J.O.J., G.M., and M.K.C., and subsequently revised and added to by P.P. The UK-REBOA Trial was designed by the UK-REBOA Trial Investigators, who contributed to the writing and revision of the manuscript.
The authors declare no conflicts of interest.
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Health Technology Assessment program, National Institute for Health Research, NHS, or the Department of Health. The Health Services Research Unit receives funding from the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. The opinions expressed in this article are those of the authors alone. The UK-REBOA Trial is funded by the National Institute for Health Research Health Technology Assessment program (project 14/199/09). P.P. was supported by the MRC Network of Hubs for Trials Methodology Research (MR/L004933/1-R/N/P/B1).
1. Brenner ML, Moore LJ, DuBose JJ, Tyson GH, McNutt MK, Albarado RP, Holcomb JB, Scalea TM, Rasmussen TE. A clinical series of resuscitative endovascular balloon occlusion of the aorta for hemorrhage control and resuscitation. J Trauma Acute Care Surg
2. DuBose JJ, Scalea TM, Brenner M, Skiada D, Inaba K, Cannon J, Moore L, Holcomb J, Turay D, Arbabi CN, et al. The AAST Prospective Aortic Occlusion for Resuscitation in Trauma and Acute Care Surgery (AORTA) Registry: data on contemporary utilization and outcomes of aortic occlusion and resuscitative balloon occlusion of the aorta (REBOA). J Trauma Acute Care Surg
3. Abe T, Uchida M, Nagata I, Saitoh D, Tamiya N. Resuscitative endovascular balloon occlusion of the aorta versus aortic cross clamping among patients with critical trauma: a nationwide cohort study in Japan. Crit Care
4. Norii T, Miyata S, Terasaka Y, Guliani S, Lu SW, Crandall C. Resuscitative endovascular balloon occlusion of the aorta in trauma patients in youth. J Trauma Acute Care Surg
5. Moore LJ, Brenner M, Kozar RA, Pasley J, Wade CE, Baraniuk MS, Scalea T, Holcomb JB. Implementation of resuscitative endovascular balloon occlusion of the aorta as an alternative to resuscitative thoracotomy for noncompressible truncal hemorrhage. J Trauma Acute Care Surg
6. Norii T, Crandall C, Terasaka Y. Survival of severe blunt trauma patients treated with resuscitative endovascular balloon occlusion of the aorta compared with propensity score–adjusted untreated patients. J Trauma Acute Care Surg
7. Barnard EB, Morrison JJ, Madureira RM, Lendrum R, Fragoso-Iñiguez M, Edwards A, Lecky F, Bouamra O, Lawrence T, Jansen JO. Resuscitative endovascular balloon occlusion of the aorta (REBOA): a population based gap analysis of trauma patients in England and Wales. Emerg Med J
8. Parmar MK, Sydes MR, Morris TP. How do you design randomised trials for smaller populations? A framework. BMC Med
9. Flight L, Julious SA, Goodacre S. Can emergency medicine research benefit from adaptive design clinical trials? Emerg Med J
10. Lewis RJ, Bessen HA. Sequential clinical trials in emergency medicine. Ann Emerg Med
11. Jennison C, Turnbull BW. Group Sequential Methods With Applications to Clinical Trials
. London CRC Press; 1999.
12. Whitehead J. The Design and Analysis of Sequential Clinical Trials
. Chichester John Wiley & Sons; 1997.
13. Hampson LV, Whitehead J, Eleftheriou D, Brogan P. Bayesian methods for the design and interpretation of clinical trials in very rare diseases. Stat Med
14. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol
15. Glaser DN. The controversy of significance testing: misconceptions and alternatives. Am J Crit Care
16. Morey RD, Hoekstra R, Rouder JN, Lee MD, Wagenmakers EJ. The fallacy of placing confidence in confidence intervals. Psychon Bull Rev
17. Gsponer T, Gerber F, Bornkamp B, Ohlssen D, Vandemeulebroecke M, Schmidli H. A practical guide to Bayesian group sequential designs. Pharm Stat
18. Gerber F, Gsponer T. gsbDesign: an R Package for evaluating the operating characteristics of a group sequential Bayesian design. J Stat Soft
19. Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res
20. European Medicines Agency. Guideline on Clinical Trials in Small Populations
. London: EMA; 2006.
21. Food and Drug Administration. Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials
. FDA; 2010.
22. Buxton MJ. Problems in the economic appraisal of new health technology: the evaluation of heart transplants in the UK. In: Drummond MF, ed. Economic Appraisal of Health Technology in the European Community
. Oxford: Oxford Medical Publications; 1987:103–118.