Interpreting regulatory trials in epilepsy

Abstract

Purpose of review 

This review discusses the interpretation of regulatory randomized controlled trials of antiepileptic drugs. An ever increasing number of drugs have been licensed, more so for add-on treatment than for monotherapy. Regulatory trials constitute the main body of evidence to inform clinical decisions about their use. Designs of regulatory trials for monotherapy are the most controversial as regulators take different views as to the type of designs that are required, which has implications for interpretation and for the risks to which patients are exposed.

Findings 

From the perspective of the regulator, the interpretation of placebo controlled add-on trials is straightforward as they have the capacity to show efficacy compared with placebo. These designs do not, however, inform clinical decisions where a choice among drugs is required. For monotherapy the European Medicines Agency will accept head-to-head trials designed to show noninferiority for licensing. The Food and Drug Administration in the USA will not, and requires trials that show superiority. The resulting ‘pseudoplacebo’ trials expose patients allocated to pseudoplacebo to risks that may be unacceptable.

Summary 

Regulatory trials continue to have major limitations for informing clinical decisions and this should be addressed.

Introduction

Over the past 15–20 years there has been a steady increase in the number of randomized controlled trials undertaken to assess the effects of treatments for epilepsy. The majority of these have investigated antiepileptic drugs and have been sponsored by industry. Such trials are undertaken primarily to inform regulatory decisions in order to get a license indication from bodies such as the Food and Drug Administration (FDA) in the USA or the European Medicines Agency (EMEA). These agencies require evidence that a new treatment has efficacy, is well tolerated and has a reasonable risk–benefit ratio ^[1]. The trials undertaken to produce this evidence are designed to have high internal validity and are randomized, double blinded and typically placebo controlled. Further safety data are usually collected during open label extension studies that allow further patient years of exposure to the new treatment. When a new drug is licensed, the only data that are usually available to inform clinical decisions regarding its use are the data from these regulatory studies. From the perspective of the clinician, patient or policy maker these trials have a number of short comings as the efficacy outcomes (measure of reduction in seizure frequency) are difficult to translate into clinical practice, and the trials are usually of short duration and results are, therefore, difficult to extrapolate to longer term effectiveness; they typically make comparisons with placebo rather than other drugs, and they use fixed dose regimens with rapid titration.

In this article, when discussing the interpretation of regulatory epilepsy trials, we have chosen to consider them from two broad standpoints, that of the regulator and that of the clinician and patient. We should highlight that neither of the authors is a regulator, both are trialists, one is a clinician and the other a statistician.

Add-on trials

Current guidelines recommend that new antiepileptic drugs be first tested in refractory populations as an add-on treatment. Once efficacy and safety have been demonstrated as an add-on treatment, monotherapy trials can be undertaken in pursuit of a monotherapy license ^[2]. The population targeted first of all in add-on designs is adults with refractory partial onset seizures who represent the largest group with treatment refractory epilepsy. Thereafter, add-on trials might be undertaken in children or in patients with generalized onset seizures in order to widen the licensed indications. The design template for epilepsy add-on trials is well established and has changed little over the past 15 years ^[3,4]. There are strict inclusion and exclusion criteria, and trials typically recruit patients who are taking no more than two or three antiepileptic drugs and are having at least four seizures per month. Patients first of all enter a baseline period, usually 8 weeks duration, during which no changes are made to their medication and seizures are counted. Those that have had sufficient seizures during this phase are randomized to have either placebo or new drug added to their existing treatment. Trials may have more than one active treatment arm, with patients randomized to two or three differing doses of the new AED or placebo, thus allowing the trial to provide evidence of dose response and provide data regarding likely effective doses. Patients are rapidly titrated to the target dose over 2–4 weeks and then followed for a further 8–16 weeks. The efficacy outcomes are measures of change in seizure frequency compared with baseline and include a continuous measure such as the median reduction in seizure frequency and a categorical measure such as the achievement of a 50% or greater reduction in seizure frequency. Data on adverse events are collected and presented as incidence rates, whereas adverse events scales might be used to gain an overall measure of tolerability.

For the regulator, the interpretation of these trials should be straightforward. Provided a significant difference is found in favour of new drug over placebo, and replicated in another study, there will be strong evidence to support efficacy. One potential confounder is that pharmacokinetic interactions might occur; for example, a new drug might inhibit the metabolism of a standard drug thus increasing the plasma levels of the standard drug, which might result in a reduction in seizure frequency even if the new drug has no direct effect on seizure frequency. The regulator will, therefore, require information on plasma levels of standard drugs and their metabolites as well as data from pharmacokinetic studies to underpin the assumption that beneficial effects are due to a direct antiepileptic effect of the new drug ^[1]. Safety data will come from placebo-controlled trials and from open label extension studies that provide further patient years of exposure. Licensing decisions are based upon an overall assessment of risk and benefit. This risk–benefit assessment is actually undertaken in a subjective way, rather than utilizing formal decision analysis, which might leave the regulators open to criticisms of inconsistency ^[5].

Interpreting these trials from the perspective of patients, clinicians and policy makers poses a much greater challenge. In clinical practice choices are made between drugs rather than between drug and placebo. Few head-to-head add-on studies have been undertaken and such designs would be unpopular with industry as there is a risk that such trials would fail to find a significant difference between treatment groups, complicating their interpretation (see discussion on assay sensitivity in active control equivalence trials). Comparisons can be made between placebo controlled trials of different drugs, but these are indirect comparisons and can be unreliable ^[3] and are no substitute for properly conducted head-to-head comparisons.

Epilepsy is a long-term condition, particularly for those with drug refractory epilepsy who are likely to continue having seizures for the rest of their life. Measuring outcomes at around 16 weeks is unlikely to generate the data that will inform decisions for what is a long-term condition. Moreover, trials have predominantly focussed on adults with partial onset seizure and few data are available for patients with generalized onset seizures, or children. In an attempt to increase the evidence base for children the EMEA now requires companies to submit a Paediatric Investigation Plan ^[6•], whereas in the USA this has been a requirement for some time. A further problem is the choice of efficacy outcome, for example percentage reduction in seizure frequency, which might be of little relevance to patients who wish to become seizure free; the proportion of patients becoming seizure free in these studies is small, and for those who do become seizure free it is not clear whether this is maintained over the longer term. A number of epilepsy centres have published ‘retention studies’ in which they describe the length of time that patients stay on a new drug as an overall measure of effectiveness. These studies typically show that less than half of patients are still taking a new drug after 12 months ^[7–9]. Some argue that longer term efficacy data can be provided by open label extension studies and other observational studies. However, a recent report demonstrates problems with the analysis used in these studies ^[10•] whereas another report demonstrates that efficacy results from these studies are so heterogeneous that the potential biases cannot be adequately investigated. The result is that efficacy data from these studies are largely uninterpretable ^[11•]. Thus, when a new drug is licensed as an add-on treatment, the trials that inform that regulatory decision have significant shortcomings from the perspective of patients, clinicians and policy makers.

Monotherapy

Once an antiepileptic drug has a license for add-on treatment a programme of investigation can be undertaken with a view to gaining a license for monotherapy ^[2]. Given that there are accepted effective treatments for epilepsy, placebo-controlled monotherapy trials recruiting newly diagnosed patients would be considered unethical and in contravention to the Declaration of Helsinki ^[12], since the placebo group would be put at risk and denied appropriate treatment. As a result there are a number of methodological problems associated with monotherapy trials of antiepileptic drugs, and regulatory authorities have differing requirements as to the design of trial that is required for gaining a monotherapy license. Two broad designs are considered in this article; active controlled equivalence (and noninferiority designs) and pseudoplacebo trials.

Active control equivalence and noninferiority designs

The ultimate goal of drug development in epilepsy is to develop new drugs that are both more effective and safer than existing standard treatments. However, few trials have found superior efficacy of one drug over another and no trial has found a new drug to have superior efficacy compared with a standard first-line treatment. From a clinical perspective, a new treatment might, however, be a useful treatment if it is proven to be equivalent to or noninferior to a standard treatment in terms of efficacy, and better tolerated ^[13]. From a regulatory perspective, proving that a new drug is equivalent or noninferior to a standard treatment for efficacy might provide evidence for licensing provided a reasonable safety profile is also demonstrated. Regulators including the EMEA will accept evidence of equivalence or noninferiority for granting a monotherapy license ^[1], whereas the FDA will not ^[14], the reasons for which are discussed in further paragraphs.

The protocol for any equivalence or noninferiority trial will need to state the primary outcome, for example achieving a 6 months remission from seizures during follow-up as recommended by the EMEA ^[1], and define the limits of equivalence (or noninferiority boundary) often referred to as delta (Δ). There are a number of approaches that can be used to define Δ, one of which is to define Δ as the smallest important clinical difference. For example, by 12 months, one might expect 50% of patients given a standard treatment to have a 6-month seizure free period. If, it was agreed that there would be no important difference if on a new treatment 40% (or 60%) of patients had a 6-month seizure free period, then Δ would be set at 10%. Clearly, this method of assigning a value to Δ requires a judgement to be made by trialists, ideally including the views of patients. Having defined a value of Δ, attention can be turned to the point estimate of effect and its associated 95% confidence interval (CI). In order to show equivalence, the CI for the difference in outcome between new and standard treatment will need to lie within the boundaries defined by ±Δ, as illustrated in Fig. 1. Thus, any such trial will need to be powered in order to generate confidence limits that are narrow enough to infer equivalence, usually needing the recruitment of significantly more patients than trials designed to find a difference between treatments. For noninferiority trials the focus is primarily upon the lower limit of the CI (Fig. 1). The name of such trials is somewhat of a misnomer as they are not designed to demonstrate that a new treatment is not inferior to a standard treatment; rather, they are designed to exclude the possibility that when compared with a standard treatment, a new treatment is no worse than a prespecified amount (Δ).

Figure 1: Estimates and 95% confidence intervals for a hypothetical outcome, illustrating equivalence and noninferiority

None of the noninferiority epilepsy trials published to date have used a noninferiority limit (Δ) defined as the smallest important clinical difference. For some the justification in the choice of Δ has not been clear ^[15,16]. Others have attempted to estimate the likely efficacy (6-month seizure free rate) in a hypothetical placebo group, referred to as a putative placebo ^[17], and have then defined a delta that is a comfortable margin away from this putative placebo rate ^[18•]. Such definitions of Δ will tend to be larger than the smallest important clinical difference, which may be attractive, as fewer patients will need to be recruited to trials with larger values of Δ. However, although results of such a trial might provide evidence that a new treatment is better than a putative placebo, hence demonstrating efficacy, it may not exclude the possibility of the new treatment being clinically inferior to a standard treatment.

Additional evidence to support a treatment effect

If a trial does find equivalence or noninferiority of a new treatment compared with a standard treatment, assuming that the trial was conducted appropriately and the result is a true positive, there are two possible interpretations, firstly that both treatments were effective and secondly that both treatments were ineffective. Additional data are required to make a distinction between these two alternatives. One option is to have a placebo arm as well as the active treatment arms. Given that placebo controlled trials would be unethical for patients with newly diagnosed epilepsy this is not an option for AED monotherapy trials. A second option is to use data from previous trials as external evidence for the efficacy of a standard treatment (Fig. 1). Unfortunately, there are no randomized placebo controlled monotherapy trials of the standard AEDs such as carbamazepine, valproate or phenytoin, as these treatments were licensed in an era when rigorous placebo controlled trials were not required. There are, however, data from randomized controlled trials that support the efficacy of standard drugs such as carbamazepine. This includes double blind trials that have found carbamazepine superior to a new drug ^[16,19], as well as data from unblinded trials comparing carbamazepine and valproate with no treatment in patients with single seizures or early epilepsy ^[20].

The lack of unequivocal evidence of efficacy from placebo controlled trials of standard AEDs is a major stumbling block for the reliable interpretation of epilepsy monotherapy equivalence trials, which is of particular importance for drug licensing authorities.

Assay sensitivity

Even if we accept that we have evidence that a standard treatment does have an overall effect as monotherapy, when interpreting an equivalence or noninferiority trial, another fundamental issue to consider is that of assay sensitivity – would the standard treatment have had an effect in the particular population recruited into that trial. This issue is highlighted in Fig. 2, which represents hypothetical results for proportion with a seizure remission (y-axis) in a population of newly diagnosed patients with increasing risk of remission (x-axis). Hypothetical treatment responses to placebo and standard AED are shown by the curves. Although the standard AED might have an overall effect across this population, the magnitude of treatment effect compared with placebo was small for patients at high risk of remission (e.g. a patient with two seizures 2 years apart and a normal EEG and MRI), and for patients at low risk of remission (e.g. a patient with 10 seizures, neurological signs and abnormal EEG and MRI). A noninferiority trial recruiting a population of patients biased towards either extreme may well find noninferiority, but due to treatments being much less effective. Hence patient selection, through inclusion and exclusion criteria or other selection processes, could bias the trial toward finding equivalence.

Figure 2: Results for seizure recurrence from a hypothetical placebo controlled trial demonstrating treatment effect for patients with increasing risk of seizure remissions

These issues are of particular importance for drug licensing. The International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) has published guidance for the choice of control group and related issues in clinical trials (ICH E10) ^[21•]. For historical evidence of sensitivity to drug effects, they require evidence ‘that similarly designed trials in the past regularly distinguished effective treatments from less effective or ineffective treatments…’ and ‘Specifically, it should be determined that, in the specific therapeutic area under study, appropriately designed and conducted trials that used a specific active treatment, or other treatments with similar effects, reliably showed an effect. Optimally, this is demonstrated by finding that the active treatment intended for use as the active control was reliably found superior to placebo.’

Deciding whether there are sufficient data to allow the interpretation of active control equivalence trials requires a judgement to be made after considering the available data. The EMEA has judged that active control equivalence trials of antiepileptic drugs can be interpreted and used for monotherapy licensing ^[1] whereas the FDA has judged that they cannot ^[14]. As a result of their judgement, the FDA requires monotherapy trials that prove that a new treatment is significantly better than a control treatment. This has resulted in trials of questionable ethical conduct ^[12] where patients in the control group are given low or inadequate doses of treatment in designs such as the pseudoplacebo designs.

Pseudoplacebo designs

In these trials the control group is allocated to treatment with an inadequate dose (pseudoplacebo) of a standard or a new treatment, whereas the active treatment groups are given an adequate (often high) dose of a new AED; the aim is to demonstrate that the new drug is superior to pseudoplacebo. These trials have been undertaken in three broad populations of patients; newly diagnosed patients ^[22,23], drug refractory patients taking one or two AEDs ^[24–29] (withdrawal to monotherapy design), and patients undergoing presurgical evaluation who have had their treatment withdrawn (surgical paradigm) ^[30–32]. The primary outcome for these trials is usually time to treatment exit, for which there are a variety of criteria.

For example, Beydoun et al.^[25] report a withdrawal to monotherapy trial of patients with refractory partial onset seizures. After a 56 days baseline, patients having 2–40 seizures per 28 days were randomly allocated to one of two treatment groups; 300 mg (pseudoplacebo) or 2400 mg of oxcarbazepine. The double blind treatment period was of 112-day duration, and during the first 6 weeks withdrawal of the baseline AEDs was attempted, aiming for oxcarbazepine monotherapy. The primary outcome was time to exit from the trial, and exit criteria included a twofold increase in partial seizure frequency in any 28-day period relative to baseline; a twofold increase in the highest consecutive 2-day partial seizure frequency relative to baseline; occurrence of a single generalized seizure if none occurred during the 6 months prior to randomization; or prolongation or worsening of generalized seizure duration or frequency requiring intervention. Time to exit was significantly longer for 2400 mg oxcarbazepine compared with 300 mg.

Pseudoplacebo designs thus allow the demonstration of statistically significant differences between new drug and pseudoplacebo, overcoming problems of assay sensitivity associated with active control equivalence and noninferiority trials. The main concern with this design, however, is that patients allocated pseudoplacebo are put at unacceptable risk.

Clinical interpretation of regulatory monotherapy trials

It is randomized controlled trials of AED monotherapy that best illustrate the tensions between the requirements of regulators on the one hand and clinicians, patients and policy makers on the other. Trials using a pseudoplacebo control group have the ability to demonstrate unequivocal superiority of a new AED when compared with pseudoplacebo, demonstrating efficacy without concerns about assay sensitivity. They might, therefore, provide the most reliable information to inform licensing decisions.

Clinicians and patients, however, need to make a choice among the available treatment alternatives. Evidence to inform these decisions will come from trials in which drugs have been compared head-to-head. For trials comparing new and standard treatments, the standard treatment (control group) should be titrated and prescribed as per usual clinical practice. Of the AED trials undertaken for regulatory purposes, it is the active control equivalence trials that are best able to inform these clinical decisions. However, as currently designed, even these trials have significant limitations for informing clinical decisions. Firstly, there are problems with patient selection; trials have focussed upon adults with partial onset seizures and, therefore, provide little evidence to inform decisions for patients with generalized epilepsy or children. Secondly, there are limitations in the choice of outcomes. The most common efficacy measure is being seizure free for 6 months, with patients followed up for no more than 12 months. We would suggest that this is too short a time period to inform treatment decisions for patients who are likely to have a chronic and sometimes lifelong condition.

Conclusion

Considerations made by regulators differ to those made by clinicians and patients. At present, we do not have trial designs that meet the requirements of both. Indeed, the trial designs that provide the most robust data to inform regulatory decisions are those that provide the least useful data for clinicians and patients and vice versa.

The interpretation of AED placebo controlled add-on trials and pseudoplacebo monotherapy trials are straightforward on the one hand as they provide unequivocal evidence of a treatment effect as required by regulators; however, these trials do not inform clinical decisions where a choice among alternatives has to be made. Head-to-head AED trials provide the most useful information to inform clinical decisions, but where equivalence (or noninferiority) is found, there are only limited data to support assay sensitivity, which poses problems with interpretation from the perspective of regulatory authorities. The EMEA accepts that active controlled trials of AEDs will have assay sensitivity whereas the FDA does not. More recently, a withdrawal to monotherapy design using historical control data has been proposed, which might be acceptable to the FDA. Although this design might remove the risk of exposing a control group to pseudoplacebo, the design is observational rather than randomized and comes with its own hazards of inference, particularly if there is a drift in the population recruited compared with previous (historical) trials, which may occur even if the inclusion criteria remain the same.

For the foreseeable future, it is likely that the differing regulatory authorities will have differing requirements for granting a monotherapy license for epilepsy, and that trials with a variety of designs will be undertaken. However, all of these designs have significant limitations when interpreted from the perspective of clinicians and patients. Longer term trials measuring outcomes over a number of years are required, as is an assessment of quality of life and health economic outcomes. Such randomized trials may need to maximize external validity at the expense of internal validity by using an unblinded design, thus ensuring the recruitment of a large and broad population of patients including those that might find blinding unacceptable such as women of child bearing age, and enabling long-term follow up and avoiding biases associated with dropouts ^{[33•,34•]}. Unblinded trials also better reflect clinical practice, particularly where double dummy techniques might otherwise have to be used. Unblinded designs are of course unacceptable to a regulator, and will only be undertaken after a new drug has been licensed. Thus, when a new drug is licensed, for add-on or monotherapy, there are inadequate data to inform clinical usage, and for the time being it seems that this is unlikely to change, particularly as some guidelines for clinical usage seem to perpetuate this problem by utilizing evidence hierarchies that emphasize internal validity but fail to take external validity and clinical applicability and the potential trade-offs between the two into account ^[35], whereas other guidelines use a more balanced hierarchy ^[36,37].