Number needed to treat values based on 50% pain reduction did not differ significantly from NNT values based on 30% pain reduction (Spearman Rho = −0.02, P = 0.89). The use of an inert or active placebo did not change over time, and there was no significant difference in NNT values in studies using an inert or active placebo (Supplementary Table 1a, available at http://links.lww.com/PAIN/A628). Similarly, there was no difference in NNT values over time between studies allowing for and not allowing for concomitant treatment with other analgesics (Supplementary Table 1a, Supplementary Fig. 2, available at http://links.lww.com/PAIN/A628). NNHs increased over time, but there was no significant change in the percentage who dropped out during active or placebo treatment because of side effects (Supplementary Table 1a, available at http://links.lww.com/PAIN/A628). In newer studies published from 2008 and beyond (n = 71), the NNT correlated with study size, duration, and quality, and placebo response.
We conducted additional analysis in parallel design studies (Supplementary Table 1b, available at http://links.lww.com/PAIN/A628). All the factors mentioned above were related to higher NNT except studies where outcomes were based on ITT analysis and 30% or 50% pain reduction as outcome measures, which were all more recent (Supplementary Table 1b, available at http://links.lww.com/PAIN/A628). In studies where NNT based on both 30% or 50% pain reduction and PGIC could be calculated, there was a tendency that NNT was lower when based on PGIC (Supplementary Table 2, available at http://links.lww.com/PAIN/A628).
3.3. Drug classes and pain conditions
Figure 5 illustrates the change in NNT over time for each drug class. Although not part of our planned analysis, we observed that NNT values were affected by the maximum dose administered in a trial. When only pregabalin studies with maximum daily doses up to 600 mg were included, NNTs were lower (Supplementary Fig. 3, available at http://links.lww.com/PAIN/A628). Several recent trials assessing pregabalin as a positive control only used a maximum daily dose of 300 mg compared with many earlier trials using 600 mg. Drug dose may be important for other drug and drug classes also, but too few trials provided this information for other drugs than pregabalin. Tricyclic antidepressants were studied mainly in early trials (Supplementary Fig. 4A, available at http://links.lww.com/PAIN/A628), and the number of patients responding to active drug and placebo for each drug class is illustrated in Supplementary Figure 4B (available at http://links.lww.com/PAIN/A628). Cumulative NNT for tricyclic antidepressants, serotonin–noradrenaline reuptake inhibitors, and pregabalin in daily doses up to 600 mg are illustrated in Supplementary Figure 5 (available at http://links.lww.com/PAIN/A628). There was no clear change in pain conditions examined over time, and NNTs were similar across pain conditions, except for a tendency for high NNT in studies of painful polyneuropathy due to HIV, which also had high placebo responses (Supplementary Fig. 6, available at http://links.lww.com/PAIN/A628). Supplementary Figure 7, available at http://links.lww.com/PAIN/A628, illustrates the NNT for each drug class for each of the pain conditions. Most trials that form the basis for treatment recommendations are performed in peripheral neuropathic pain.
3.4. Placebo responses
The weak correlation between placebo response and publication year was mainly caused by low placebo responses in very early trials. For example, as can be seen from Figure 3C, studies published before 1996 all had low placebo responses, and including only studies published after 1995, there was no statistically significant increase in placebo responses in trials with increasing publication year (Spearman Rho = 0.10, P = 0.31). Similarly, there was no correlation between placebo response and publication year, if only studies that reported 30% or 50% pain reduction were included (Spearman Rho = 0.18, P = 0.13), nor, as discussed above, if only parallel group design studies were included. The correlation between placebo response and NNT did, however, persist after excluding studies before 1996 and studies that did not report 30 or 50% pain reduction (Spearman Rho > 0.40, P < 0.001). Large sample size and long study duration correlated with high placebo responses, and parallel design studies had higher placebo responses than cross-over studies (Supplementary Table 1a, available at http://links.lww.com/PAIN/A628). As can be seen from the L'Abbé plot in Figure 6, studies that were negative on the primary outcome had both low and high placebo responses. Although 83% of trials with a placebo response rate lower than 30% were positive on the primary outcome (ie, showed superiority of the study drug over placebo), only 52% of trials with a placebo response rate above 30%, and 33% of trials with a placebo response above 33%, were positive on the primary outcome.
The major finding from this study was that estimated effect size of drug trials for neuropathic pain decreased from 1982 to 2017, with increases in overall NNT and NNT per drug class. This decrease was apparent until 2010 then the effect size tended to stabilize. Importantly, this change over time was paralleled by changes in study design. More specifically, larger sample size, longer study duration, better reporting of randomization and blinding, ITT analysis, and more complete reporting of efficacy (ie, the use of 30% or 50% pain reduction as outcome measures) were all significantly associated with reduced effect size. By contrast, there was no difference in effect size whether NNT was based on 30% or 50% pain reduction, which is in line with studies in acute pain.23,25 Similarly the use of concomitant medication did not affect effect size.
The increase in placebo response over time was explained mainly by low placebo responses in very early trials. It is possible that the placebo responses are less associated with trial failure than previously thought.8,19 We did, however, find a reduced percentage of positive trial outcomes in studies with more than 30% placebo responders (although there are statistical explanations for such associations27). This is in line with studies of bipolar disorders12 and depression,13,18 where a placebo response rate greater than 30% showed greatly reduced drug–placebo separation, supporting the suggestion that such trials might be considered failed or uninformative rather than negative trials.
The changes in NNT and study design over time may be seen in light of the evolution in the standards for RCTs. Our understanding of biases has improved, and along with changes in requirements from regulatory bodies, the standards for RCTs have changed. Although early studies were typically small single-center cross-over trials, a change in NNT appeared when large gabapentin trials were published, which initiated a shift in trial design towards larger parallel group trials of longer duration and with multiple sites. In the mid-2000s, the Food and Drug Administration began to require a 12-week duration for phase 3 trials and clinical trials in the United States migrated from academic medical centers to commercial centers in the community that conduct trials across many therapeutic areas, which may have influenced the second jump in NNT. The question is what is the “true NNT.” In early trials, the use of per-protocol-analysis and lack of reporting of 30% or 50% reduction in pain intensity may have overestimated the treatment effect (underestimated the NNT). In later trials, however, there may be an underestimation of the effect size. Although not analyzed in this study, it has been suggested that large multicenter trials may have a less careful patient selection and study implementation and integrity, which may underestimate treatment effects, because of the introduction of a higher rate of underlying variability.16,20 Recruitment pressures at commercial sites and limited access to health care may also result in inflation of baseline scores and high placebo responses. Therefore, investigator and patient training and improved diagnostic accuracy and assessment, as well as implementation of methods to secure patient adherence, have been advocated to improve trial assay sensitivity.16,22
Several limitations of this study should be acknowledged. Although the study included all high-quality pharmacotherapy trials with first to third line agents in neuropathic pain, there were too few studies regarding each drug or each pain conditions to allow for testing interactions, the role of different drug doses or the role of fixed vs flexible dosing, and to allow for examining each drug separately rather than lumping different drugs into drug classes. Because of the collinearity between factors, regression analyses were not feasible. Therefore, the presentation of the associations between factors is mainly descriptive and does not make it possible to identify whether variables independently explain the changes over time. A previous study using standardized effect size, which is the ratio of the treatment effect and the within-group standard deviation, did not find an association between publication year and standardized effect size,3 and it is possible that the decrease in estimated effect sizes with increasing publication year is partly related to the use of NNT as the primary outcome. There are limitations to the use of NNT as a summary measure of treatment effect, and other analyses using risk rations, standardized effect sizes, or other measures may be relevant to consider in systematic reviews. Using other outcomes, such as PGIC, pain relief, quality of life, and pain impact, may, however, have yielded somewhat different results. Outcomes based on PGIC or pain relief may result in lower NNTs than outcomes based on reduction in pain intensity, and these types of sensitivity analyses should be encouraged in future trials. We did not have access to single-patient data and may not have extracted all factors that could be associated with the study outcome, such as number of study sites, pain duration, compliance, comorbidities, age, and number of treatments tried before the trial, which may affect assay sensitivity.2–4 Some of the factors included were not always clearly described in the articles. For example, ITT analysis may be mislabeled and is difficult to estimate in cross-over trials,9 and too few studies used baseline observation carried forward methods to examine the impact of imputation method. Also, several possible important factors were rarely assessed or reported in the publications, such as patient expectation, pain characteristics, and serum drug concentrations. We did not identify unpublished studies before 2007, and it is likely that several unpublished studies exist, although our primary analysis suggested minimal publication bias.6 The correlation between year of publication and NNT persisted even if unpublished or negative trials were not included, suggesting that the increase in NNT over time was not only due to publication bias or delays in publication. Finally, the drugs may have been included in trials for different purposes, eg, establishing whether there was an effect of the drug over placebo in a given condition, to evaluate predictors for response, or to have an active control for an investigational drug, and not all were designed to measure the magnitude of an overall treatment effect.
We examined the changes in effect sizes of neuropathic pain clinical trials in the past 35 years. Across all drug classes, the NNT has increased (estimated drug effect size decreased) over time, with stabilization around 2010. Altered study design methodology with larger study size and duration, as well as changes in reporting of outcomes, paralleled this decrease in overall estimated drug effect sizes. Except for very low placebo responses in early trials, placebo responses did not increase over time, but high placebo responses were generally associated with higher NNTs. Our analysis supports the suggestions that systematic reviews and meta-analyses should go beyond the aggregate findings of a meta-analysis and carefully look at all characteristics of the individual studies. Developing alternative study designs and studying phenotype-specific drug effects may lead to improved drug development in the future. Thus to increase our understanding and to improve treatment, future studies should record detailed patient characteristics at baseline and meta-analysis should preferably be made on patient-level data.
Conflict of interest statement
N. Attal received speakers' fee from Pfizer and reported consultant fees from Novartis, Teva, Grünenthal, Mundipharma, Sanofi Pasteur, Aptynix. R. Baron received speakers' or consultancy fees from Pfizer, Genzyme GmbH, Grünenthal GmbH, Mundipharma, Allergan, Sanofi Pasteur, Medtronic, Eisai, Lilly GmbH, Boehringer Ingelheim Pharma GmbH&Co KG, Astellas, Novartis, Bristol-Myers Squibb, Biogenidec, AstraZeneca, Merck, Abbvie, Daiichi Sankyo, Glenmark Pharmaceuticals, Seqirus, Teva Pharma, Genentech, Galapagos NV, Kyowa Kirin GmbH, Vertex Pharmaceuticals Inc, Biotest AG, Celgene, Densitin, Bayer-Schering, MSD, TAD Pharma GmbH, and research support from Pfizer, Genzyme GmbH, Grünenthal GmbH, Mundipharma. R. Baron is member of the EU Project No 633491: DOLORisk. Member of the IMI “Europain” collaboration and industry members of this are AstraZeneca, Pfizer, Esteve, UCB-Pharma, Sanofi Aventis, Grünen-thal GmbH, Eli Lilly, and Boehringer Ingelheim Pharma GmbH&Co KG, German Federal Ministry of Education and Research (BMBF), the ERA_NET NEURON/IM-PAIN Project (01EW1503), the German Research Network on Neuropathic Pain (01EM0903), NoPain system biology (0316177C), and the German Research Foundation (DFG). R.H. Dworkin has received in the past 36 months research grants and contracts from US Food and Drug Administration and US National Institutes of Health, and compensation from Abide, Adynxx, Aptinyx, Astellas, AstraZeneca, Biogen, Boston Scientific, Braeburn, Celgene, Centrexion, Chromocell, Concert, Coronado, Daiichi Sankyo, Dong-A, Eli Lilly, Eupraxia, Glenmark, Grace, Hope, Hydra, Immune, Johnson & Johnson, Medavante, Novartis, NSGene, Olatec, Periphagen, Pfizer, Phosphagenics, Quark, Reckitt Benckiser, Regenacy, Relmada, Sandoz, Semnur, Spinifex, Syntrix, Teva, Thar, Trevena, and Vertex. N.B. Finnerup has received honoraria for serving on advisory boards or speakers panels from Grünenthal, Teva Pharmaceuticals, Novartis Pharma, Astellas, and Mitshubishe Tanabe Pharma. I. Gilron has received support from Biogen, Adynxx, TARIS Biomedical, AstraZeneca, Pfizer, and Johnson & Johnson and has received grants from the Canadian Institutes of Health Research, Physicians' Services Incorporated Foundation, and Queen's University. S. Haroutounian has received a research grant from Pfizer Inc. M. Haanpaa received fees for consulting from Abbvies, Astellas, and Pfizer, and for lecturing from Astellas, Mundipharma, and Pfizer. T.S. Jensen received speakers or consultancy fees from Pfizer, Mundipharma, Daichi-Sankyo, Novartis, Orion Pharma and Biogen. P.R. Kamerman has received payment for consultancy work or as a speaker for Janssen Pharmaceutica, Partners in Research, and Reckitt Benckiser. A. Moore has received grant support from Grünenthal and Novartis and has received honoraria for attending boards with R. Baron and honoraria from Omega Pharma and Futura Pharma. S.N. Raja has served on advisory boards for Allergan, Aptinyx, and Daichi-Sankyo. S.N. Raja is supported by an NIH grant-NS26363. A.S.C. Rice has received funding from Orion Pharma and undertakes consultancy and advisory board work for Imperial College Consultants in the last 12 months; this has included remunerated work for Merck, Galapagos, Toray, Quartet, Lateral, Novartis, and Orion. A.S.C. Rice was the owner of share options in Spinifex Pharmaceuticals from which personal benefit accrued on the acquisition of Spinifex by Novartis in July 2015 and from which future milestone payments may occur. A.S.C. Rice is named as an inventor on patents related to N-(2-propenyl) hexadecanamide and related amides and inhibition of vgf activity. The remaining authors have no conflicts of interest to declare.
Supplemental digital content
Supplemental digital content associated with this article can be found online at http://links.lww.com/PAIN/A628.
. Bartfai T, Lees GV. Pharma TARP: a troubled asset relief program for novel, abandoned projects in the pharmaceutical industry. ScientificWorldJournal 2011;11:454–7.
. Dworkin RH, Turk DC, Peirce-Sandner S, Burke LB, Farrar JT, Gilron I, Jensen MP, Katz NP, Raja SN, Rappaport BA, Rowbotham MC, Backonja MM, Baron R, Bellamy N, Bhagwagar Z, Costello A, Cowan P, Fang WC, Hertz S, Jay GW, Junor R, Kerns RD, Kerwin R, Kopecky EA, Lissin D, Malamut R, Markman JD, McDermott MP, Munera C, Porter L, Rauschkolb C, Rice AS, Sampaio C, Skljarevski V, Sommerville K, Stacey BR, Steigerwald I, Tobias J, Trentacosti AM, Wasan AD, Wells GA, Williams J, Witter J, Ziegler D. Considerations for improving assay sensitivity in chronic pain clinical trials: IMMPACT recommendations. PAIN 2012;153:1148–58.
. Dworkin RH, Turk DC, Peirce-Sandner S, He H, McDermott MP, Farrar JT, Katz NP, Lin AH, Rappaport BA, Rowbotham MC. Assay sensitivity and study features in neuropathic pain
trials: an ACTTION meta-analysis. Neurology 2013;81:67–75.
. Fanelli D, Costas R, Ioannidis JP. Meta-assessment of bias in science. Proc Natl Acad Sci U S A 2017;114:3714–19.
. Finnerup NB, Otto M, McQuay HJ, Jensen TS, Sindrup SH. Algorithm for neuropathic pain
treatment: an evidence based proposal. PAIN 2005;118:289–305.
. Finnerup NB, Attal N, Haroutounian S, McNicol E, Baron R, Dworkin RH, Gilron I, Haanpaa M, Hansson P, Jensen TS, Kamerman PR, Lund K, Moore A, Raja SN, Rice AS, Rowbotham M, Sena E, Siddall P, Smith BH, Wallace M. Pharmacotherapy for neuropathic pain
in adults: a systematic review and meta-analysis. Lancet Neurol 2015;14:162–73.
. Finnerup NB, Sindrup SH, Jensen TS. The evidence for pharmacological treatment of neuropathic pain
. PAIN 2010;150:573–81.
. Furukawa TA, Cipriani A, Leucht S, Atkinson LZ, Ogawa Y, Takeshima N, Hayasaka Y, Chaimani A, Salanti G. Is placebo response
in antidepressant trials rising or not? A reanalysis of datasets to conclude this long-lasting controversy. Evid Based Ment Health 2018;21:1–3.
. Gewandter JS, McDermott MP, McKeown A, Smith SM, Pawlowski JR, Poli JJ, Rothstein D, Williams MR, Bujanover S, Farrar JT, Gilron I, Katz NP, Rowbotham MC, Turk DC, Dworkin RH. Reporting of intention-to-treat analyses in recent analgesic clinical trials: ACTTION systematic review and recommendations. PAIN 2014;155:2714–19.
. Greenland S, Robins JM. Estimation of a common effect parameter from sparse follow-up data. Biometrics 1985;41:55–68.
. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated march 2011]. Cochrane Collaboration 2011. Available at: http://handbook-5-1.cochrane.org
. Accessed July 27 2018.
. Iovieno N, Nierenberg AA, Parkin SR, Hyung Kim DJ, Walker RS, Fava M, Papakostas GI. Relationship between placebo response
rate and clinical trial
outcome in bipolar depression. J Psychiatr Res 2016;74:38–44.
. Iovieno N, Papakostas GI. Correlation between different levels of placebo response
rate and clinical trial
outcome in major depressive disorder: a meta-analysis. J Clin Psychiatry 2012;73:1300–6.
. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:1–12.
. Jensen TS, Baron R, Haanpaa M, Kalso E, Loeser JD, Rice AS, Treede RD. A new definition of neuropathic pain
. PAIN 2011;152:2204–5.
. Kemp AS, Schooler NR, Kalali AH, Alphs L, Anand R, Awad G, Davidson M, Dube S, Ereshefsky L, Gharabawi G, Leon AC, Lepine JP, Potkin SG, Vermeulen A. What is causing the reduced drug-placebo difference in recent schizophrenia clinical trials and what can be done about it? Schizophr Bull 2010;36:504–9.
. Khan A, Bhat A, Kolts R, Thase ME, Brown W. Why has the antidepressant-placebo difference in antidepressant clinical trials diminished over the past three decades? CNS Neurosci Ther 2010;16:217–26.
. Khan A, Detke M, Khan SR, Mallinckrodt C. Placebo response
and antidepressant clinical trial
outcome. J Nerv Ment Dis 2003;191:211–18.
. Khan A, Fahl MK, Faucett J, Khan SS, Brown WA. Has the rising placebo response
impacted antidepressant clinical trial
outcome? Data from the US Food and Drug Administration 1987–2013. World Psychiatry 2017;16:181–92.
. Liu KS, Snavely DB, Ball WA, Lines CR, Reines SA, Potter WZ. Is bigger better for depression trials? J Psychiatr Res 2008;42:622–30.
. Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 1959;22:719–48.
. McCann DJ, Petry NM, Bresell A, Isacsson E, Wilson E, Alexander RC. Medication nonadherence, “professional subjects,” and apparent placebo responders: overlapping challenges for medications development. J Clin Psychopharmacol 2015;35:566–73.
. Moore RA, Edwards JE, McQuay HJ. Acute pain: individual patient meta-analysis shows the impact of different ways of analysing and presenting results. PAIN 2005;116:322–31.
. Moore RA, Gavaghan D, Tramer MR, Collins SL, McQuay HJ. Size is everything–large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects. PAIN 1998;78:209–16.
. Moore RA, McQuay HJ. Single-patient data meta-analysis of 3453 postoperative patients: oral tramadol versus placebo, codeine and combination analgesics. PAIN 1997;69:287–94.
. Percie du SN, Rice AS. Improving the translation of analgesic drugs to the clinic: animal models of neuropathic pain
. Br J Pharmacol 2014;171:2951–63.
. Senn S. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials. Stat Med 1994;13:293–6.
. Smith BH, Torrance N, Bennett MI, Lee AJ. Health and quality of life associated with chronic pain of predominantly neuropathic origin in the community. Clin J Pain 2007;23:143–9.
. Torrance N, Lawson KD, Afolabi E, Bennett MI, Serpell MG, Dunn KM, Smith BH. Estimating the burden of disease in chronic pain with and without neuropathic characteristics: does the choice between the EQ-5D and SF-6D matter? PAIN 2014;155:1996–2004.
. Tu YK, Clerehugh V, Gilthorpe MS. Collinearity in linear regression is a serious problem in oral health research. Eur J Oral Sci 2004;112:389–97.
. van Hecke O, Austin SK, Khan RA, Smith BH, Torrance N. Neuropathic pain
in the general population: a systematic review of epidemiological studies. PAIN 2014;155:654–62.
Neuropathic pain; Clinical trial; Numbers needed to treat; Placebo response; Trial design
Supplemental Digital Content
© 2018 International Association for the Study of Pain