Post-traumatic stress disorder (PTSD) results from experiencing or witnessing an emotionally traumatic event that is perceived to present a threat to the life or physical integrity of one's self or others. PTSD is defined by 8 criteria (Table 1),[1,2] including exposure to a traumatic event and resulting symptoms from each of 4 clusters:
- (1) intrusion,
- (2) avoidance,
- (3) negative alterations in cognitions and mood, and
- (4) alterations in arousal and reactivity.
The sixth criterion requires that symptoms have been present for >1 month; the seventh assesses functioning (ie, symptoms interfere with the ability to go about normal daily tasks); and the eighth criterion is that symptoms are not attributable to another medical condition. Whereas some symptoms associated with PTSD (eg, hypervigilance) function to maintain an ongoing sense that the inciting event could happen again, which in turn, maintains fear and avoidance, others are associated with persistent feelings of shame and guilt (eg, negative-trauma related emotions). The diagnostic and statistical manual of mental disorders (DSM-5) also includes a dissociative subtype of PTSD, in recognition that 15% to 30% of patients with PTSD presents with symptoms of depersonalization and derealization.[5–15]
The lifetime prevalence of PTSD among the general public is 8% to 9%,[16–18] and between a quarter and a third of workers exposed to traumatic events develop symptoms of PTSD.[19,20] Patients suffering with PTSD experience social impairment, high absenteeism, unemployment[21–27] and work-related disability, particularly when completing tasks requiring high concentration and cognitive demands.[24–25] According to a 2016 review, 2.5 million of the general population and 70,000 first responders in Canada would suffer with PTSD in at some point in their lifetime. In a retrospective study of 44 PTSD claims awarded by the Workers Compensation Board of British Columbia, Canada, only 43% of disabled workers returned to their previous job 4 years after their trauma, 23% returned to alternate employment, and 34% did not returned to any type of work. Similarly, a prospective study of 94 PTSD patients in Ontario, Canada, found 57% had not returned to work in any capacity 9 months after their motor vehicle accident; among the 43% who had, a little more than half (57%) required job modification.
There is uncertainty regarding what treatments are most effective for facilitating return-to-work among traumatized workers. Existing systematic reviews focus on the effect of selected therapies instead of exploring the relative effectiveness of competing therapies, and on amelioration of PTSD symptoms versus functional recovery.[31–37] A previous network meta-analysis (NMA) of pharmacological treatments for PTSD has a number of limitations, including:
- (1) an outdated search (February 2016),
- (2) consideration of pharmacologic therapies only,
- (3) narrow focus on exploring treatment effects for only symptom reduction and all-cause drop-outs, and
- (4) use of the surface under the cumulative ranking curve (SUCRA) approach to rank interventions – an approach that does not consider the quality of evidence, or the estimates of precision around treatment effects.
Another NMA was similarly limited by a narrow focus only on psychotherapeutic approaches, an outdated search (January 2011), time-restricted search strategies (1980–2010), inclusion of trials with patients who had subclinical or sub-threshold PTSD and treatment effects limited to only symptom reduction.
As existing reviews of treatment for PTSD have largely focussed on individual treatment's effect on symptom reduction and none have compared all treatments against each other. Due to which there exist considerable uncertainty regarding what treatments are effective for promoting symptom control and functional recovery. Our proposed NMA will evaluate all treatments for PTSD, provide relative effectiveness of treatments, and evaluate the quality of the evidence in a thorough and consistent manner using the grading of recommendations assessment, development, and evaluation (GRADE) approach.
We registered our protocol on PROSPERO (CRD42018072682). Our review will conform to the PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions.
2.1 Search strategy
We identified all relevant randomized trials, in any language, through a systematic search of published international literature on traumatic stress, CINAHL, EMBASE, MEDLINE, AMED allied and complementary medicine, HealthSTAR, database of abstracts of reviews of effects, PsychINFO, and the Cochrane Central Registry of Controlled Trials (Appendix#1). An experienced medical librarian refined our search strategy for each database. Reviewers scanned the bibliographies of all eligible trials and other relevant publications for additional trials. We did not have any language restriction.
2.2 Eligibility criteria and study selection
Trials were eligible if they enrolled adult patients diagnosed with primary or secondary PTSD according to operationalized criteria such as DSM criteria, international classification of diseases criteria, or clinician diagnosed and randomized them to any pharmacologic (monotherapy or add-on therapy) or nonpharmacologic treatment strategy compared to an alternative treatment, their combinations, placebo or control. Eligible pharmacologic interventions included but were not restricted to antidepressants such as selective serotonin inhibitor, nonselective serotonin inhibitors, anxiyoltics, antipsychotics, and antiepileptics or mood stabilizers. Eligible nonpharmacologic interventions included but were not restricted to: cognitive behavioral therapy, exposure therapy, yoga, supportive therapy, and eye movement desensitization and reprocessing.
We will exclude trials with less than 10 participants per arm, pilot or feasibility or preliminary studies, and cross-over trials. We will also exclude trials in which the population does not constitute 100% PTSD patients including sub-threshold or subclinical or partial PTSD. We will also exclude studies that enrolled patients suffering with acute stress disorders. Conference abstracts and rarely used interventions or interventions that are discontinued due to serious adverse effects (such as Nefazadone, Brofaromine) rarely used for interventions in North America will be excluded.
Using standardized forms, reviewers will work independently in duplicate to screen title and abstracts, and the full text of all potentially eligible studies. Before starting literature screening, all reviewers will complete a pilot exercise to ensure understanding of the process and improve reliability. Any discrepancies will be resolved through consensus between reviewers or with the help of an adjudicator.
2.3 Data abstraction
To help ensure the reliability of independent data extraction, we will begin by piloting our data extraction form and then we will conduct calibration exercises between reviewers. Data abstraction will be guided by a detailed instruction manual generated from our piloting and calibration exercises. Teams of reviewers will extract data independently and in duplicate from eligible trials and resolve discrepancies through discussion. Data abstracted will include study characteristics (the first author, publication year, funding source); patient and trial characteristics such as sample size, participant demographics (age, gender, litigation and disability status of the participants, associated co-morbid psychological and substance status, traumatic brain injury and first responder status such as veterans, police officers and fire fighters); characteristics of interventions and comparators (dose of pharmacological treatments, frequency for nonpharmacologic treatments, use of other intervention such as medications or psychotherapies during the trial).
We will extract data on the on return to work (percent patients on partial or complete sick leave, percent patients with partial or complete functional recovery). We will also extract data on the PTSD symptoms such as reported with clinician-administered PTSD scale (CAPS),[41–43] PTSD checklist or any other validated scales, quality of life such as short-form 36 with Physical (PCS) and mental (MCS) component summaries, or any other validated quality of life scale such as the World Health Organization Quality of Life Questionnaire, The Quality of Life Enjoyment and Satisfaction Questionnaire; functional impairment with Sheehan's disability scale, and social and occupational impairment with any validated scale such as social adjustment scale.
If an individual trial reports more than 1 outcome in a common domain (eg, return to work), we will select only 1 outcome per domain based on the following criteria:
- (1) most commonly used outcome measure across eligible trials;
- (2) most validated outcome; or
- (3) most precise estimate of treatment effect.
After consulting with our expert panel members, the CAPS will be preferred over any other scale for assessing PTSD symptoms. In the case of multiple follow-ups, we will collect data on the longest follow-up.
2.4 Risk of bias assessment
Reviewers will assess risk of bias among the eligible studies using the Cochrane risk of bias instrument that has modified response options of “definitely or probably yes” – considered as low risk of bias – or “definitely or probably no” – considered as high risk of bias. We will evaluate the following risk of bias issues: random sequence generation, allocation concealment, blinding of study participants, personnel, outcome assessors, and data analysts; selective reporting, and incomplete outcome data (>20% missing participant's data). We will assess risk of bias for each outcome separately on a component-by-component basis.
2.5 Data synthesis
We will do drug adjudication to collapse similar interventions or drugs belonging to the same class into 1 treatment node. For dichotomous outcomes (eg, percent patients on partial or complete sick leave, percent patient with partial or complete functional recovery), we will calculate relative risk and absolute risk (using baseline risk estimates from the control arm of eligible studies), and the associated 95% confidence intervals (CIs) for outcomes. For studies reporting continuous outcomes (eg, PTSD symptom severity, quality of life such as SF-36 (PCS and MCS), disability, social, and occupational impairment when the same instrument is used, we will calculate the weighted mean difference, and the associated 95% CIs. For trials that use different instruments for the same underlying construct such as functional recovery, we will convert all outcomes to a common instrument based on the recommendations by Thorlund et al.
We will contact study authors in case data is not completely reported or will use methods suggested by Cochrane handbook and Hozo et al to impute missing standard deviations when P-values, t-values, CIs, range, or standard errors (SEs) are reported in articles.
2.6 Methods for direct comparisons
We will perform standard pairwise meta-analyses using the DerSimonian–Laird random-effects model for all outcomes with at least 2 studies. We will determine statistical heterogeneity using the Q statistic and I2. For each direct comparison, we will report study and participant characteristics, risk of bias findings, and pooled estimates for outcomes of interest.
2.7 Methods for multiple treatment comparisons (NMA)
To assess the comparative effectiveness of competing treatments, we will perform random-effects NMA using the frequentist approach.[56–59]
Although the assumptions for NMA are similar to conventional meta-analysis, key extra assumptions are transitivity (there are no effect modifiers influencing the indirect comparisons) and coherence, (direct and indirect effect estimates are similar). We will identify incoherence comparing direct evidence (ie, estimates from pairwise comparisons) with indirect evidence (ie, estimates form network meta-analysis) using the node splitting method. In this approach, the incoherence will be assessed locally within each closed loop of the network separately as the difference between direct and indirect estimates for a specific comparison in the loop.[56–57] We will use a Wald test to test any statistical difference between the direct and the indirect estimates.
We will report our findings with probability statements of intervention effects. Probability rankings allow us to report a chance percentage of which interventions rank higher; however, simplifying the results of a network down to probabilities can lead to misinterpretations, specifically, when particular comparisons (ie, nodes) are not well-connected and/or when certainty in evidence varies between comparisons. Following display of the rank probabilities using rankogram, we will use the SUCRA to aid in the interpretation of relative effect of the interventions; an intervention with a SUCRA value of 100 is certain to be the best, whereas an intervention with 0 is certain to be the worst. We will use STATA (StataCorp, Release 15.1, College Station, TX) for statistical analyses.
2.8 Certainty (quality) of evidence
We will use the GRADE approach to rate the certainty in evidence of direct, indirect, and network estimates[61–64] on an outcome-by-outcome basis that classifies evidence as “high,” “moderate,” “low,” or “very low.” The starting point for certainty in estimates from randomized trials is high, but maybe rated down based on limitations in risk of bias, imprecision, inconsistency, and indirectness, and publication bias.
We will follow detailed guidance that the GRADE working group has provided for rating the quality of treatment effect estimates from network meta-analysis. In brief, the rating consists of 4 steps: first, we will present direct and indirect treatment estimates for each comparison of the evidence network. The direct estimate of effect is provided by a head-to-head comparison (trials of A vs B), and the indirect estimate is provided by 2 or more head-to-head comparisons that share a common comparator (for example, we infer the effects of A vs B from trials of A vs C and trials of B vs C). Second, we will rate the quality of each direct and indirect effect estimate; third, we will present the NMA estimate for each comparison of the evidence network and finally, we will rate the quality of each NMA effect estimate.
We will base hierarchy of the treatment options in the NMA according to 3 categories:
- (1) those that are clearly superior;
- (2) those with intermediate effectiveness, and
- (3) those that are inferior.
Treatments no better than placebo will be categorized in the lowest tier; treatments better than placebo will be categorized in the intermediate tier; whereas those superior to at least 1 tier 1 treatment will be judged superior. Furthermore, treatments will be categorized according to the quality of evidence such as high and moderate versus low or very low. Interventions with high quality or moderate evidence will be ranked as either “among the most effective,” “inferior to the most effective/superior to the least effective,” or “among the least effective.” Interventions supported by low or very low quality evidence will be ranked into the same 3 categories but prefaced with “may be” to acknowledge the reduced confidence in supporting evidence (eg, “may be among the most effective”) and will be presented separately from those supported by moderate or high quality evidence.
We will assess small study effects in direct comparisons using the Harbord test for dichotomous outcomes and the Egger test for continuous outcome when at least 10 studies are available.[62,64]
2.9 Subgroup and sensitivity analyses
We will test 7 a priori hypotheses to explain variability between studies:
- (1) patients in receipt of disability benefits and/or involved in litigation will show smaller effects than those not so involved;
- (2) trials at high risk of bias will show larger effects compared to trials at low risk of bias;
- (3) trials with longer follow-up will show smaller treatment effects;
- (4) trials that enrolled patients formally diagnosed with PTSD according to validated criteria (eg, DSM-III, DSM IV, DSM IV-R, DSM-5, CAPS, or symptom severity measures) will show smaller effects than trials that do not require formal diagnosis;
- (5) military and first-responder samples with PTSD will show smaller effects than civilian samples;
- (6) patients with PTSD and co-morbid substance abuse will show smaller effects than patients with PTSD without co-morbid substance abuse;
- (7) patients with PTSD and co-morbid traumatic brain injury will show smaller effects than patients with PTSD without co-morbid traumatic brain injury.
We have an additional a-priori to explore the small study effect we will run network meta-regression based on the SE.
2.10 Patient and public involvement
We have engaged an individual living with PTSD to help inform our research design, and when our review is completed we will ask them to assist in the interpretation and reporting of our findings. We plan to disseminate the results of our review to organizations supporting patients with PTSD, including Homewood Health and the Workers Compensation Board of Manitoba.
3 Discussion and knowledge translation
The results of our NMA will help clinicians and inform patients with PTSD about their therapeutic options, and facilitate informed health management decisions. The results of our review will be of interest to a broad audience including patients with PTSD, physicians, mental health clinicians, and third party payors – including insurers and compensation boards. Our review will identify key areas of future research and will provide a framework for conducting large systematic reviews involving indirect comparisons.
Our proposed review has several strengths. First, we will explore all currently available nonpharmacological and pharmacological treatment options for PTSD reported among eligible trials. Second, we will update the search to present date. Third, we will use the GRADE approach to evaluate the quality of evidence supporting treatment effects. Fourth, we will ensure interpretability by presenting risk differences and measures of relative effect for all pooled outcomes, and by presenting our findings with GRADE evidence profiles. Moreover, in order to explore the heterogeneity in our effect estimates, we have set 7 a-priori hypotheses and will conduct meta-regression and subgroup analyses consistent with best current practices.
The possible limitations of our review include the potential shortcomings of primary studies such as the presence of publication bias, high heterogeneity, and poor quality of reporting. Another likely limitation, unique to multiple treatment comparison meta-analyses, will be the nature of available treatment comparisons to build robust networks for our analyses.
Conceptualization: Yasir Rehman, Behnam Sadeghirad, Jason Busse, Gordon Guyatt.
Funding acquisition: Jason Busse, Yasir Rehman, Gordon Guyatt.
Methodology: Yasir Rehman, Behnam Sadeghirad, Jason Busse, Gordon Guyatt.
Supervision: Yasir Rehman, Behnam Sadeghirad, Jason Busse.
Writing – original draft: Yasir Rehman, Behnam Sadeghirad, Jason Busse.
Writing – review and editing: Yasir Rehman, Behnam Sadeghirad, Jason Busse, Gordon Guyatt, Margaret McKinnon, Randi McCabe, Ruth Lanius, Donald Richardson.
. Stein DJ, McLaughlin KA, Koenen KC, et al. DSM-5 and ICD-11 definitions of posttraumatic stress disorder: investigating “narrow” and “broad” approaches. Depress Anxiety 2014;31:494–505.
. Pai A, Suris AM, North CS. Posttraumatic stress disorder in the DSM-5: controversy, change, and conceptual considerations. Behav Sci (Basel) 2017;7:7.
. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed.Arlington, VA: American Psychiatric Publishing; 2013.
. Ehlers A, Clark DM. A cognitive model of posttraumatic stress disorder. Behav Res Ther 2000;38:319–45.
. Armour C, Karstoft KI, Richardson JD. The co-occurrence of PTSD
and dissociation: differentiating severe PTSD
. Soc Psychiatry Psychiatr Epidemiol 2014;49:1297–306.
. Blevins CA, Weathers FW, Witte TK. Dissociation and posttraumatic stress disorder: a latent profile analysis. J Trauma Stress 2014;27:388–96.
. Frewen PA, Brown MFD, Steuwe C, et al. Latent profile analysis and principal axis factoring of the DSM-5 dissociative subtype. Eur J Psychotraumatol 2015;6:26406.
. Lanius RA, Brand B, Vermetten E, et al. The dissociative subtype of posttraumatic stress disorder: rationale, clinical and neurobiological evidence, and implications. Depress Anxiety 2012;29:701–8.
. Lanius RA, Vermetten E, Loewenstein RJ, et al. Emotion modulation in PTSD
: clinical and neurobiological evidence for a dissociative subtype. Am J Psychiatry 2010;167:640–7.
. Putnam FW, Carlson EB, Ross CA, et al. Patterns of dissociation in clinical and nonclinical samples. J Nerv Ment Dis 1996;184:673–9.
. Spiegel D, Lewis-Fernández R, Lanius R, et al. Dissociative disorders in DSM-5. Annu Rev Clin Psychol 2013;9:299–326.
. Steuwe C, Lanius RA, Frewen PA. Evidence for a dissociative subtype of PTSD
by latent profile and confirmatory factor analyses in a civilian sample. Depress Anxiety 2012;29:689–700.
. Tsai J, Armour C, Southwick SM, et al. Dissociative subtype of DSM-5 posttraumatic stress disorder in U.S. veterans. J Psychiatr Res 2015;66-67:67–74.
. Wolf EJ, Lunney CA, Miller MW, et al. The dissociative subtype of PTSD
: a replication and extension. Depress Anxiety 2012;29:679–88.
. Wolf EJ, Miller MW, Reardon AF, et al. A latent class analysis of dissociation and posttraumatic stress disorder: evidence for a dissociative subtype. Arch Gen Psychiatry 2012;69:698–705.
. Ameringen M, Mancini C, Patterson B, et al. Post-traumatic stress disorder in Canada. CNS Neurosci Ther 2008;14:171–81.
. Brunellos N, Davidson JRT, Deahl M, et al. Posttraumatic stress disorder: diagnosis and epidemiology, comorbidity, and social consequences, biology and treatment. Neuropsychobiology 2001;43:150–62.
. Corneil W, Beaton R, Murphy S, et al. Exposure to traumatic incidents and prevalence of posttraumatic stress symptomatology in urban firefighters in two countries. J Occup Health Psychol 1999;4:131–41.
. Bryant RA, Harvey AG. Posttraumatic stress reactions in volunteer firefighters. J Trauma Stress 1996;9:51–62.
. Vedantham K, Brunet A, Boyer R, et al. Posttraumatic stress disorder, trauma exposure, and the current health of Canadian bus drivers. Can J Psychiatry 2001;46:149–55.
. Hull AM, Alexander DA, Klein S. Survivors of the Piper Alpha oil platform disaster: long-term follow-up study. Br J Psychiatry 2002;181:433–8.
. Kunst MJ. Employment status and posttraumatic stress disorder following compensation seeking in victims of violence. J Interpers Violence 2011;26:377–93.
. Matthews LR. Work potential of road accident survivors with post-traumatic stress disorder. Behav Res Ther 2005;43:475–83.
. Wald J. Work limitations in employed persons seeking treatment for chronic posttraumatic stress disorder. J Trauma Stress 2009;18:312–5.
. Dewa C, Keefe V, Small K. Choosing to work when sick: workplace presenteeism. Soc Sci Med 2005;60:2273–82.
. Amaya-Jackson L, Davidson JR, Hughes DS, et al. Functional impairment and utilization of services associated with post traumatic stress in the community. J Trauma Stress 1999;12:709–24.
. Breslau N. Outcomes of posttraumatic stress disorder. J Clin Psychiatr 2001;62(Suppl 17):55–9.
. Wilson S, Guliani H, Boichev G. On the economics of post-traumatic stress disorder among first responders in Canada. J Commun Safety Well-Being 2016;1:26–31.
. MacDonald HA, Colotla V, Flamer S, et al. Posttraumatic stress disorder (PTSD
) in the workplace: a descriptive study of workers experiencing PTSD
resulting from work injury. J Occup Rehabil 2003;13:63–77.
. Friedland JF, Dawson DR. Function after motor vehicle accidents: a prospective study of mild head injury and posttraumatic stress. J Nerv Ment Dis 2001;189:426–34.
. Amoroso T, Workman M. Treating posttraumatic stress disorder with MDMA-assisted psychotherapy: a preliminary meta-analysis and comparison to prolonged exposure therapy. J Psychopharmacol 2016;30:595–600.
. Cusack K, Jonas DE, Forneris CA, et al. Psychological treatments for adults with posttraumatic stress disorder: a systematic review and meta-analysis. Clin Psychol Rev 2016;43:128–41.
. George KC, Kebejian L, Ruth LJ, et al. Meta-analysis of the efficacy and safety of prazosin versus placebo for the treatment of nightmares and sleep disturbances in adults with post-traumatic stress disorder. J Trauma Dissociation 2016;17:494–510.
. Gu W, Wang C, Li Z, et al. Pharmacotherapies for posttraumatic stress disorder: a meta-analysis. J Nerv Ment Dis 2016;204:331–8.
. Kuester A, Niemeyer H, Knaevelsrud C. Internet-based interventions for posttraumatic stress: a meta-analysis of randomized controlled trials. Clin Psychol Rev 2016;43:1–6.
. O’Toole SK, Solomon SL, Bergdahl SA. A meta-analysis of hypnotherapeutic techniques in the treatment of PTSD
symptoms. J Trauma Stress 2016;29:97–100.
. Trevizol AP, Barros MD, Silva PO, et al. Transcranial magnetic stimulation for posttraumatic stress disorder: an updated systematic review and meta-analysis. Trends Psychiatry Psychother 2016;38:50–5.
. Cipriani A, Williams T, Nikolakopoulou A, et al. Comparative efficacy and acceptability of pharmacological treatments for post-traumatic stress disorder in adults: a network meta-analysis
. Psychol Med 2018;48:1975–84.
. Gerger H, Munder T, Gemperli A, et al. Integrating fragmented evidence by network meta-analysis
: relative effectiveness of psychological interventions for adults with post-traumatic stress disorder. Psychol Med 2014;44:3151–64.
. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med 2015;162:777–84.
. Weathers FW, Bovin MJ, Lee DJ, et al. The clinician-administered PTSD
scale for DSM-5 (CAPS
-5): development and initial psychometric evaluation in military veterans. Psychol Assess 2018;30:383–95.
. Weathers FW, Keane TM, Davidson JR. Clinician administered PTSD
scale: a review of the first ten years of research. Depress Anxiety 2001;13:132–56.
. Blake DD, Weathers FW, Nagy LM, et al. A clinician rating scale for assessing current and lifetime PTSD
: the CAPS
-1. Behavior Therapist 1990;13:187–8.
. Blanchard EB, Jones-Alexander J, Buckley TC, et al. Psychometric properties of the PTSD
Checklist (PCL). Behav Res Ther 1996;34:669–73.
. Ware J Jr, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Medical Care 1996;34:220–3.
. WHOQoL Group. WHOQoL-Brief: Introduction, Administration, Scoring, and Generic Version of the Assessment. Geneva: World Health Organization; 1996.
. Stevanovic D. Quality of life
enjoyment and satisfaction questionnaire – short form for quality of life
assessments in clinical practice: a psychometric study. J Psychiatr Ment Health Nurs 2011;18:744–50.
. Sheehan DV, Lecrubier Y, Sheehan KH, et al. The mini-international neuropsychiatric interview (M.I.N. I. ): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59(Suppl 20):22–33.
. Akl EA, Sun X, Busse JW, et al. Specific instructions for estimating unclearly reported blinding status in randomized trials were reliable and valid. J Clin Epidemiol 2012;65:262–7.
. The Cochrane Collaboration, Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions, Version [5.1.0] (updated March 2011). 2011.
. Dechartres A, Altman DG, Trinquart L, et al. Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA 2014;312:623–30.
. Thorlund K, Walter SD, Johnston BC, et al. Pooling health-related quality of life
outcomes in meta-analysis-a tutorial and review of methods for enhancing interpretability. Res Synth Methods 2011;2:188–203.
. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from: www.handbook.cochrane.org
. Accessed July 31, 2019.
. Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol 2005;5:13doi:10.1186/1471-2288-5-13.
. Jackson D, Bowden J, Baker R. How does the DerSimonian and Laird procedure for random effects meta-analysis compare with its more efficient but harder to compute counterparts? J Stat Plan Infer 2010;140:961–70.
. Chaimani A, Salanti G. Visualizing assumptions and results in network meta-analysis
: the network graphs package. Stata J 2015;15:905–50.
. White IR. Network meta-analysis
. Stata J 2015;15:951–85.
. Chaimani A, Higgins JP, Mavridis D, et al. Graphical tools for network meta-analysis
in STATA. PLoS One 2013;8:e76654.
. White IR, Barrett JK, Jackson D, et al. Consistency and inconsistency in network-meta analysis: model estimation using multivariate meta-regression. Res Syn Meth 2012;3:111–25.
. Donegan S, Williamson P, D’Alessandro U, et al. Assessing key assumptions of network meta-analysis
: a review of methods. Res Synth Methods 2013;4:291–323.
. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924–6.
. Puhan MA, Schunemann HJ, Murad MH, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
. BMJ 2014;349:g5630.
. Brignardello-Petersen R, Bonner A, Alexander PE, et al. Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis
. J Clin Epidemiol 2018;93:36–44.
. Harbord RM, Harris RJ, Sterne JAC, et al. Updated tests for small-study effects in meta-analyses. Stata J 2009;9:197–210.