Schizophrenia has remained a chronic and often severely impairing mental disorder despite the development of effective antipsychotic (AP) treatments. One of the reasons for relapses is nonadherence with prescribed treatment (Kane et al., 2013a). To improve treatment adherence and outcomes, long-acting injectable (LAI) formulations of APs have been developed. The potential benefit of treatment delivered as a LAI versus a daily orally administered AP agent lies in advantages associated with removing the need for daily medication administration and signaling the clinician when nonadherence occurs. Treatment discontinuations (Zipursky et al., 2014) and intermittent treatment (Sampson et al., 2013) have been associated with increased relapses. Treatment with LAIs should increase the likelihood of continuous effective exposure over extended periods. An increasing number of published studies have compared the effects of LAI and oral APs in patients with schizophrenia. On the basis of the association between nonadherence and relapse, these studies hypothesized an advantage for the LAI treatment. Although mirror-image studies, which arguably include more broadly representative patient populations, have reported advantages on the basis of this difference in modality (Kishimoto et al., 2013), randomized-controlled trials have frequently failed to show advantages (Fusar-Poli et al., 2013; Kirson et al., 2013; Kishimoto et al., 2014; Buckley et al., 2015).
Although highly controlled studies are the gold standard to address many clinical research questions, we believe that more pragmatic approaches are required to address questions associated with adherence. Pragmatic (often referred to as effectiveness) studies aim for a high degree of external validity, seeking to answer whether an intervention works under usual clinical or ‘real-world’ conditions. In contrast, explanatory (often referred to as efficacy) studies aim for a high degree of internal validity, exploring whether an intervention works under more constrained conditions. To achieve this goal, explanatory trials are conducted under highly controlled and well-defined treatment conditions, which are necessary to minimize ambiguity and address the primary questions for which this type of trial is designed. They typically include populations that do not reflect the full clinical population in which the intervention will be used. Design elements inherent to explanatory trials may obscure factors that drive the advantage of certain treatment approaches. This is particularly true for studies that address the common issue of nonadherence. For example, the clinical advantage of ensured longer exposure to therapeutic doses with long-acting formulations of AP medications compared with oral formulations may not be evident in an explanatory trial that strongly reinforces adherence. Other explanatory design features that may obscure differences that occur under real-world conditions may include the frequent use of extensive but burdensome healthcare assessments, exceptionally close follow-up and reconnection with the patient, and financial incentives for patient participation. In addition, selection bias may result from the enrollment of participants in clinical trials who tend to be more adherent to research procedures. Individuals with less severe illness and greater insight into their illness may also be more likely to adhere to their assigned treatment regimen (Kane et al., 2013b).
Understanding the inconsistent body of literature comparing LAI and daily oral APs has been the focus of several recent publications. Although an earlier meta-analysis found a significant benefit of LAI versus daily oral APs (Leucht et al., 2011), two larger and more recent meta-analyses of randomized-controlled trials concluded that there is no advantage for LAI formulations in preventing relapse and hospitalization (Fusar-Poli et al., 2013; Kishimoto et al., 2014). The focus of these analyses on controlled, randomized studies likely resulted in a bias toward inclusion of highly explanatory trials. Some authors note that their findings contrast with those of recent naturalistic mirror-image and cohort studies, and suggest that pragmatic trial designs be utilized in future research to be more reflective of actual clinical care received by patients with schizophrenia (Kane et al., 2013b; Kishimoto et al., 2014; Buckley et al., 2015). In particular, these authors expressed concern that patients undergoing intensive consent and assessment procedures may be more adherent and less severely ill than those encountered in everyday practice. Consequently, they suggest that using a LAI AP formulation in a naturalistic setting might confer additional benefit over the corresponding daily oral formulation (Kane et al., 2013a). Supporting this consideration, in randomized-controlled trials where adherence was formally assessed, no differences were observed in adherence between LAI and daily oral AP formulations (Leucht et al., 2011; Kishimoto et al., 2014).
Recently, a meta-analysis by Kirson et al. (2013) was published that included studies of varying designs (randomized-controlled, prospective observational, and retrospective observational trials). These authors reported significant advantages for LAI treatments studied in trials with observational designs, but not in those with randomized-controlled designs. These conclusions are supported in a recent meta-analysis by Kishimoto et al. (2013) with 25 mirror-image studies in which 22 showed significant advantages of the LAI versus daily oral AP treatment for preventing psychiatric hospitalization. However, the authors acknowledge that mirror-image studies can also be biased by the fact that treatment status is not blinded, thresholds for hospitalization can change over time, and that LAIs are always started after suboptimal outcomes on daily oral APs. They also note that reverse mirror-image studies (i.e. from LAI to oral formulation) are lacking.
None of these meta-analyses used a formalized measure of the explanatory or the pragmatic nature of specific trial design features. In practice, most trial designs are neither purely explanatory nor purely pragmatic. Instead, most lie along a continuum between these two extremes. The research reported here uses a novel approach for quantifying an individual study’s design along this continuum and examines the hypothesis that a more pragmatic design is important for showing advantages for LAI versus daily oral AP treatment.
The objective of this review was to identify comparative studies of the clinical efficacy of LAI versus daily oral APs. Selection criteria included studies published from 1993 to 2013, whose duration was 6 months or longer, and that had enrolled at least 100 patients with a diagnosis of schizophrenia. The publication period reflects a time when clinical trial designs were likely to be better described and when treatment modalities that are reflective of current realities were studied. The requirement for a 6-month or longer duration of follow-up was imposed to provide an adequate period for observing potential differences between long-acting and daily oral AP treatments. The 100-patient enrollment criterion was incorporated to increase the likelihood that the study would be sufficiently powered to detect meaningful differences between treatments.
This literature review consisted of three components: (i) a search engine-based literature review; (ii) an examination of relevant review articles; and (iii) any other published studies known to the authors (Fig. 1). The literature search was performed using MEDLINE/PubMed. Search terms and criteria were as follows: (((Antipsychotic) AND schizophrenia) AND ((depot OR injection OR long-acting))) AND oral. Filters included clinical trial, human, English language, and publication dates of 1 January 1993 to 31 December 2013. The manual review of citations identified by MEDLINE/PubMed removed those that: (i) did not include both a LAI and an oral AP treatment arm; (ii) did not include a measure of clinical efficacy or effectiveness; (iii) represented findings from a pooled analysis (vs. a single study); (iv) had a duration of less than 6 months; (v) enrolled less than 100 participants; (vi) were not in English; and (vii) were a secondary publication of a previously included study (i.e. post-hoc subpopulation data). This literature search was then supplemented by an examination of references cited in relevant review articles and any other published studies known to the authors through December 2013.
ASPECT-R, the tool
‘A Study Pragmatic : Explanatory Characterization Tool-Rating’ or ASPECT-R ((c) 2014 Janssen Pharmaceuticals, Inc., Titusville, New Jersey, USA) is a tool informed by the PRECIS tool (Thorpe et al., 2009; Tosh et al., 2011) that characterizes the explanatory : pragmatic nature of a study’s design (L.D. Alphs and C.A. Bossie, 2015, submitted). ASPECT-R considers six study design domains important to the conduct of clinical trials along the explanatory : pragmatic spectrum: (i) participant eligibility criteria; (ii) intervention flexibility; (iii) medical practice setting/practitioner expertise; (iv) follow-up intensity/duration; (v) primary trial outcomes; and (vi) participant compliance assessment. Each domain is rated using a detailed anchored seven-point scale where 0=extremely explanatory; 1=very explanatory; 2=explanatory; 3=elements of both designs; 4=pragmatic; 5=very pragmatic; and 6=extremely pragmatic. Specific descriptive anchors for each of the seven ratings are provided for each of the six domains.
The interclass correlation of the ASPECT-R total score is 0.87 (C.A. Bossie, L.D. Alphs, D. Williamson, L. Mao, C. Kurut, the ASPECT-R Rater Team, 2015, submitted), which corresponds to an excellent inter-rater reliability (Cicchetti, 1994). The domains included in ASPECT-R are generally accepted trial design elements relevant for distinguishing pragmatic and explanatory trials, as evidenced by peer-reviewed publications (Thorpe et al., 2009; Tosh et al., 2011; Lurie and Morgan, 2013; Roche et al., 2013; Alphs et al., 2014; Sedgwick, 2014), which lend support for the face validity of ASPECT-R.
Full references of the studies identified were used as the source information for rating the study designs with the ASPECT-R tool. Two of the authors (C.A.B. and L.D.A.) independently rated each of the studies identified by the literature review using the ASPECT-R and then compared their ratings. Differences in domain ratings were resolved through a consensus rating process. The basis of the consensus ratings for each domain for each study was documented.
Illustrating ASPECT-R ratings relative to study results
ASPECT-R consensus ratings for each study were plotted using radar graphs.
Studies were then categorized according to the outcome as reported in the original publication, yielding two groups: those showing an advantage for LAI over daily oral AP treatment and those not showing such an advantage. Total and domain ASPECT-R scores were compared across the two groups of studies using the nonparametric Wilcoxon rank-sum test to address the non-normal distribution of the scores. Data were analyzed in JMP5 (5.0.1, 1989–2003; SAS Institute Inc., Cary, North Carolina, USA). All tests were two-sided and α was set at 0.05. No adjustment was made for multiplicity.
Citation review and selection
Using the literature search terms and criteria summarized above, a total of 126 citations were identified through the MEDLINE/PubMed literature search. Three additional citations were identified through manual review of the reference lists of four meta-analyses (Leucht et al., 2011; Fusar-Poli et al., 2013; Kirson et al., 2013; Kishimoto et al., 2014). An additional citation (Bitter et al., 2013) was identified through one author’s (C.A.B.) general knowledge of the literature. Thus, a total of 130 citations were identified (Fig. 1).
One author (C.A.B.) and another contributor (S.R. in acknowledgments) reviewed the titles, abstracts, and full publication of these articles for compliance with search criteria and appropriateness of filters. A total of 119 citations were excluded as they did not fulfill the criteria as described in Fig. 1. The remaining 11 study citations (N=21 159 participants) included: Zhu et al. (2008), Olivares et al. (2009), Gaebel et al. (2010), Kane et al. (2010), Macfadden et al. (2010), Tiihonen et al. (2011), Rosenheck et al. (2011), Grimaldi-Bensouda et al. (2012), Bitter et al. (2013) (methods described in Ascher-Svanum et al., 2006), Keks et al. (2007), and Tiihonen et al. (2006). Study design features and main findings for these 11 studies are summarized in Table 1. The 11 studies were placed into two groups: those that showed a difference between LAI and daily oral AP treatments [seven studies, 18 829 participants (range 297–9567)] and those that did not [four studies, studies, 2330 participants (range 349–1065)].
ASPECT-R ratings of the seven studies concluding a benefit of LAI versus daily oral APs are shown in Fig. 2. Ratings of the four studies concluding no LAI versus daily oral AP difference are shown in Fig. 3. Total ASPECT-R scores (maximum possible score=36) ranged from 18 to 36 in the former group of studies and from 9 to 13 in the latter group (Table 2).
In five of the seven studies concluding a benefit of LAI compared with daily oral AP treatment, all domains were rated as more pragmatic (i.e. ASPECT-R ratings of 4, 5, or 6; Tiihonen et al., 2006; Zhu et al., 2008; Olivares et al., 2009; Tiihonen et al., 2011; Bitter et al., 2013). In one study, most domains were rated as more pragmatic (Grimaldi-Bensouda et al., 2012). In one study, domains were variously characterized as more pragmatic or more explanatory (Gaebel et al., 2010).
In three of the four studies concluding no benefit for LAI compared with daily oral AP treatment, most domains were rated as more explanatory (i.e. ASPECT-R rating of 0, 1, or 2; Keks et al., 2007; Kane et al., 2010; Macfadden et al., 2010). In one study, domains were variously characterized as more pragmatic or more explanatory (Rosenheck et al., 2011).
The mean ASPECT-R total score and individual domain scores were significantly higher (more pragmatic) in the seven studies finding an advantage of LAIs over daily oral APs compared with the four studies that did not (Table 2). The rank order of greatest significant differences in the six domains between the two groups of studies was as follows: ‘participant compliance assessment’ (P=0.005), ‘medical practice setting/practitioner expertise’ (P=0.006), ‘intervention flexibility’ (P=0.007), ‘follow-up intensity/duration’ (P=0.009), ‘primary trial outcomes’ (P=0.012), and ‘participant eligibility’ (P=0.015).
A novel quantitative approach was used to examine the hypothesis that a more pragmatic study design is important for showing the advantages of LAI over oral AP treatment for patients with schizophrenia who are frequently nonadherent, increasing the risk of relapse. Theoretical advantages of LAIs are associated with removing the need for daily adherence. Several meta-analytic approaches have been used to examine this question, with mixed conclusions (Leucht et al., 2011; Fusar-Poli et al., 2013; Kirson et al., 2013; Kishimoto et al., 2014). This report describes the application of a new tool, ASPECT-R (L.D. Alphs, C.A. Bossie, 2015, submitted; C.A. Bossie, L.D. Alphs, D. Williamson, L. Mao, C. Kurut, the ASPECT-R Rater Team, 2015, submitted), which quantifies the pragmatic : explanatory nature of a study’s design and explores the relevance of the result to treatment failure, including relapse, hospitalization, and treatment discontinuation. The findings presented here support a hypothesis that explanatory designs introduce features that obscure advantages related to medication treatment adherence, whereas pragmatic design features enable identification of these advantages for LAIs that would be expected in a naturalistic setting for patients who clinicians would select for this treatment. In fact, the range of ASPECT-R total scores for the two groups of studies did not overlap (Table 2).
On the basis of the expected advantage of LAI AP treatment, it was hypothesized that the ‘Participant Compliance Assessment’ domain would be the most differentiating between two groups of studies. Findings were consistent with this hypothesis (P=0.005), although the mean scores for all domains differed significantly between the two groups.
Several limitations of this work must be considered. Studies with conventional (typical) depot AP agents were not well represented (i.e. three studies: Tiihonen et al., 2006; Zhu et al., 2008; Tiihonen et al., 2011). Consequently, it is unclear to what degree findings would translate to work with conventional depot APs. Nevertheless, Kishimoto et al. (2014) have noted that studies of first-generation LAIs [fluphenazine (n=8) and haloperidol (n=1)] show a significant benefit for LAI over oral treatment. Second, only the consensus ratings of two authors (C.A.B., L.D.A.) who developed the ASPECT-R were used for this analysis. Consequently, ASPECT-R ratings found in this study may not be representative of ratings from individuals less familiar with the instrument. However, a recently completed inter-rater reliability assessment with novice, but trained raters found an interclass correlation of 0.87, which corresponds to an excellent inter-rater reliability (C.A. Bossie, L.D. Alphs, D. Williamson, L. Mao, C. Kurut, the ASPECT-R Rater Team, 2015, submitted). Finally, relevant information to fully establish ASPECT-R ratings may not have been fully documented in the primary reports used for this study. Lack of access to source documentation, such as trial protocols, may impact ASPECT-R scores and the ability to assess all domains accurately.
Criteria for our literature search included a 20-year publication date range (1 January 1993 to 31 December 2013). However, a recently published study (PROACTIVE; Buckley et al., 2015) is quite important and relevant to our research question and requires comment (Buckley et al., 2015). The authors state that their time to relapse or hospitalization study of patients with schizophrenia randomized to either a LAI (risperidone) or an oral AP incorporated both explanatory and pragmatic design features. As such, and similar to the findings of the four studies in this analysis that found no difference (Keks et al., 2007; Kane et al., 2010; Macfadden et al., 2010; Rosenheck et al., 2011), these investigators found no significant difference in either time to relapse or hospitalization, and add that their study design is similar to several of these earlier trials. Many of their study design characteristics leaned strongly toward a more explanatory trial, such as uniform and frequent monitoring (i.e. every 2-week office visits) and LAI informed consent treatment requirements that may have diluted the potential for those with documented nonadherence to enroll. In their discussion, the authors acknowledge that these explanatory study design characteristics may have resulted in the enrollment of patients who are more engaged in their care, with a reduced inclusion of participants with documented nonadherence. These types of patients are less likely to stop taking oral medication, making it more difficult to detect differences between the LAI and oral treatment.
In conclusion, this research adds to the previous literature by providing a novel and informative approach that quantifies the pragmatic : explanatory design of studies that compare LAI and oral APs for the treatment of schizophrenia. Previous meta-analytic approaches applied to these studies are based on study results without a detailed and quantitative reference to their specific design and methodological features. The use of ASPECT-R represents a very different approach by providing a structured quantification of specific design elements, without consideration of study results (L.D. Alphs, C.A. Bossie, 2015, submitted; C.A. Bossie, L.D. Alphs, D. Williamson, L. Mao, C. Kurut, the ASPECT-R Rater Team, 2015, submitted). These two distinct approaches to address the same question are complementary and provide more information than either approach alone. Although highly controlled studies remain the gold standard for evidence-based trial designs to answer most questions in medicine and psychiatry, pragmatic study design elements are arguably more valuable for addressing questions such as those related to real-world populations, practice, and outcomes, especially when the primary target is enhancing adherence. Their use can add to the generalizability of available evidence. Our findings suggest that pragmatic study characteristics are important in showing the expected advantage of LAI over daily oral AP treatment in schizophrenia.
Editorial support was provided by Susan Ruffalo, PharmD, MedWrite Inc. (Newport Coast, California, USA). The authors acknowledge the contributions of Joseph Hulihan, MD, for his contributions toward the early development and early use of the ASPECT-R.
Funding for this research was provided by Janssen Scientific Affairs, LLC.
Conflicts of interest
Drs Bossie and Alphs disclose that they are full-time employees of Janssen Scientific Affairs, LLC and that they are stockholders in Johnson & Johnson. They were responsible for the research design, development of the ASPECT-R instrument, the collection and analysis of the data, and the decision to publish these findings, and the drafting of the manuscript and review of the final manuscript version.
Dr Correll discloses that he has been a consultant and/or advisor to or has received honoraria from AbbVie, Alkermes, Bristol-Myers Squibb, Eli Lilly, Genentech, Gerson Lehrman Group, IntraCellular Therapies, Janssen/J&J, Lundbeck, MedAvante, Medscape, Otsuka, Pfizer, ProPhase, Reviva, Roche, Sunovion, Supernus, and Takeda. He has received grant support from BMS, Otsuka, and Takeda. Dr Correll was involved in the analyses of these data, the drafting of the manuscript, and review of the final manuscript.
Alphs L, Schooler N, Lauriello J (2014). How study designs influence comparative effectiveness outcomes: the case of oral versus long-acting injectable antipsychotic treatments for schizophrenia. Schizophr Res 156:228–232.
Ascher-Svanum H, Zhu B, Faries D, Landbloom R, Swartz M, Swanson J (2006). Time to discontinuation of atypical versus typical antipsychotics in the naturalistic treatment of schizophrenia. BMC Psychiatry 6:8.
Bitter I, Katona L, Zámbori J, Takács P, Fehér L, Diels J, et al. (2013). Comparative effectiveness of depot and oral second generation antipsychotic drugs in schizophrenia: a nationwide study in Hungary. Eur Neuropsychopharmacol 23:1383–1390.
Buckley PF, Schooler NR, Goff DC, Hsiao J, Kopelowicz A, Lauriello J, et al. (2015). Comparison of SGA oral medications and a long-acting injectable SGA: the PROACTIVE study. Schizophr Bull 41:449–459.
Cicchetti DV (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psych Assess 6:284–290.
Fusar-Poli P, Kempton MJ, Rosenheck RA (2013). Efficacy and safety of second-generation long-acting injections in schizophrenia: a meta-analysis of randomized-controlled trials. Int Clin Psychopharmacol 28:57–66.
Gaebel W, Schreiner A, Bergmans P, de Arce R, Rouillon F, Cordes J, et al. (2010). Relapse prevention in schizophrenia and schizoaffective disorder with risperidone long-acting injectable vs quetiapine: results of a long-term, open-label, randomized clinical trial. Neuropsychopharmacology 35:2367–2377.
Grimaldi-Bensouda L, Rouillon F, Astruc B, Rossignol M, Benichou J, Falissard B, et al. (2012). Does long-acting injectable risperidone make a difference to the real-life treatment of schizophrenia? Results of the Cohort for the General study of Schizophrenia (CGS). Schizophr Res 134:187–194.
Kane JM, Detke HC, Naber D, Sethuraman G, Lin DY, Bergstrom RF, McDonnell D (2010). Olanzapine long-acting injection: a 24-week, randomized, double-blind trial of maintenance treatment in patients with schizophrenia. Am J Psychiatry 167:181–189.
Kane JM, Kishimoto T, Correll CU (2013a). Non-adherence to medication in patients with psychotic disorders: epidemiology, contributing factors and management strategies. World Psychiatry 12:216–226.
Kane JM, Kishimoto T, Correll CU (2013b). Assessing the comparative effectiveness of long-acting injectable vs. oral antipsychotic medications in the prevention of relapse provides a case study in comparative effectiveness research in psychiatry. J Clin Epidemiol 66 (Suppl):S37–S41.
Keks NA, Ingham M, Khan A, Karcher K (2007). Long-acting injectable risperidone v. olanzapine tablets for schizophrenia or schizoaffective disorder. Randomised, controlled, open-label study. Br J Psychiatry 191:131–139.
Kirson NY, Weiden PJ, Yermakov S, Huang W, Samuelson T, Offord SJ, et al. (2013). Efficacy and effectiveness of depot versus oral antipsychotics
in schizophrenia: synthesizing results across different research designs. J Clin Psychiatry 74:568–575.
Kishimoto T, Nitta M, Borenstein M, Kane JM, Correll CU (2013). Long-acting injectable versus oral antipsychotics
in schizophrenia: a systematic review and meta-analysis of mirror-image studies. J Clin Psychiatry 74:957–965.
Kishimoto T, Robenzadeh A, Leucht C, Leucht S, Watanabe K, Mimura M, et al. (2014). Long-acting injectable vs oral antipsychotics
for relapse prevention in schizophrenia: a meta-analysis of randomized trials. Schizophr Bull 40:192–213.
Lurie JD, Morgan TS (2013). Pros and cons of pragmatic
clinical trials. J Comp Eff Res 2:53–58.
Leucht C, Heres S, Kane JM, Kissling W, Davis JM, Leucht S (2011). Oral versus depot antipsychotic drugs for schizophrenia – a critical systematic review and meta-analysis of randomised long-term trials. Schizophr Res 127:83–92.
Macfadden W, Ma YW, Thomas Haskins J, Bossie CA, Alphs L (2010). A prospective study comparing the long-term effectiveness of injectable risperidone long-acting therapy and oral aripiprazole in patients with schizophrenia. Psychiatry (Edgmont) 7:23–31.
Olivares JM, Rodriguez-Morales A, Diels J, Povey M, Jacobs A, Zhao Z, et al. (2009). Long-term outcomes in patients with schizophrenia treated with risperidone long-acting injection or oral antipsychotics
in Spain: results from the electronic Schizophrenia Treatment Adherence Registry (e-STAR). Eur Psychiatry 24:287–296.
Roche N, Reddel HK, Agusti A, Bateman ED, Krishnan JA, Martin RJ, et al. (2013). Integrating real-life studies in the global therapeutic research framework. Lancet Respir Med 1:e29–e30.
Rosenheck RA, Krystal JH, Lew R, Barnett PG, Fiore L, Valley D, et al. (2011). Long-acting risperidone and oral antipsychotics
in unstable schizophrenia. N Engl J Med 364:842–851.
Sampson S, Mansour M, Maayan N, Soares-Weiser K, Adams CE (2013). Intermittent drug techniques for schizophrenia. Cochrane Database Syst Rev 20:CD006196.
Sedgwick P (2014). Explanatory
trials versus pragmatic
trials. BMJ 349:g6694.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, et al. (2009). A pragmatic
continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 62:464–475.
Tiihonen J, Wahlbeck K, Lönnqvist J, Klaukka T, Loannidis JPA, Volavka J, et al. (2006). Effectiveness of antipsychotic treatments in a nationwide cohort of patients in community care after first hospitalization due to schizophrenia and schizoaffective disorder: observational follow-up study. Br Med J 333:224–231.
Tiihonen J, Haukka J, Taylor M, Haddad PM, Patel MX, Korhonen P (2011). A nationwide cohort study of oral and depot antipsychotics after first hospitalization for schizophrenia. Am J Psychiatry 168:603–609.
Tosh G, Soares-Weiser K, Adams CE (2011). Pragmatic
trials: the pragmascope tool to help measure differences in protocols of mental health randomized controlled trials. Dialogues Clin Neurosci 13:209–215.
Zhu B, Ascher-Svanum H, Shi L, Faries D, Montgomery W, Marder SR (2008). Time to discontinuation of depot and oral first-generation antipsychotics in the usual care of schizophrenia. Psychiatr Serv 59:315–317.
Zipursky RB, Menezes NM, Streiner DL (2014). Risk of symptom recurrent with medication discontinuation in first-episode psychosis: A systematic review. Schizophr Res 152:408–414.