Secondary Logo

Journal Logo

The Impact of Evidence-Based Education on Prescribing in a Psychiatry Residency

Benjamin, Dinesh, MD; Swartz, Marvin, MD; Forman, Leslie, MD

Journal of Psychiatric Practice®: March 2011 - Volume 17 - Issue 2 - p 110–117
doi: 10.1097/01.pra.0000396062.12893.5b

Objective Recent clinical trials comparing the effectiveness of antipsychotics have found no advantage for second-generation antipsychotics over older first-generation agents. However, the former are much more commonly used despite their significantly higher cost and potential for contributing to the metabolic syndrome. To date, educational interventions have been unsuccessful in influencing this pattern. The Duke University Medical Center Department of Psy chiatry began a program based on principles of academic detailing designed to educate psychiatry residents about generic psychotropics. To encourage residents to gain experience with these medications, samples of selected generic drugs were provided. To assess the initiative's impact, the authors measured the prescribing patterns of residents.

Methods We measured the amount of generic drug use 6 months after the program began and compared it with data from a 6-month control period. The data were analyzed based on overall psychotropic use, class of medication, site, and diagnosis.

Results We found a consistent increase in generic use across analyses. There was an increase in overall generic prescribing from 55.8% to 58.6% [X2=10.37, odds ratio (OR)=1.12, P=0.0013] and a particularly large increase in prescribing of generic antipsychotic medications from 39.5% to 47.7% [X2=36.12, OR=1.39, P<0.0001]. Conclusion. The implementation of this educational program was correlated with increasing use of generic medications and brought antipsychotic prescribing into concordance with the new evidence. This is the first such study in a psychiatry residency program and has implications for promoting cost-effective care while preserving patient choice in the mental health system. The findings from this study also suggest potential techniques for expanding residents' prescribing skills across specialties. (Journal of Psychiatric Practice. 2011;17:110–117)

Duke University Medical Center, Durham, NC

Financial disclosure: None of the investigators has any affiliations or financial involvement (eg, employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties) that conflict with material presented in the report.

Please send correspondence to: Dinesh Benjamin, MD, 4404 Dula Street, Durham, NC 27705.

Back to Top | Article Outline


Since their introduction in the 1990s, second-generation antipsychotics (SGAs) have become the drugs of choice in the treatment of schizophrenia at an annual cost of over $10 billion in the United States, 75% of which is paid through Medicaid.1 To place this figure in perspective, this is 1.3 times the combined annual professional income of all U.S. psychiatrists.2 To evaluate the effectiveness of antipsychotic medications, the National Institute of Mental Health initiated the Clinical Antipsychotics Trials of Inter vention Effective ness (CATIE), a randomized controlled trial of almost 1,500 subjects that compared the effectiveness of a first-generation antipsychotic (FGA) (perphenazine) and four SGAs (risperidone, olanzapine, quetiapine, and ziprasidone) over 18 months. While patients stayed on olanzapine longer than on the other antipsychotics, there were no differences between the other SGAs and perphenazine in clinical efficacy or side effects.3 However, perphenazine was far less expensive, with the SGAs costing approximately $2400–$6000 more per year.4 Similar large effectiveness trials in the U.S. Veterans Affairs5 system and in the United Kingdom6 have also found that the newer antipsychotic drugs have no advantages for clinical effectiveness but do involve a significant increase in cost. A recent meta-analysis of 150 double-blind randomized controlled trials compared nine SGAs with FGAs. They found that four SGAs-amisulpride, clozapine, olanzapine and risperidone-were more efficacious than the FGAs. However, the other SGAs that were examined-aripiprazole, quetiapine, sertindole, ziprasidone, and zotepine-were not significantly better than the FGAs.7

Recently Covell et al reviewed results from the CATIE study concerning the effectiveness of antipsychotic medications and the implications of those findings for influencing prescribers' behaviors. They highlighted the need to tailor prescribing to clinical and medical history as well as patient preference using a variety of agents, including FGAs.8 In psychiatry, reported barriers to using FGAs include lack of organizational support, characteristics of individual health care professionals, and perceptions of guidelines.9 To address these issues, the Duke University Medical Center Department of Psychiatry undertook an educational initiative to provide evidence-based information to psychiatry residents about generic medications. This program followed the principles of “academic detailing”, employing education to increase recipients' knowledge of generic drugs. While samples of newer SGAs have been available in areas where residents practice for many years, the department also made samples of selected generic medications available. This strategy was modeled on the principles of academic detailing and sampling that were also being used in the Study Assessing the Effect of Cardiovascular Medications Provided as Low-cost, Evidence-based Generic Samples (SAMPLES), in which primary care physicians are being provided with samples of two common generic cardiovascular medications.10

Interventions used in studies to bring about behavior change in public health are often not well described and their effects are often difficult to evaluate, limiting subsequent replication of successful interventions.11 Anticipating some of these difficulties, we designed a prospective observational study to measure effects on prescribing patterns as well as prescriber attitudes towards generic medications before and after the implementation of a generic drug prescribing program. The frequency of generic drug prescribing was used as a marker of change in the clinical practice of psychiatry residents. The study was designed to test the hypothesis that the proportion of generic medications prescribed by residents would increase after the educational initiative. This is the first such study of academic detailing in a psychiatric residency.

Back to Top | Article Outline


Setting and Participants

Quantitative data from clinical services, such as the inpatient ward, the consult service, and the outpatient clinics, were collected using the Duke University psychiatry database. Other sites where residents work, such as the state psychiatric hospital and the Veterans Affairs system, were excluded since their data were not readily available and they have pre-existing formulary restrictions on prescribing. Only trainees of the Duke University Medical Center Department of Psychiatry were included in the generic initiative. The subjects consisted of 48 trainees in the historical control period of 2008 and 52 trainees after the educational initiative was started in 2009. Since one class had graduated and a new class had joined the residency, a paired analysis was not done. This also resulted in a difference in the number of trainees in the control and intervention periods, but we did not believe this would be a significant confounder. The study was declared exempt from IRB review, since all data were de-identified and there was no active intervention.

Back to Top | Article Outline


The intervention consisted of two components: an educational initiative and the provision of generic medication samples. The generic prescribing initiative consisted of an educational program targeted at increasing residents' knowledge of generic medications, which was adapted into the current psychiatry teaching curriculum. The initiative consisted of (1) four journal club sessions of 1 hour each in which residents and faculty were given an opportunity to critically appraise the evidence concerning generic psychotropic medications and discuss its implications; (2) 1.5 hours of didactic teaching to the residents focused on content concerning dosing, side effects, and monitoring of generic medications, and (3) two sessions of 1.5 hours each of problem-based learning concerning generic drug use for first and second year residents. Attendance at each session was not measured.

In addition, samples of selected generic medications were made available for residents to use in both inpatient and outpatient settings These samples were chosen on the basis of their cost and to represent the three drug classes on which we were focusing in this study. Samples were provided for carbamazepine, lithium, valproic acid, citalopram, fluoxetine, nortriptyline, paroxetine, trazodone, sertraline, haloperidol, thiothixene, and benztropine. The educational program and the provision of samples were done by the Department of Psychiatry independently from this study.

Back to Top | Article Outline

Quantitative Data

The electronic database, Clinical Management Research Information System (CRIS), is used by the Department of Psychiatry at Duke University Medical Center to document clinical care. In this study, we used CRIS to assess the frequency of drug prescriptions by residents on the inpatient ward, in the consult service, in the outpatient psychiatry clinic, and in the emergency department. CRIS allows one to extract medical information such as date, site of care, medications, and diagnosis from patient charts without any identifying information, thus preserving both patient and clinician anonymity. This information has to be documented before the note can be signed, ensuring that there are no missing values. We collected data on the site, the primary diagnosis, and the medications prescribed at each documented encounter. To establish that any change in prescribing reflected the generic initiative, we used information from the previous academic year as a comparator. Since we anticipated that prescribing patterns among residents may vary with experience gained through the academic year, we compared data for the same time periods (ie, data from July to December 2008 were used as a control for data from July to December 2009). The educational initiative was begun in June 2009. The study was restricted to antidepressant, mood stabilizer, and antipsychotic medications since these classes of medications were the focus of the educational initiative.

Back to Top | Article Outline

Qualitative Data

An anonymous questionnaire was distributed to residents before the educational program began in June 2009 and then again 6 months later (see Table 1; note that respondents were asked to rate each drug class or specific type of drug separately). The questionnaire was designed to assess residents' level of comfort with prescribing generic drugs, by class as well as with regard to the medications for which samples were made available. The questions were scored on a simple Likert scale of 1 to 5. Scores 6 months after the intervention were compared with scores before the intervention began.



Back to Top | Article Outline

Statistical Analysis

Primary Analysis

Our primary outcome was the change in frequency of generic prescribing which was assessed using a chi square analysis. An odds ratio (OR) and a P value were also calculated. Given the strength of the recent evidence for FGAs, we anticipated that there was likely be a large change frequency in this drug class. Therefore we analyzed the data by three classes of medication: antipsychotics, antidepressants, and mood stabilizers.

Shortly before the study began, one SGA became available in a generic formulation. To ensure that the increase in generic prescribing was not being driven by use of generic risperidone alone, a sensitivity analysis was conducted on the overall data with generic risperidone excluded. No values were missing.

Back to Top | Article Outline

Secondary Analysis

Because of variation in the economic status of patients at different sites of service, with a higher number of indigent patients being served through the emergency department, we assessed whether the response to the generic initiative was influenced by location of care. The data were analyzed by each site of care: inpatient, outpatient, and emergency department.

In addition, because patients with bipolar disorder are often treated with both mood stabilizers and SGAs, analysis of this diagnostic group by class of medication alone is not fully representative. Therefore we conducted a subgroup analysis in which patients were grouped according to DSM-IVTR diagnoses: psychotic disorders, depressive disorders, and bipolar disorders.

Changes in residents' attitudes and comfort in using generic medications was our secondary outcome. To reduce the dimensionality of the questionnaire, we combined the scores from each subject into an average score that was treated as a summary measure. There were 11 missing values which were excluded from the analysis. As there were no substantial deviations from normality between the two groups with respect to the summary measures in the pre- and post-scores, the two sample t-test was used to test the hypothesis, with two-tailed P values and an accepted Type 1 error rate of 0.05.

The analysis strategy was specified a priori with the exception of the sensitivity analysis. Alpha was set at 0.05 for a two-tailed test. Since each hypothesis was substantially important and stated a priori, we controlled for type 1 error at the level of the comparison. If our statistically significant results were due to chance, we would have observed a random pattern. Instead, we observed the nonrandom pattern described in the following section. All statistical analyses were performed using SAS E-Guide Version 4.0 (Cary, NC).

Back to Top | Article Outline


We identified 13,303 documented clinical encounters, 5,856 during the control period and 7,447 during the study period. The percentage of generic drug use increased from 55.8% in the control period to 58.6% after the intervention began (Fig. 1). In this primary analysis of overall change, we found a statically significant increase in generic drug use (X2=10.3687, P=0.0013). This increase persisted in the sensitivity analysis when risperidone was excluded (X2=4.7741, P=0.0289). This was a far greater increase in generic prescriptions than could be accounted for solely by the number of generic samples that were used during the study period, which accounted for three prescriptions in the outpatient service and nine in the inpatient service.

Figure 1

Figure 1

When analyzed by class of medication (antipsychotics, antidepressants, and mood stabilizers), we found a significant increase in the use of generic antipsychotics from 39.5% to 47.7% (X2=36.1151, P≤0.0001) (Fig. 2) as well as a similar increase in use of generic antidepressants from 71.1% to 74.6% (X2=7.5243, P=0.0061). However, there was a decrease in use of generic mood stabilizers from 58.7% to 51.8% (X2=13.1668, P=0.0003) (Table 2).



Figure 2

Figure 2

We found a statistically significant increase in generic prescribing in both inpatient and outpatient sites but no significant change in the emergency department. There was also a consistent increase in generic prescribing for psychotic and depressive disorders (Table 3), although this only reached statistical significance in generic prescribing for psychotic disorders, which went from 60% in the control period to 63.6% in the intervention period.



For the qualitative analysis, the response rate to the questionnaire was 62.5% at the baseline assessment and 57.7% after 6 months. We did not find a statistically significant difference between the mean summary scores at baseline (3.57, SD=0.397, median=3.56) and after 6 months (3.63, SD=0.30, median=3.64). The mean difference between these scores was 0.063, 95% confidence interval (0.245, 0.12), standard error 0.912, P=0.495.

Back to Top | Article Outline


This observational study found a clinically significant increase in generic use after the educational program was implemented compared with the control period, suggesting the effectiveness of academic detailing in changing prescribing practices of graduate medical trainees in psychiatry. The SGA risperidone had recently become available in a generic formulation and we were concerned that this might be responsible for the increase in generic prescribing. Nevertheless, the difference in generic prescribing patterns continued to be significant even when risperidone was excluded from the analysis, demonstrating that the increase was not driven solely by use of generic risperidone. The program focused on three major drug classes: antipsychotics, antidepressants, and mood stabilizers. The use of generic antipsychotics and antidepressants increased significantly from the control to the study period, while the use of generic mood stabilizers did not. Consistent with our hypothesis, the largest increase in clinical use occurred in the class of generic antipsychotics.

While it appears that the use of mood stabilizers decreased, patients with bipolar disorder are also frequently prescribed an SGA. Therefore, an examination of the prescribing patterns for mood stabilizers alone does not fully represent the management of bipolar disorders. We examined prescribing patterns based on DSM-IV-TR diagnostic categories of depressive disorders, bipolar disorders, and psychotic disorders and found a consistent increase in use of generic medications in depressive and psychotic disorders, although this increase reached statistical significance only for psychotic disorders. We also found a significant change in both the inpatient and outpatient practices but no change in the emergency services. Our data show that emergency room patients were receiving higher rates of generic drugs at baseline, possibly because of socioeconomic characteristics. Further, emergency psychiatric care involves crisis management and stabilization. It is less likely that a physician would change the patient's regular psychotropic regimen in this brief clinical encounter, decreasing the possibility of detecting a change in prescribing practices.

Any educational program can be evaluated on four levels-reaction, learning, transfer, and results.12 Like most educational initiatives, our ultimate goal was at the fourth level, i.e., implementation of newly learned skills in clinical practice. Surprisingly, when assessed at the first level, there was no significant change in the subjects' reaction to the questionnaire at baseline and then 6 months later. This may reflect a ceiling effect since the average score was higher than 3.5 out of a possible 5 at baseline. It is also possible that the effect of any educational program is poorly measured by subjective measures. Such discrepancies have previously been found when studying the role of the pharmaceutical industry in education,13 where physicians are often unaware of the influence of industry detailing. However, prescribing patterns illustrate that pharmaceutical industries have a great deal of influence.14 We hypothesize that a similar phenomenon may have affected the results of this study. Documentation of clinical application may be a more robust indicator of the effectiveness of an educational program, and the use of such application as a primary outcome is a significant strength of our study. We recommend that future evaluations of educational programs also use implementation of new information as their primary measure.

This educational program used the principles of academic detailing. It was a service-oriented educa-tional outreach that combined the interactive communication approach of industry detailers with the evidence-based, noncommercial means of sharing information of academia. The underlying principles of the intervention included highlighting a specific group of doctors, establishing credibility, and having clear educational objectives. In a systematic review of 99 studies of medical education, outreach education was found to be more effective and consistent in creating positive changes in physician practices than audits with feedback or formal continuing medical education.15 Outreach educational endeavors most often resulted in a small but significant change in practice. In this study, we demonstrated that academic detailing appears to be an effective intervention for bringing psychiatric prescribing into concordance with current evidence.

The results of our study are consistent with those of other studies in mental health in which academic detailing has been effectively used in primary care settings to change prescribing practices.16 A similar method was recently used at a university-affiliated Veterans Affairs Medical Center, where lectures and handouts encouraged providers to use more cost-effective and scientific prescribing of antipsychotics. However, preliminary results from this study did not demonstrate a significant change in antipsychotic prescribing, leading to the conclusion that a more active approach was needed for a successful intervention.17 Educational packages have been shown to be more efficacious when combined with other active interventions.15 For example, in a large psychiatric hospital, physicians were provided with evidence supporting appropriate dosing of quetiapine. Following this, providers received personal feedback from the unit medical director when low doses of quetiapine were used.18 In our own program, we used both academic detailing as well as sampling of generic drugs in clinics in a two pronged approach, demonstrating the utility of a combined intervention.

An important limitation is the observational nature of our study and the absence of a real time control group. However, it was not feasible to randomize residents so that one group was deprived of information. In this situation, using a historic control was the next best option. Nevertheless, the consistent pattern of increasing generic drug use, and the increase in generic antipsychotic use in particular, supports the plausibility of a real association. Further studies are needed to replicate these results and validate this intervention in other populations. There was also an increase in the number of clinical encounters documented due to an increase in the number of patients being cared for. However, there was no change in the socioeconomic demographics in the clinical population and we do not believe this was a significant confounder. Another limitation is that it was not possible to do a prior validation of the questionnaire used to measure residents' perceptions because of the small number of residents. However, the questionnaire used a simple Likert scale and demonstrated a consistent pattern across subjects.

This combination of two active interventions, academic detailing and sampling, was an effective tool in bringing the clinical practice of psychiatry residents into concordance with the current best evidence. Physicians who are familiar with a wider range of medications, including older drugs, have more options in the care of patients with treatment-resistant illness so that they are better able to tailor care to individual patients while preserving patient choice.19 This is an important element of resident development and has broad implications for psychiatric providers as well as other specialties interested in evidence-based, cost-effective care. This innovative approach, with its large promise of cost savings, is especially pertinent in the current environment of healthcare expenditure.

Back to Top | Article Outline


1. Duggan M. Do new prescription drugs pay for themselves? The case of second-generation antipsychotics J Health Econ.. 2005;24:1–31
2. Rosenheck RA, Leslie DL, Doshi JA. Second-generation antipsychotics: Cost-effectiveness, policy options, and political decision making Psychiatr Serv.. 2008;59:515–520
3. Lieberman JA, Stroup TS, McEvoy JP, et al. Effectiveness of antipsychotic drugs in patients with chronic schizophrenia N Engl J Med.. 2005;353:1209–1223
4. Rosenheck RA, Leslie DL, Sindelar J, et al. Cost-effectiveness of second-generation antipsychotics and perphenazine in a randomized trial of treatment for chronic schizophrenia Am J Psychiatry.. 2006;163:2080–2089
5. Rosenheck R, Perlick D, Bingham S, et al. Effectiveness and cost of olanzapine and haloperidol in the treatment of schizophrenia: A randomized controlled trial JAMA.. 2003;290:2693–2702
6. Jones PB, Barnes TR, Davies L, et al. Randomized controlled trial of the effect on quality of life of second- versus first-generation antipsychotic drugs in schizophrenia: Cost Utility of the Latest Antipsychotic Drugs in Schizophrenia Study (CUtLASS 1) Arch Gen Psychiatry.. 2006;63:1079–1087
7. Leucht S, Corves C, Arbter D, et al. Second-generation versus first-generation antipsychotic drugs for schizophrenia: A meta-analysis Lancet.. 2009;373:31–41
8. Covell NH, Finnerty MT, Essock SM. Implications of CATIE for mental health services researchers Psychiatr Serv.. 2008;59:526–529
9. Forsner T, Hansson J, Brommels M, et al. Implementing clinical guidelines in psychiatry: A qualitative study of perceived facilitators and barriers BMC Psychiatry.. 2010;10:8
10. Shrank WH, Choudhry NK, Solomon DH, et al. Rationale and design of the Study Assessing the Effect of Cardiovascular Medications Provided as Low-cost, Evidence-based Generic Samples (SAMPLES) trial Am Heart J.. 2009;157:613–619
11. Michie S, Fixsen D, Grimshaw JM, et al. Specifying and reporting complex behaviour change interventions: The need for a scientific method Implement Sci.. 2009;4:40
12. Kirkpatrick D Evaluating training programs: The four levels. 19982nd edition San Francisco, CA Berrett-Koehler Publishers
13. Steinman MA, Schillinger D. Drug detailing in academic medical centers: Regulating for the right reasons, with the right evidence, at the right time Am J Bioeth.. 2010;10:21–23
14. Avorn J, Chen M, Hartley R. Scientific versus commercial sources of influence on the prescribing behavior of physicians Am J Med.. 1982;73:4–8
15. Davis DA, Thomson MA, Oxman AD, et al. Changing physician performance. A systematic review of the effect of continuing medical education strategies JAMA.. 1995;274:700–705
16. Soumerai SB. Principles and uses of academic detailing to improve the management of psychiatric disorders Int J Psychiatry Med.. 1998;28:81–96
17. Rosenheck RA, Sernyak MJ. Developing a policy for second-generation antipsychotic drugs Health Aff (Millwood).. 2009;28:w782–w793
18. Stowell KR, Ghinassi FA, Fabian TJ, et al. Best practices: An intervention to promote evidence-based prescribing at a large psychiatric hospital Psychiatr Serv.. 2009;60:294–296
19. Jefferson JW. Old versus new medications: How much should be taught? Acad Psychiatry.. 2005;29:162–166

prescribing patterns; psychotropic medications; antipsychotics; generic medications; psychiatric residents; residency training

Copyright © 2011 Wolters Kluwer Health, Inc. All rights reserved.