Secondary Logo

Journal Logo

Chronic Care

Teaching Systems-Based Practice to Primary Care Physicians to Foster Routine Implementation of Evidence-Based Depression Care

Sherman, Scott E. MD, MPH; Fotiades, John MD, MPH; Rubenstein, Lisa V. MD, MSPH; Gilman, Stuart C. MD, MPH; Vivell, Susan PhD, MBA; Chaney, Edmund PhD; Yano, Elizabeth M. PhD, MSPH; Felker, Bradford MD

Author Information
doi: 10.1097/ACM.0b013e31802d9165

Abstract

Depression is a common, widespread chronic illness associated with substantial personal, social, and financial costs.1–3 Depressive disorders are very prevalent in primary care settings, with rates two to three times those of community samples.4,5 The World Health Organization estimates that depression will be the second leading cause of morbidity worldwide by the year 2020.6 Despite its high prevalence and morbidity burden, depression remains underrecognized and undertreated in primary care.5,7,8

Although many approaches to improve the management of depression in primary care have been tried, current evidence favors the use of a collaborative care model.9–11 This approach requires close collaboration between primary care and mental health providers, typically with an interdisciplinary team supporting the primary care delivery of evidence-based depression care. To facilitate this collaboration, many features from the chronic care model may be included, such as depression care managers, top management support, a registry of patients, and computer information systems.

For a collaborative care approach to be successful for depression, a wide range of educational needs must be met for patients, health care providers, and staff. This article focuses on the needs of health care providers in implementing a collaborative care approach. Primary care providers often have little experience with either managing depression or using a collaborative care approach. In contrast, mental health care providers have extensive experience with the management of depression, but they have little experience with the collaborative care model or a population-based perspective to patient management. These gaps in knowledge and experience point to educational needs that must be addressed to improve the system of care for depression.

Our conceptual model for education in this setting is shown in Figure 1. First, education is separated into content education and process education, with content education focusing on ensuring that the learners know what they need to know about depression (e.g., diagnostic criteria, prescribing guidelines, when to refer patients to specialized services). Process education includes learning about the new care processes (e.g., how can this content be applied in practice?). Second, tasks are separated into initial education and ongoing/reinforcing education. Initial education typically has a balance of content and process education, whereas ongoing education typically focuses predominantly on the process. All of these aspects (particularly ongoing education) are generally necessary to affect provider behavior.12–14

Figure 1
Figure 1:
Conceptual framework for educating and training providers on depression management and collaborative care.

A complementary approach to the educational needs of providers is the competency framework of the Accreditation Council for Graduate Medical Education (ACGME).15 The content education addresses the ACGME core competency for medical knowledge, whereas the process education addresses the competency for systems-based practice (as well as for practice-based learning and improvement and interpersonal and communication skills). Much of medical education has historically been devoted to increasing medical knowledge.12,16 In contrast, there is very little information available about how to teach (or improve providers’ level of competency with) systems-based practice.

As part of a large, multisite study of improving primary care management of depression through a collaborative care approach, we developed and implemented a provider education program, which represents a model for teaching systems-based practice. This article describes the program and how the same principles might be used in other settings and for other conditions.

The TIDES Program

The Translating Initiatives for Depression into Effective Solutions (TIDES) study, funded by the Veterans Health Administration (VA), was conducted from 2002 to 2004 to evaluate the effectiveness of adopting a collaborative care model at seven different VA sites; three other control VA sites did not adopt the model. The VA treats an aging, primarily male population that is below average in socioeconomic resources and above average in comorbidity and functional disability.17 The seven medical centers participating in the study, which are located within the midwestern and southern regions of the United States, were assured of the confidentiality of their participation. Sites were selected based on the following criteria: (1) moderate or greater priority for depression care improvement on the part of the local leadership, (2) more than 3,000 patients with at least three visits each in the preceding year, and (3) moderate or lower organizational complexity on the VA Complexity Score, which is based on characteristics such as size, scope of services, and level of academic and research mission. This project was approved by the institutional review board at each participating site.

The collaborative care approach called for primary care providers to screen for depression and to refer patients with a positive screening test to the depression care manager (DCM) for further assessment. The DCMs, all of them RNs in primary care who were specially trained to provide support to patients with depression, were responsible for implementing and coordinating the primary care provider’s treatment plan. Patients who were diagnosed with depression were followed by the DCM for six months, using the nine-item Patient Health Questionnaire (PHQ-9) both for diagnosis and for monitoring progress.18,19 For patients managed within primary care, the DCM conducted regular 10- to 15-minute phone calls at 2, 4, 8, and 16 weeks and at six months, to assess symptoms, monitor treatment adherence, and reinforce patient education about self-management skills. For patients referred to mental health specialists, the DCM called the patient at 4 and 12 weeks and at six months.

TIDES Provider Education Program

Before implementation of TIDES, two expert medical education leaders (JF, SES) were selected to facilitate the provider education component of the study. Our process for identifying and meeting the educational needs of providers involved the following five steps (described in detail below):

  1. Determine the educational needs of providers.
  2. Develop educational materials for providers.
  3. Develop an educational intervention to use at each site.
  4. Implement the intervention.
  5. Monitor the intervention’s effectiveness.

Step 1: Determining the educational needs of providers

Determination of educational needs consisted of three phases: demonstrated needs (based on organizational data), presumed needs (what the study leaders expected based on our experience), and expressed needs (as articulated by the providers). First, we estimated the providers’ demonstrated needs. Global data from each site were collected through the VA’s electronic health record system. Facility-level clinical performance data were identified for such issues as incidence of depression screening in primary care, referral rate to specialty care, and likelihood of depression being managed within primary care. Because the electronic health record is implemented throughout all VA health care facilities, this could be done systematically and comparably for most sites. Unfortunately, data for these measures for individual clinics within the VA facilities were not available before or during the study period.

Presumed needs, a component of the TIDES study from the outset, were also addressed using expert opinion. On the basis of the literature regarding changing provider behavior,13,14 we hypothesized that many of the primary care providers would not have adequate knowledge to manage depression within primary care and that few providers would have any experience with using a collaborative care approach. However, we solicited local leaders (top administrators in medicine and psychiatry) at each site to assess what they presumed were their colleagues’ educational needs. The senior organizational leaders were asked what the best methods would be for disseminating educational interventions to the providers in their clinics once these educational tools were developed.

We then assessed, validated, and augmented these demonstrated and presumed needs through a process of collecting practitioners’ expressed needs at study sites. The two education leaders (JF, SES), along with the study investigators (LVR, EC), met with primary care and mental health providers from all the participating sites, as well as with senior organizational leaders. During this day-long meeting, a needs assessment was conducted with primary care providers, mental health providers, and senior leadership. Primary care providers were asked to describe the type of information they would need to assist them in managing patients with depression. Mental health providers were asked about the type of information they would require to improve collaboration with primary care. We then followed up on this initial meeting with visits to each of the individual sites, where one of the two provider education leaders met with all of the clinical staff to assess their local educational needs.

Step 2: Developing educational materials for providers

Because few sites had both the time and resources to create their own depression educational materials, this task was carried out by the TIDES provider education leaders in conjunction with members of the TIDES Faculty Committee (which consisted of TIDES investigators and senior administrators from participating sites). One of the purposes of TIDES was to create a standard set of educational materials that was easily adaptable to local settings. Using the presumed, demonstrated, and expressed needs collected and distilled from Step 1, we created a series of lectures and associated notes, a depression pocket card, a sample depression education plan, materials for educational outreach,20 and an educational section of the TIDES Web site. The Web site included information for patients, clinicians, and managers, including all of the educational materials mentioned above, links to other resources, and a broad range of other supporting material.* We did not expect that each site would use all of these materials, but rather that they would choose those that best fit their local needs.

We developed one lecture summarizing the project and three lectures summarizing the recognition and management of depression. A lecture on suicide was subsequently added as a fifth lecture, in response to requests from providers for more information on this topic. We created these lectures as slide presentations with accompanying speakers’ notes, designed to be given over a one-hour time period. The one-hour time period seemed ideal because it was determined that most of the sites had at least bimonthly conferences lasting from one to three hours.

Step 3: Developing locally tailored educational interventions

Each site was asked to review the educational materials; develop a plan for how they would educate their providers, preferably using as many of the educational materials as possible; and then implement and monitor the plan. The initial lecture, an overview of the TIDES study and the collaborative care model, was always given either by one of the study’s principal investigators (LVR, EC) or by one of the study’s two provider education experts (JF, SES) and was presented to the entire clinic, which included both primary care and mental health providers. Additional lectures were presented by the local educational leaders and were given only to primary care providers. The lectures are accessible on the TIDES Web site and are available for use.

Step 4: Local implementation of the educational program

Each site identified an individual to coordinate educational activities locally, as well as an expert opinion leader in primary care and another opinion leader in mental health. At each site, the coordinator was a lead primary care clinician who agreed to serve as the principal investigator for that site and who served as liaison with the study team. Each of the sites was advised that these opinion leaders20–22 should be individuals who were well respected by their peers, had an interest in provider education, and would be willing to disseminate the educational materials we developed.

Once each site had an individual in charge of education, we sent all of the educational products, a sample education plan (Figure 2), and a five-page handout to guide in the development of site-specific education plans. Each site was asked to develop an educational plan that outlined the scope, intended audience, goals, and content of its educational efforts. Both study education expert leaders were available to each of the local leaders for consultation throughout the implementation period.

Figure 2
Figure 2:
Sample education plan for teaching a collaborative care model of depression management to primary care and mental health practitioners.

Sites were allowed to decide how to best implement the initial and ongoing educational activities based on their local needs. All sites were strongly encouraged to use educational outreach (also called academic detailing) visits,20,23 and the five-page handout included two pages on educational outreach. Specifically, it included a script for conducting the outreach visit, as well as the following six recommended points to be covered at each outreach visit:

  1. Description of TIDES and its rationale.
  2. Review pocket card.
  3. Review the role of the DCM.
  4. Mention warning signs that might prompt different management.
  5. Demonstrate the TIDES Web site.
  6. Answer any questions.

Step 5: Monitoring the educational program

The fifth and final step (monitoring the intervention’s effect) was left up to each individual site. Though ongoing reporting can help ensure better performance, the sites were not required to report to us regularly on their progress because of concerns of overwhelming them with details they perceived as “nonclinical.”

To evaluate the extent to which each site implemented its education plan, the lead clinician or the DCM at each site was asked to fill out a one-page survey, which included the following questions:

  • How many lectures/talks on depression were given during the study period?
  • How many people attended the talks?
  • Was academic detailing/educational outreach conducted with providers? (Yes/No)
  • Were the pocket cards used? (Yes/No)
  • Was the TIDES Web site used? (Yes/No)

Responses were received from all seven intervention sites.

Effects of the Provider Education Intervention

Determining the educational needs of providers

As we expected, primary care providers did, in fact, feel that they needed information on managing depression. At the initial education session, the overwhelming majority of primary care providers brought up two issues: (1) the fear that treating depression in a primary care setting would be an overwhelming and time-consuming task, and (2) how to get patients to accept the diagnosis of depression and/or any recommended treatment. Neither primary care nor mental health providers reported any experience providing collaborative care on a routine basis, although representatives from several sites felt there was a good working relationship between mental health and primary care practitioners (usually because of physical proximity).

Local implementation of the educational program

The characteristics of the individual sites are shown in Table 1, along with whether they reported using the different education components we developed. None of the sites chose to write up their education plan or to individualize their plan differently than the model that was provided. Nearly all of the sites conducted lectures/talks on the management of depression, and nearly all reported using the pocket cards. Few sites reported using the Web site, and the use of academic detailing was variable. Two sites opted not to conduct educational outreach because providers did not have time for the five-minute detailing visit. A few sites used audit and feedback of quarterly site-level performance data, and none of the sites used provider-level feedback on performance.

Table 1
Table 1:
Extent of Implementation of Provider Education Components at Seven Translating Initiatives for Depression into Effective Solutions (TIDES) Intervention Sites, 2002–2004

Although few of the sites reported using the Web site systematically, it did receive a fair amount of traffic. Site-use data are available for three full quarters of 2004, and during that time, the Web site was receiving 2,600 to 2,800 visits per quarter (28–30 visits per day). These visits represented approximately 1,100 to 1,300 unique visitors per quarter. The depression pocket card was downloaded over 1,000 times during the nine-month period. Because of confidentiality considerations, we did not track who used the Web site.

Discussion

Teaching systems-based practice to full-time providers is quite challenging. To be effective, implementing evidence-based practice is going to require identifying ways to educate busy providers in the context of routine care. In implementing the TIDES program, we found that providers’ educational needs varied in some expected and some unexpected ways. In particular, primary care providers wanted more information about developing a treatment plan and negotiating with the patient about the plan, all in a feasible amount of time. Mental health providers wanted more information about collaborating with primary care providers.

Our approach to increasing providers’ competency at a systems-based practice (the collaborative care model for depression) was generally successful. There was moderate use of our educational tools, which we discuss further below. More important, patient care did change, with all sites referring patients to the DCMs for evaluation and follow-up. Although the VA data available do not give us pre- and poststudy period data on depression screening and referral at the implementation sites, the TIDES program did become part of routine care at the implementation sites. This suggests strongly that our educational efforts were largely successful, both in instructing providers about the new collaborative process of care, with which they reported no prior experience, and in raising their comfort level with treating depressed patients in primary care.

During the 18-month intervention period, nearly all primary care providers referred patients (n = 458) with depressive symptoms to the program, showing that it became routine care at all intervention sites. The DCMs completed assessments of 398 (87%) of those patients; of these, 356 patients (89%) received ongoing collaborative depression care support from the DCM. The vast majority (80%) of care management patients were followed for depression solely within primary care (referral to a mental health provider was not necessary); most of these patients (77%) took their antidepressant medications, and most had resolution of their symptoms. Thus, although a lack of existing data prevents a comparison with the control sites, we believe that our educational plan did succeed in its twin goals of content and process education of site providers because providers were not only aware of the program, they were also comfortable enough to integrate the new program into their daily clinical practice. Given these findings, it is worth examining our intervention to try to determine how to make the next project more successful.

We developed several specific tasks to facilitate educating providers at each site. First, we developed and provided education material, in the form of the first lecture, which was given by TIDES provider education leaders or study investigators. Second, we developed a wide range of VA-adapted educational materials for sites to use as is or to adapt locally. Third, we gave them easy instructions on how to implement these materials, through educational outreach and other approaches.

As expected, no site used all of the materials. In general, sites were more likely to implement passive education approaches (e.g., lecture, distribution of pocket cards) than active approaches (e.g., audit and feedback, educational outreach). The relatively low use of active strategies is unfortunate because these educational strategies are often shown to be more effective than passive strategies.12,20 Our study reduced many barriers to using these active strategies (particularly by conducting the needs assessment, educational content creation, and training), yet such strategies were still not widely implemented. Even the sites that did use educational outreach did not do so consistently and thoroughly, despite frequent reminders and having materials already developed. Future research is needed to identify what additional barriers to using active educational strategies remain and how they can be mitigated. This is a vital issue, as continuing medical education will increasingly focus on competency improvement, and active education strategies, especially audit and feedback, are important techniques.

One area that warrants further discussion is that of the sites adapting and writing up their education plan, an exercise that has face value but that none of them chose to do. This echoes our experience in an earlier 18-site study of evidence-based quality improvement for smoking cessation,24,25 where most of the nine intervention sites wrote up a plan but none were willing to adapt it based on the experts’ feedback. Why is it that the seven sites in TIDES did not write up their plans? If the sites in this large, funded study would not do it, how can we expect future sites to comply as we roll out this intervention into routine care? We think this suggests it is not a realistic expectation. We ultimately may have been able to get the seven sites to create a written plan, but it likely would have required frequent reminders and would need to have been seen as a mandatory part of project participation. Because each site’s resources are limited, including this component would have come at the expense of something else. It would be worth studying whether having a written plan does, in fact, increase adherence to the plan and achievement of target outcomes. Until such evidence is available, we would suggest not requiring a written plan, but rather incorporating the essential activities into the routine process of care, as described further below.

The use of the study Web site also merits further discussion. Although those using the site and downloading files were not identified, it did get a considerable amount of use. It seems likely that the users were predominantly from the seven study sites, because we did not publicize the Web site anywhere else. On the basis of our experience, we feel that the Web site is best used for sharing materials rather than as a specific means of educating staff. There is no approach better than a Web site for ensuring that all staff have access to the most up-to-date version of handouts, training materials, and other resources.

Limitations

There were three main limitations to implementing the TIDES education program. The first is that this intervention took place at seven moderate- to small-sized VA sites spread across the midwestern and southern United States. Whether the same principles can be generalized to other settings (such as small groups or large academic centers) is open to question. With trainees rotating in and out frequently, more complex sites would benefit even more from approaches to build education into processes that were already going to happen. Our principles and our focus on generalist–specialist collaboration are sufficiently general to apply to a wide range of health care systems and health conditions. The difficult part is getting the system to change the approach to care (such as by adopting a collaborative care approach). Our research suggests that integrating provider education into existing clinical processes will facilitate implementing changes in the structure of care delivery.

A second limitation is that this is one study focused on one condition, depression. We have tried in this article to be sufficiently general to apply our experience to other conditions, but some sections are inherently related to depression. We do not feel, however, that this is a major limitation, because it is preferable to develop a program such as ours under more constrained conditions before generalizing and disseminating it. This approach should be relevant to improving the management of any chronic disease, particularly those that require collaboration between generalists and specialists.

A third limitation is that all seven sites were chosen in part because their leadership had already committed to improving depression screening and referral. This means that a top-down willingness to implement new procedures to meet these goals was already in place when the TIDES project began. Thus, we may have seen more success in referral rates and use of educational materials at these sites than we would have at other sites. It is possible that implementing a similar depression care project and achieving buy-in at sites with less of a mandate for improving the quality of care would be more difficult.

Lessons learned

The TIDES program and its implementation provides a model for how to translate research findings into practice. This goal of getting organizations to adopt processes shown to be both efficacious and effective is central to an institution’s efforts to improve its quality of care. Any such initiative, whether it is for depression, smoking cessation, or treating some other condition, will also need an educational component. Using our framework, there is often a need for content education to improve knowledge of the conditions involved. In addition, there is always a need for process education to inform providers and other staff about the new approach to care. In our study, providers used the collaborative care approach, which was effective, suggesting that our own approach did result in an improvement in systems-based practice. The most important question to us then becomes: how can we make our intervention more effective?

Some of the barriers mentioned by sites such as time pressure and lack of comfort adopting new approaches are real and must be dealt with as any large program moves to widescale implementation. Looking at the issue of time constraints, how do we get more time set aside for training? It is tempting to fall back on an autocratic solution. Many health care systems mandate that providers participate in specific educational programs. For example, the VA mandates that all employees receive 40 hours of education annually, with roughly half designated for specified topics. Similarly, all physicians in California must undergo 12 hours of medical education on “pain management” and “the appropriate care and treatment of the terminally ill,” a requirement of Assembly Bill No. 487.26 However, we do not think mandated education is the best approach. Compulsory solutions tend to be disliked by providers, and the number of things one can mandate is limited.

Lack of comfort is another issue that must be addressed proactively to give a similar program the best chance for success. By this, we mean the discomfort felt by primary care providers and mental health specialists when new procedures and programs are introduced into the clinical setting. Providers, unfamiliar and inexperienced with a new system and sometimes unsure that they have the necessary content knowledge to handle different patient responsibilities, can be resistant to participating in quality-improvement changes such as adopting collaborative care programs. That is precisely where education models focusing on increasing provider comfort, and therefore provider participation, become crucial to the success of a new clinical care model.

Overall, we feel that the best way to improve our intervention is to build education into the system, rather than building a system for education. More attention needs to be given to adapting educational approaches to meet the needs of providers, rather than requiring providers to adapt to what educators are offering. In retrospect, there are two simple, straightforward techniques that we could have applied. First, education could be embedded within the computer system. All providers were already using the VA’s electronic medical record system,27 and referrals to the DCM were sent electronically. Embedding relevant information in this referral form can be an effective way to modify provider behavior,28 and we have incorporated this approach in our current project.

The second change is in how the educational outreach was delivered. Some sites were resistant to having one person walk around “detailing” all of the providers. However, few sites would have had any qualms about providers receiving five minutes of feedback from the DCM. The same educational outreach principles could easily be used, with the DCM having a checklist of points to review with each referring provider. This could be given as structured follow-up on the first few referrals, with intermittent repeats every three to six months for reinforcement. This patient-specific education would help to get around reluctance, just as clinical vignettes often heighten our interest in an article or talk.

In List 1 we have summarized what we see as the key learning points about education: planning, organization, efficiency, and engagement. In his Provincial Letters, the famous French mathematician Blaise Pascal wrote, “I have made this letter longer than usual because I lack the time to make it shorter.” Education is too important to leave as a “long letter” that providers do not have the time to read.

List 1 Lessons learned about identifying and meeting the educational needs of providers
List 1 Lessons learned about identifying and meeting the educational needs of providers:
List 1 Lessons learned about identifying and meeting the educational needs of providers

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs. The research reported here was supported by Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service Grant MNT 01–027.

Acknowledgments

This paper is dedicated to the memory of Dr. John Fotiades, who tragically died in a fire at age 39 on January 1, 2005.

The authors would particularly like to thank Drs. JoAnn Kirchner, John Fortney, and Jennifer Magnabosco and Ms. Carol Simons for their very helpful comments on an earlier draft of this manuscript and Ms. Carole Oken for her support of the TIDES project. In addition, the authors gratefully acknowledge the editorial support of Laura York, MA.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs. The research reported here was supported by Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service Grant MNT 01-027.

References

1 Wells KB, Stewart A, Hays RD, et al. The functioning and well-being of depressed patients. Results of the Medical Outcomes Study. JAMA. 1989;262:914–919.
2 Broadhead WE, Blazer DG, George LK, et al. Depression, disability days, and days lost from work in a prospective epidemiologic survey. JAMA. 1990;264:2524–2528.
3 Conti DJ, Burton WN. The economic impact of depression. J Occup Med. 1994;36:983–988.
4 Katon W, Schulberg HC. Epidemiology of depression in primary care. Gen Hosp Psychiatry. 1992;14:237–247.
5 Simon GE, Von Korff M. Recognition, management, and outcomes of depression in primary care. Arch Fam Med. 1995;4:99–105.
6 World Health Organization. The World Health Report: 2001. Mental Health: New Understanding, New Hope. Geneva, Switzerland: World Health Organization; 2001.
7 Goldman LS, Nielsen NH, Champion HC. Awareness, diagnosis, and treatment of depression. J Gen Intern Med. 1999;14:569–580.
8 Simon GE, Goldberg D, Tiemens BG, Ustun TB. Outcomes of recognized and unrecognized depression in an international primary care study. Gen Hosp Psychiatry. 1999;21:97–105.
9 Katon W, Von Korff M, Lin E, et al. Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial. Arch Gen Psychiatry. 1999;56:1109–1115.
10 Von Korff M, Gruman J, Schaefer J, Curry SJ, Wagner EH. Collaborative management of chronic illness. Ann Intern Med. 1997;127:1097–1102.
11 Hedrick SC, Chaney EF, Felker B, et al. Effectiveness of collaborative care depression treatment in Veterans’ Affairs primary care. J Gen Intern Med. 2003;18:9–16.
12 Thomson O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2001;2:CD003030.
13 Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–1230.
14 Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39(2 suppl):S112–S145.
15 Accreditation Council for Graduate Medical Education. ACGME Outcome Project. General Competencies Minimum Program Requirements Language. Available at: (http://www.acgme.org/outcome/comp/compMin.asp). Accessed October 16, 2006.
16 Davis D, Barnes BE, Fox RD, eds. Continuing Professional Development of Physicians: From Research to Practice. Chicago, Ill: American Medical Association Press; 2003.
17 Yu W, Ravelo A, Wagner TH, et al. Prevalence and costs of chronic conditions in the VA health care system. Med Care Res Rev. 2003; 60(3 suppl):S146–S167.
18 Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–613.
19 Lowe B, Unutzer J, Callahan CM, Perkins AJ, Kroenke K. Monitoring depression treatment outcomes with the patient health questionnaire-9. Med Care. 2004;42:1194–1201.
20 Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, Bero LA. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2000;2:CD000172.
21 Lomas J, Enkin M, Anderson GM, Hannah WJ, Vayda E, Singer J. Opinion leaders vs audit and feedback to implement practice guidelines. Delivery after previous cesarean section. JAMA. 1991;265:2202–2207.
22 Pereles L, Lockyer J, Ryan D, Davis D, Spivak B, Robinson B. The use of the opinion leader in continuing medical education. Med Teach. 2003;25:438–441.
23 Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:iii–iv, 1–72.
24 Sherman SE, Yano EM. Implementing Smoking Cessation Guidelines: A Randomized Trial of Evidence-Based Quality Improvement: Final Report. Washington, DC: Department of Veterans Affairs, VA Health Services Research & Development Service; 2003. Project CPG-97-002.
25 Sherman SE, Lanto AB, Nield M, Yano EM. Smoking cessation care received by veterans with chronic obstructive pulmonary disease. J Rehab Res Devel. 2003;40(2 suppl):1–12.
26 Medical Board of California. New Laws Relating to Medical Education. Available at: (http://www.medbd.ca.gov/cme_laws.htm). Accessed October 16, 2006.
27 Rundle RL. Oft-derided veterans health agency puts data online, saving time, lives. Wall Street Journal. December 10, 2001:A1.
28 Guterman JJ, Chernof BA, Mares B, Gross-Schulman SG, Gan PG, Thomas D. Modifying provider behavior: a low-tech approach to pharmaceutical ordering. J Gen Intern Med. 2002;17:792–796.

* For more information, go to (http://www1.va.gov/tides_waves) or contact the authors.
Cited Here

© 2007 Association of American Medical Colleges