TRANSLATING COMPARATIVE EFFECTIVENESS RESEARCH INTO PRACTICE
Comparative effectiveness research (CER) is defined by the Congressional Budget Office as “… a rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy. The analysis may focus only on the relative medical benefits and risks of each option or it may also weigh both the costs and benefits of those options.”1 Much of the discussion related to CER has been focused around generating knowledge about comparative treatments, and hence what represents the ideal, evidence-based care for patients. However, a key component for realizing the value of CER is the translation of this new knowledge for ideal care into routine practice. Surprisingly, very little attention has been paid to this key component. History tells us that generating new knowledge takes a long time to translate into practice. This problem has been described eloquently in a number of recent publications.2–5 A second translation problem that has been observed frequently with CER, but rarely discussed is that of a clear “winner.” Many of the EPC reports have not been able to demonstrate superior efficacy of one agent over another in terms of reaching the goals of treatment; however, they have identified differences in the adverse clinical outcomes that may be experienced across the agents. These types of results may lead to further confusion in discussing the optimal treatment regimens for providers and patients.
One of the stated goals of conducting CER is to reduce health care spending through more efficient treatment decisions. Much of the translation of CER occurs through formulary coverage decisions, and these decisions guide the patient and clinician discussions related to treatment selection. This model of translation does not allow for the patient to exercise his/her values and preferences since all the potential options may not be presented to the patient in a meaningful way. In addition, this may also limit the extent to which the clinician would present the options to the patient if the decision may be based on coverage. A lack of involving patients in the decision-making process presents a number of problems. First, it does not lead to optimal knowledge transfer of evidence to the patient for all the treatment options and also limits the conversation between the patient and clinician. This, in turn, may lead to patient nonadherence because of knowledge gaps. Nonadherence may be the means by which a patient may express their preferences when their values, goals, and preferences are not incorporated into treatment decisions during the encounter. Finally, translation done solely by cost and coverage choices may lead to lower clinician satisfaction since it limits the extent to which professionalism and professional experience can be exercised.
To address these translational challenges, Agency for Healthcare Research and Quality has funded the Eisenberg Center, which was established to translate the findings of CER into formats that are understandable to consumers, clinicians, and policymakers. However, it is unclear whether these tools aid in translating the findings into real-world practice settings. Many other potential mechanisms exist for translating CER into practice and may include clinical guidelines, continuing medical education, patient decision aids, and academic detailing. Studies of specific strategies and tools for translating this evidence into practice are required.
THE GAP IN TRANSLATING EVIDENCE ABOUT THE COMPARATIVE EFFECTIVENESS OF DIABETES MEDICATIONS INTO PRIMARY CARE PRACTICE
There have been significant advances in the agents available for the management of glycemic control for diabetes patients; however, it is difficult to compare these agents because there are so many individual drugs and drug classes and so little synthesis of what is known.6 The sheer volume and contradictory nature of the existing literature on the topic make it difficult for primary care clinicians to determine the optimal management of individual patients. In addition, glycemic control is only one of many problems people with diabetes face, all of which need to be addressed in short clinical encounters. Few efficacy studies allow for CER translation in the context of multiple morbidities.7
People with diabetes continue to have poorly controlled blood glucose.8–10 Recent evidence suggests that lack of intensification of diabetes treatment in a timely manner may be an explanatory factor.11 The decision to intensify treatment may be affected by many factors, including likelihood of patient adherence, competing demands on the patients’ time, or provider knowledge and attitudes. One of the key challenges clinicians face is to individualize the optimal set of medications for glycemic control consistent with each patient's goals, values, and preferences. Since few of the medications have been compared against each other in terms of patient-important outcomes in randomized trials, it is difficult to determine how to best manage a patient whose blood sugars are not adequately controlled with one or more medications.
To adequately inform clinicians and policymakers about alternative oral diabetes medications, Agency for Healthcare Research and Quality commissioned a comparative effectiveness of all oral diabetes medication classes.12 The striking finding of this systematic review was that while there were only minor differences in glycemic control achieved with these agents in the trials summarized, there were significant differences in adverse effects of the alternative medications. Overall, thiazolidinediones, sulfonylureas, meglitinides (repaglinide), and biguanides (metformin) produced similar reductions in hemoglobin A1c levels. Meglitinides (nateglinide) and alpha-glucosidase inhibitors had a slightly weaker effect on hemoglobin A1c lowering (A1c). An observational study comparing the effects of these medications in the real-world setting found similar results13 where biguanides, sulfonylureas, thiazolidinediones, and insulin had a similar effects, on average, in reducing hemoglobin A1c. However, most patients failed to achieve adequate glycemic control due to a combination of delays in intensification of treatment and poor adherence to prescribed medications. This suggests that involving the patients in shared decision-making to select medications that match their preferences may help in achieving optimal glycemic control by improving communication about intensifying treatment and increasing medication adherence.
DECISION AIDS TO ENHANCE PATIENT INVOLVEMENT IN TRANSLATING INFORMATION ABOUT COMPARATIVE EFFECTIVENESS INTO PRACTICE
Decision aids, at a minimum, offer a structured and accessible presentation of all the relevant options and of their relative advantages and disadvantages, as identified in the CER, to help patients and clinicians make the choice that is most consistent with patient values and preferences and their context.14 Allowing patients to observe this balance of harms and benefits can improve decisional quality, adherence, and their control of chronic conditions15 and can stimulate use of the beneficial options overcoming clinical inertia while promoting adherence. For “preference sensitive” health services, in which the ratios of benefit to harm are either uncertain or dependent on patient values (ie, diabetes medications), participation of patients improves quality of decisions and may prevents overuse in the subset of informed patients for whom the undesirable consequences outweigh the desirable ones.16 Decision aids could also reduce unwarranted practice variation. In a recent commentary on comparative effectiveness, Dr. Carolyn Clancy states that, “variation attributable solely to informed decision making should be celebrated in a smart system, and it offers opportunities to evaluate the outcomes of different decisions.”17 In the following section, we will describe the process for developing the decision aids for translation of CER, preliminary testing of the tool in clinical practice, and an overview of a larger trial currently getting underway to assess the role of decision aids for translating CER.
Development of the Decision Aid
We approached the translation of CER into practice for diabetes medications by developing a patient decision aid. Patient decisions aids are becoming increasingly common for a variety of screening, and treatment decisions and can be developed using various mediums including DVDs, web-based modules, as an addition to the electronic medication record, and simply on paper. The medium to be used may also depend on the mode of administration for the decision aid. Decision aids may be administered to patients at home, in the clinic (self or nurse administered) before the encounter with the clinician, with the clinician during the encounter, or using a combination of these approaches. Relatively little research currently exists on the medium for developing decision aids and for mode of administration. The most common medium to date has been DVDs administered before the visit with the clinician. The goal of our decision aids was to enhance the discussion relating to diabetes medications during the clinical encounter. Thus, we focused on using a medium for developing the decision aid that would be amenable to use during the clinical encounter.
The development of the decision aid was conducted using a multidisciplinary team including patients, providers, and designers. In addition, it was an iterative process that has been described in detail in previous articles.18,19 The process of developing the decision aid focuses on combining 2 main aspects, synthesis of research evidence and patient values and preferences, such that the tool will enhance the communication in the clinical encounter related to medication discussions. The first step of this process was to synthesize the research evidence, which in this case was done by the review conducted by Bolen et al for the EPC program.12 Next, a nonhealthcare member of the team observed a series of patient-physician encounters when discussions of diabetes medications occurred. On the basis of these initial observations, a prototype of the decision aid was developed. This prototype was iteratively tested by physicians in real-world patient encounters and was also assessed by the diabetes patient advisory group. On the basis of multiple iterations of this process, a final set of diabetes decision cards (Fig. 1) were developed. This final iteration of the cards focused on issues that were deemed important to the patient as opposed to providing exhaustive and detailed information about each of the medications/drug class.
Preliminary Testing of the Decision Aid
We conducted a cluster randomized pilot trial to evaluate the effectiveness of decision aid in clinical practice. The trial is fully described in a prior article.20 Briefly, the trial took place in 11 primary care and family practice sites within the Mayo Clinic Health System and Olmsted Medical Center. Eligible providers included physicians, physician assistants, and nurse practitioners. Eligible patients were adults with type 2 diabetes for at least 1 year and who had a scheduled appointment with a provider. In addition, patients had to have a hemoglobin A1c between 7% and 9.5%, use less than 3 medications for managing glycemic control, and not to use insulin. Providers randomized to the intervention arm received a brief demonstration from the study coordinator on how to use the decision aid prior to meeting the first enrolled patient.
Patient outcomes were measured through both surveys, medical record review, and through collection of pharmacy records. The measures collected as part of this study are described in Mullan et al20; however, in this article we focus on 3 main outcomes: (1) knowledge transfer, (2) patient acceptability, and (3) medication choice and adherence. The self-administered survey after the visit included 15-knowledge questions to assess patient understanding of the comparative effectiveness of diabetes medications. Ten of these questions were addressed in the decision aid whereas 5 were not. The pertinent questions assessed the patient's knowledge of the comparative effectiveness of diabetes medications and other issues, such as side effects, which would be relevant to patient decision-making about the medications. The survey also included five 7-point Likert-type scales to assess patient acceptability of the information. Provider acceptability of the tool was also assessed at the completion of the trial. Medication adherence was measured using both self-report and pharmacy records. Self-reported medication adherence was assessed using the following single question: “People often have difficulty taking their pills for one reason or another. How many times do you think you may have missed taking your pills in the last week?”21 Medication adherence based on pharmacy records was assessed by calculating the proportion of days covered.22
Summary of Results
A total of 21 providers were randomized to the intervention arm and delivered decision aids to 48 patients whereas 19 providers were randomized to provide usual care to 37 patients. Descriptive baseline statistics for the enrolled patients are presented in Table 1. Most patients were well educated, had diabetes for about 8 years, and had a hemoglobin A1c less than 8%.
Both patients and physicians found the decision aid to be acceptable for translation of comparative effectiveness into practice. Patients found that the decision aid provided them with adequate amount of information, and the information was clear to that patient (Table 2). The information provided on the decision aid was clear and most would prefer to use this method for receiving information for other treatment decisions. Patients were also satisfied with the shared-decision making approach for making treatment decisions.
The patients in the decision aid arm had more knowledge overall about the treatment options (adjusted mean difference in overall knowledge: 1.10 [95% CI: 0.11–2.09]). Patients had better knowledge about the comparable effectiveness and the associated side effect profile of the diabetes medications (Table 2).
Approximately 33% of the patients decided to initiate a new treatment in the decision aid arm, whereas 22% of the patients decided to initiate a new treatment in the usual care arm (Fig. 2). There were also some differences in the choice of agents between the 2 groups. Most patients in the both arms of the trials continued their existing regimen at the same or increased dose.
On the basis of self-report, there were no significant differences in medication adherence between the 2 arms of the study. Twenty-four percent of the patients in the decision aid arm reported missing a dose of their medication in the last week while 19% in the usual care arm reported missing a dose in the last week. Based on pharmacy records, we found the median proportion of days covered was 100% in both arms of the study. However, due to variability, the adjusted mean difference was significant between the 2 arms and favored the control arm.
Overview of a Large Practical Randomized Trial to Evaluate the Value of Decision Aids for Translation of CER
Although our preliminary testing of the decision aid provided evidence of efficacious translation into clinical practice and limited to support to improvement in patient outcomes or adherence, it did not assess long-term effects on patient outcomes and challenges for implementation in real-world practices. Specifically, we do not know how patients translate this knowledge into behaviors, such as adherence, and whether this affects short- and long-term health outcomes. In addition, we do not know whether primary care practices would adopt such a tool or the amount of effort that would be required for primary care practitioners to integrate this tool into their practice. To that end, we have recently embarked on 2 large practical randomized trials to answer these questions. These trials will be conducted in 28 community-based practices in Midwestern United States, recruit approximately 1000 patients and will assess how these clinics are able to uptake this intervention. In addition, these trials will assess the effects this tool has on the patient, and whether it leads to improvements in the ultimate health outcomes. Specifically, it is unclear whether presenting research evidence in a patient-centered manner, such that patient preferences and context are considered for decision-making, will improve patient outcomes. These trials will help us better understand the value of patient-centered translation of CER.
In our pilot studies, patients in the decision aid arm had better knowledge about their treatment options, and both patients and providers report use of this decision aid to be acceptable. However, very few patients initiated new treatments and thus, there was limited evidence of favorable effect of the use of the decision aid for improving outcomes.
The Federal Coordinating Council for Comparative Effectiveness Research states that the purpose of CER is to, “… to provide information that helps clinicians and patients choose the options that best fit the individual patient's needs and preferences.”23 In addition, in its annual report to the President, the Council states that many results from CER have not been translated into practice or made accessible to the patients due to a lack of tools and mechanisms for delivering this information to the front line of clinical care.
Our pilot trial for diabetes medications suggests that decisions aids may be valuable tools for translating CER into practice. This tool translates CER for front-line clinical care where patients needs, context, and preferences can be incorporated into decision-making. However, the main outcome in this study, adherence, was not different between the 2 groups. This negative finding may be due to a number of reasons including having a highly educated population in the study, relatively health population, or due to a small proportion of patients deciding to initiate new treatments in the study. Alternatively, decision aids may not improve adherence or other clinical outcomes; however, larger trials may be necessary to better understand the effect of decision aids for translating CER.
We learnt a number of important lessons from this pilot trial. First, our iterative process of designing the decision aid suggested that a “less is more” approach may be more useful. Patients and clinicians seem to find a decision aid that highlighted summary information more valuable than one in which there was detailed information. Second, the use of decision aids may be more useful in patients who have to make a decision about initiating a new agent. Our analyses were limited since only one-third of the patients in the intervention group and 22% of the patients in the control group initiated a new agent. This factor will remain a nagging concern for sample size estimation in these trials, since power to address the adherence outcome depends on the number of patients who choose to initiate the index drug or behavior. Third, it is unclear whether the decision aid may be more valuable for less educated population or if there are subgroups for whom the decision aids could be more valuable. Fourth, “learning by doing” for decision aids may be important for both patients and clinicians. Many of the clinicians in the study used the decision aid only once, and all patients had only the one experience with the decision aid. It would be important to better understand the role of repeated use of decision aids from both the patient and clinician perspective. Finally, the use of decision aids in the clinical encounter was acceptable for both patients and clinicians.
This trial also highlighted a number of areas for future work in better understanding the role of decision aids in translating CER. A main area of future research needs to focus on the medium through which decision aids are designed as well as the mode through which they are administered. For example, does a self-administered DVD-based decision aid work similarly well for translating CER compared with clinician administered paper decision aid? A second main area of future work needs to focus on understanding the barriers to implementing decision aids in busy primary care practices, especially where clinicians may need to administer the decision aid. It will also be important to understand the overall economic effect of the use of decision aids for translating CER.
CER will require a robust research effort focused on the translation of CER into practice for CER to realize its goals. For this to happen, rather than limiting CER results to inform decisions at the payer level, we put forth that CER data could be translated for use at the clinical encounter level to inform patient and clinician choice. This application requires research investment in resources that can develop innovative and time-sensitive decision technologies, such as decision aids. These resources would include designers, patient advisory groups, and clinicians invested in CER translation. The identification of patients who would need to use these tools with their clinicians would require robust health information technology infrastructure for their just-in-time identification. The use of these tools with patients would require training of clinicians and alignment of quality measures, economic incentives, and legal standards to promote conversations in which CER decision aids are used rather than usual care and cursory informed consent. A cautionary note: shared decision-making may not uniformly reduce costs of care as the costs of care would result from the choice of treatment that patients and clinicians would make. As we are continuing our work in this area, we will explore whether decision aids yield benefits to the patient, the practice, and the system beyond satisfying a fundamental ethical expectation of modern medical care: that a patient will receive treatments that are consistent not only with the best available evidence but also with the context, values, and preferences of the patient.
1. Orszag PR. Research on the Comparative Effectiveness of Medical Treatments: Options for an Expanded Federal Role
. Washington, DC: Congressional Budget Office; 2007.
2. Woolf SH. The meaning of translational research and why it matters. JAMA
3. Westfall JM, Mold J, Fagnan L. Practice-based research–“Blue Highways” on the NIH roadmap. JAMA
4. Lenfant C. Clinical research to clinical practice–lost in translation
? N Engl J Med
5. Dougherty D, Conway PH. The “3T's” road map to transform US health care: the “how” of high-quality care. JAMA
6. Nathan DM. Finding new treatments for diabetes—How many, how fast. How good? N Engl J Med
7. Grant RW, Wexler DJ, Watson AJ, et al. How doctors choose medications to treat type 2 diabetes: a national survey of specialists and academic generalists. Diabetes Care
8. Hoerger TJ, Gregg EW, Segel JE, et al. Is glycemic control improving in US adults? Diabetes Care
9. Karter AJ, Moffet HH, Liu J, et al. Achieving good glycemic control: Initiation of new antihyperglycemic therapies in patients with type 2 diabetes from the Kaiser Permanente Northern California Diabetes Registry. Am J Manag Care
10. Saaddine JB, Engelgau MM, Beckles GL, et al. A diabetes report card for the United States: quality of care in the 1990s. Ann Intern Med
11. Schmittdiel JA, Uratsu CS, Karter AJ, et al. Why don't diabetes patients achieve recommended risk factor targets? Poor adherence versus lack of treatment intensification. J Gen Intern Med
12. Bolen S, Feldman L, Vassy J, et al. Systematic review: comparative effectiveness
and safety of oral medications for type 2 diabetes mellitus. Ann Intern Med
13. Karter AJ, Moffet HH, Liu J, et al. Glycemic response to newly initiated diabetes therapies. Am J Manag Care
14. Montori V, Gafni A, Charles C. A shared treatment decision-making approach between patients with chronic conditions and their clinicians: the case of diabetes. Health Expect
15. Weingarten SR, Henning JM, Badamgarav E, et al. Interventions used in disease management programmes for patients with chronic illness-which ones work? Meta-analysis of published reports. BMJ
16. O'Connor AM, Mulley AG Jr, Wennberg JE. Standard consultations are not enough to ensure decision quality regarding preference-sensitive options. J Natl Cancer Inst
17. Clancy CM. Perspective—getting to “smart” health care. Health Aff
18. Montori VM, Breslin M, Maleska M, et al. Creating a conversation: insights from the development of a decision aid. PLoS Med
19. Breslin M, Mullan RJ, Montori VM. The design of a decision aid about diabetes medications
for use during the consultation with patients with type 2 diabetes. Patient Educ Couns
20. Mullan RJ, Montori VM, Shah ND, et al. The diabetes mellitus medication choice decision aid: a randomized trial. Arch Intern Med
21. Haynes RB, McDonald HP, Garg AX. Helping patients follow prescribed treatment: clinical applications. JAMA
22. Hess LM, Raebel MA, Conner DA, et al. Measurement of adherence in pharmacy administrative databases: a proposal for standard definitions and preferred measures. Ann Pharmacother
23. Conway PH, Clancy C. Comparative-effectiveness research–implications of the Federal Coordinating Council's report. N Engl J Med